Table of Contents outline true
...
- Data Export tests finished successfully on the Eureka environment using the "Default instances export job profile" and "srs - holdings and items job profile." Data Export tests were run on Central tenants.
- During Test №1-2When comparing QECP1 Eureka with Quesnelia ECS, NON-ECS environments:
- Data Export duration for the "Default instances export job profile" was almost the same across environments.
- Data Export duration for the "srs - holdings and items job profile" showed a significant degradation: around 2.5 times longer for the 1k and 100k files, and around 5 times longer for the 500k file, and no clear reasons were identified.
- During Tests №1 and №2, we noticed some background processes for Database, so we deactivated
mod-fqm-managerdeactivatedmod-fqm-manager. As a result, the repeated test showed a twofold improvement in performance. - Data Export jobs were getting stuck and returned a [401 Unauthorized] error with the message:
"errors [{"type":"UnauthorizedException","code":"authorization_error","message":"Unauthorized"}]. To resolve this, we increased the token lifespan by modifying theKC_CONFIG_TTLparameter formod-login-keycloakfrom 360s to 3600s andKC_ADMIN_TOKEN_TTLfrom 410s to 4100s:{"name": "KC_CONFIG_TTL","value": "3600s"}{"name": "KC_ADMIN_TOKEN_TTL", "value": "4100s"}
- Data Export duration for the "Default instances export job profile" was almost the same across environments.
- Data Export duration for the "srs - holdings and items job profile" showed a significant degradation: around 2.5 times longer for the 1k and 100k files, and around 5 times longer for the 500k file.
Test Results
This table contains durations for Data Export with 2 job profiles.
Profile | CSV File | Central Tenant (fs09000000) | ||
|---|---|---|---|---|
| Result Test Set 1 | Result Test Set 2 | Status | ||
| DE MARC Bib (Default instances export job profile) | 1k.csv | 0:00:07 | 0:00:03 | COMPLETED |
| 100k.csv | 0:06:03 | 0:02:17 | COMPLETED | |
| 500k.csv | 0:09:03 | 0:04:25 | COMPLETED | |
| DE MARC Bib (srs - holdings and items) | 1k.csv | 0:00:13 | 0:00:10 | COMPLETED |
| 100k.csv | 0:20:25 | 0:12:41 | COMPLETED | |
| 500k.csv | 0:38:21 | 0:38:53 | COMPLETED | |
...
DB connections was 830.
Kafka metrics
OpenSearch Data Nodes metrics
...
DB connections was 850.
Kafka metrics
OpenSearch Data Nodes metrics
...
11 m6i.2xlarge EC2 instances located in US East (N. Virginia)us-east-1 [Number of ECS instances, instance type, location region]
1 instance of db.r6.xlarge database instance: Writer instance
OpenSearch
domain: ptf-test
Number of nodes: 7
Version: OpenSearch_2_13_R20240520-P5
MSK - fse-tenant
4 kafka.m7g.xlarge brokers in 2 zones
Apache Kafka version 3.7.x
EBS storage volume per broker 300 GiB
auto.create.topics.enable=true
log.retention.minutes=480
default.replication.factor=3
...
- Test 1: Manually tested 1k, 100k and 500k records files Data Export started on Main tenant(fs09000000) using Default instances export job profile.
- Test 2: Manually tested 1k, 100k and 500k records files Data Export started on Main tenant(fs09000000) using srs - holdings and items job profile.
- Test 3: Set Instance Count=0 for mod-fqm-manager. Manually tested 1k, 100k and 500k records files Data Export started on Main tenant(fs09000000) using Default instances export job profile.
- Test 4: Set Instance Count=0 for mod-fqm-manager. Manually tested 1k, 100k and 500k records files Data Export started on Main tenant(fs09000000) using srs - holdings and items job profile.
...