Table of Contents outline true
...
Profile | CSV File | Central Tenant (fs09000000) | ||
|---|---|---|---|---|
| Result Test Set 1 | Result Test Set 2 | Status | ||
| DE MARC Bib (Default instances export job profile) | 1k.csv | 0:00:07 | 0:00:03 | COMPLETED |
| 100k.csv | 0:06:03 | 0:02:17 | COMPLETED | |
| 500k.csv | 0:09:03 | 0:04:25 | COMPLETED | |
| DE MARC Bib (srs - holdings and items) | 1k.csv | 0:00:13 | 0:00:10 | COMPLETED |
| 100k.csv | 0:20:25 | 0:12:41 | COMPLETED | |
| 500k.csv | 0:38:21 | 0:38:53 | COMPLETED | |
...
Test №1 - №2
Introduction: The Baseline QECP1 Environment configuration was applied, and CPU=0 was set for all modules.
Objective: The objective of these tests was to collect performance measurements for the data-export process across central tenant.
Results: During the test, we observed that the database was running a background process . for Database, so we deactivated mod-fqm-manager. As a result, the repeated test showed a twofold improvement in performance.
Service CPU Utilization
Here we can see that mod-data-export used 28% CPU Instance Power in spike.
...
DB connections was 830.
Kafka metrics
OpenSearch Data Nodes metrics
...
DB connections was 850.
Kafka metrics
OpenSearch Data Nodes metrics
...
11 m6i.2xlarge EC2 instances located in US East (N. Virginia)us-east-1 [Number of ECS instances, instance type, location region]
1 instance of db.r6.xlarge database instance: Writer instance
OpenSearch
domain: ptf-test
Number of nodes: 7
Version: OpenSearch_2_13_R20240520-P5
MSK - fse-tenant
4 kafka.m7g.xlarge brokers in 2 zones
Apache Kafka version 3.7.x
EBS storage volume per broker 300 GiB
auto.create.topics.enable=true
log.retention.minutes=480
default.replication.factor=3
...
- Test 1: Manually tested 1k, 100k and 500k records files Data Export started on Main tenant(fs09000000) using Default instances export job profile.
- Test 2: Manually tested 1k, 100k and 500k records files Data Export started on Main tenant(fs09000000) using srs - holdings and items job profile.
- Test 3: Set Instance Count=0 for mod-fqm-manager. Manually tested 1k, 100k and 500k records files Data Export started on Main tenant(fs09000000) using Default instances export job profile.
- Test 4: Set Instance Count=0 for mod-fqm-manager. Manually tested 1k, 100k and 500k records files Data Export started on Main tenant(fs09000000) using srs - holdings and items job profile.
...