Overview
- This document contains the results of testing Data Import for MARC Bibliographic records with an update job on the Quesnelia [ECS] release on qcon environment.
PERF-846 - Getting issue details... STATUS
Summary
- Data Import tests finished successfully on qcon environment using the PTF - Updates Success - 2 profile and file with 25k records.
- Comparing with previous testing results Poppy and Quesnelia releases
- Data Import processed all jobs including test on 3 tenants concurrently without errors for Quesnelia releases.
- Data Import durations stayed in the same time range in Average for Quesnelia releases but it works stable and without errors.
- During testing, we noticed that mod-permission did not have any spikes and used 12% CPU for Quesnelia releases. For Poppy releases we had error.
Test Runs
Test № | Date | Test Conditions | Results |
1. | Mar 4, 2024, 5:42 PM UTC | 1 CI or CO per min for small tenants, 2 CICO per min for large tenants | Completed |
| 2. | Mar 4, 2024, 8:19 PM UTC | 2 CI or CO per min for small tenants, 4 CICO per min for large tenants | Completed |
| 3. | Mar 5, 2024, 2:39 PM UTC | 4 CICO per min for one large tenant | Completed |
Test Results
This table contains durations for Data Import.
| Profile | Test # | Tenant | MARC File | DI Duration Quesnelia (hh:mm:ss) | Results |
|---|---|---|---|---|---|
| DI MARC Bib Update (PTF - Updates Success - 2) | 1 | College | 25K.mrc | 0:17:43 | Completed |
| 2 | College (cs00000int_0001) | 25K.mrc | 0:26:04 | Completed | |
| Professional (cs00000int_0002) | 25K.mrc | 0:25:50 | Completed | ||
| 3 | College (cs00000int_0001) | 25K.mrc | 0:26:46 | Completed | |
| Professional (cs00000int_0002) | 25K.mrc | 0:38:06 | Completed | ||
| School (cs00000int_0003) | 25K.mrc | 0:37:29 | Completed |
Comparison
This table contains durations comparison between Poppy and Quesnelia releases.
| Profile | MARC File | Test run # | DI Duration | DI Delta (hh:mm:ss) | |
|---|---|---|---|---|---|
| Poppy | Quesnelia | ||||
| DI MARC Bib Update (PTF - Updates Success - 2) | 25K.mrc | 1 | 0:17:18 | 0:17:43 | +00:00:25 +2.41% |
| 2 | 0:56:41 | 0:51:54 | -00:04:47 -8.44% | ||
| 3 | 1:33:48 with errors* | 1:42:21 | +00:08:33 +9.12% | ||
Resource utilization for Test №1
Service CPU Utilization
Here we can see that mod-data-import used 150% CPU in spikes.
Service Memory Utilization
Here we can see that all modules show a stable trend.
DB CPU Utilization
DB CPU was 92%.
DB Connections
Max number of DB connections was 1690.
DB load
Top SQL-queries
| # | TOP 5 SQL statements |
|---|---|
| 1 | |
| 2 | |
| 3 | |
| 4 | |
| 5 |
Resource utilization for Test №2
Service CPU Utilization
Here we can see that mod-data-import used 130% CPU in spikes.
Service Memory Utilization
Here we can see that all modules show a stable trend.
DB CPU Utilization
DB CPU was 92%.
DB Connections
Max number of DB connections was 1685.
DB load
Top SQL-queries
| # | TOP 5 SQL statements |
|---|---|
| 1 | |
| 2 | |
| 3 | |
| 4 | |
| 5 |
Appendix
Infrastructure
PTF - environment Quesnelia (qcon)
11 m6i.2xlarge EC2 instances located in US East (N. Virginia)us-east-1 [Number of ECS instances, instance type, location region]
1 instance of db.r6.xlarge database instance: Writer instance
OpenSearch
domain: fse
Number of nodes: 9
Version: OpenSearch_2_7_R20240502
MSK - tenat
4 kafka.m5.2xlarge brokers in 2 zones
Apache Kafka version 2.8.0
EBS storage volume per broker 300 GiB
auto.create.topics.enable=true
log.retention.minutes=480
default.replication.factor=3
Kafka consolidated topics enabled
Methodology/Approach
DI tests scenario(DI MARC Bib Update) were started from UI.
Test set:
- Test 1: Manually tested 25k records files DI started on College tenant(cs00000int_0001) only.
- Test 2: Manually tested 25k records files DI started on College tenant(cs00000int_0001) and Professional tenant(cs00000int_0002) concurrently, step 30%.
- Test 3: Manually tested 25k records files DI started on College tenant(cs00000int_0001), Professional tenant(cs00000int_0002) and School tenant(cs00000int_0003) concurrently, step 30%.
To get status and time range for import jobs the query used:
select file_name,started_date,completed_date, completed_date - started_date as duration ,status from cs00000int_0001_mod_source_record_manager.job_execution order by started_date desc limit 1000; select file_name,started_date,completed_date, completed_date - started_date as duration ,status from cs00000int_0002_mod_source_record_manager.job_execution order by started_date desc limit 1000; select file_name,started_date,completed_date, completed_date - started_date as duration ,status from cs00000int_0003_mod_source_record_manager.job_execution order by started_date desc limit 1000;