Data Export Test Report (Nolana)2023/01/10
Overview
In this workflow, we are checking the performance of exporting MARC Bib records workflow running in the Nolana release - https://folio-org.atlassian.net/browse/PERF-345
We tested with 1 virtual user for 1000, 100K, 200k, and 500K records.
Summary
Data Export is stable for all test runs up to 500K Instance records.
No issues were observed for even exporting 500K Instance records.
Data Export time is about 2 times decreased compared to the KIWI release probably because of the services have a lot more CPU and RAM allocated. For further investigationhttps://folio-org.atlassian.net/browse/PERF-411
Test Runs
Job profile -"srs - holdings and items"
Test performed from JMeter. Test name "dataExport_exportMARCBibRecords_nolana" from Carrier.io export bucket. (For all 4 tests allocated time is 4h by default, with 2 min pauses between each test)
Test | Total instances |
1. | 1000 |
2. | 100,000 |
3. | 200,000 |
4. | 500,000 |
Test Results with Comparisons
Test | Total instances | 1 User - Avg Total time to Export instances (Nolana) | 1 User - Avg Total time to Export instances (KIWI) |
1. | 1000 | 20 seconds | 1 minute |
2. | 100,000 | 35 minutes | 1 hour 8 minutes |
3 | 200,000 | 1 hour 15 minutes | - |
4. | 500,000 | 3 hours 1 minute | 5 hours 48 minutes |
Service Memory Utilization
Service Memory Utilization is stable.
CPU Utilization
CPU utilization is stable. Services` CPU utilization did not exceed 13% for 100, 100K, and 500K records Data Exports with the job profile -"srs - holdings and items".
RDS CPU Utilization
Infrastructure
PTF -environment ncp3
9 m6i.2xlarge EC2 instances located in US East (N. Virginia)us-east-1
2 instances of db.r6.xlarge database instances, one reader, and one writer
MSK ptf-kakfa-3
4 m5.2xlarge brokers in 2 zones
Apache Kafka version 2.8.0
EBS storage volume per broker 300 GiB
auto.create.topics.enable=true
log.retention.minutes=480
default.replication.factor=3
Modules memory and CPU parameters
Modules | Version | Task Definition | Running Tasks | CPU | Memory | MemoryReservation | MaxMetaspaceSize | Xmx |
|---|---|---|---|---|---|---|---|---|
mod-inventory-storage | 25.0.3 | 3 | 2 | 1024 (128 in KIWI) | 2208 (864 in KIWI) | 1952 (536 in KIWI) | 512m | 1440m |
okapi | 4.14.7 | 1 | 3 | 1024 | 1684 (1512 in MG) | 1440 (1360 in MG) | 512m | 922m |
mod-authtoken | 2.12.0 | 3 | 2 | 512 (128 in MG) | 1440 (1024 in MG) | 1152 (896 in MG) | 128 | 922 (768 in MG) |
mod-source-record-storage | 5.5.2 | 4 | 2 | 1024 (128 in KIWI) | 1536 (1440 in KIWI) | 1440 (896 in KIWI) | 512m | 908m |
mod-data-export | 4.6.1 | 2 | 1 | 1024 (128 in KIWI) | 1024 (512 in KIWI) | 896 (360 in KIWI) | 128m | 768m |
mod-source-record-manager | 3.5.6 | 4 | 2 | 1024 | 4096 (2048 in MG) | 3688 (1844 in MG) | 512m (800M in MG) | 2048m (1024m in MG) |
nginx-okapi | 2022.03.02 | 1 | 2 | 128 | 1024 | 896 | - | - |
KIWI- KIWI release
MG- Morning Glory release
Front End:
Data export manager (folio_data-export-5.3.1)
Data Export CSV files used to run test
All instance records in the below files are source=MARC