Table of Contents outline true
...
| Jira Legacy | ||||||||||
|---|---|---|---|---|---|---|---|---|---|---|
|
Summary
- Data import tests finished successfully, only Test №5 had one failed record for Tenant 2(qcp1-01) when processed 50k files. Duration of DI grew in correspondence with the number of records in files.
- Check-in and Check-out with 5 virtual users was performed during DI Create new MARC authority records jobs for non-matches No issues.
- Data Import in Quesnelia without CICO perform faster than with it.
- Comparing Poppy and Quesnelia releases
- Check-in / Check-out perform better in Quesnelia. Response time improved during Create jobs for long period of work time on 15% in Avarage.
- DI durations improved - 11%-14% in Average.
...
- During testing, we noticed spikes in the mod permissions module. To mitigate this issue and prevent system slowdowns, we adjusted the order of loading files, starting with Tenant 3 (qcp1-02)
...
- , followed by Tenant 2 (qcp1-01)
...
- , and finally Tenant 1 (qcp1-00)
...
- .
Test Results and Comparison
...
- Сomparative Baseline Check-In\Check-Out results without Data Import between Poppy and Quesnelia.
CICO, Median time without | CICO, 95% time without DI (Poppy) | CICO, Median time without DI (Quesnelia) | CICO, 95% time without DI (Quesnelia) | CICO, Avg time without DI (Quesnelia) | |
|---|---|---|---|---|---|
| Check-In | 516 ms | 567 ms | 503 ms -2.5% | 593 ms | 511 ms |
| Check-Out | 910 ms | 2094 ms | 836 ms -8% | 1117 ms -46% | 876 ms |
...
- Сomparative Data Import and Check-In\Check-Out results between Poppy and Quesnelia.
# of records | DI Duration with CICO | CI time Avg | CI time 95th pct | CO time Avg | CO time 95th pct | # of records (Quesnelia) | DI Duration with CICO | CI time Avg (Quesnelia) | CI time 95th pct (Quesnelia) | CO time Avg (Quesnelia) | CO time 95th pct (Quesnelia) |
|---|---|---|---|---|---|---|---|---|---|---|---|
| 1,000 | 35 sec | 0.525 | 0.576 | 1.078 | 1.326 | 1,000 | 20 sec | 0.560 +6% | 0.754 +30% | 1.164 +8% | 1.313 -1% |
| 5,000 | 1 min 41 sec | 0.513 | 0.612 | 0.9 | 1.019 | 5,000 | 1 min 19 sec -21.7% | 0.701 +36% | 1.171 +91% | 1.141 +26% | 1.790 +75% |
| 10,000 | 3 min 4 sec | 0.581 | 0.685 | 1.016 | 1.321 | 10,000 | 2 min 35 sec -15.7% | 0.723 +24% | 1.024 +49% | 1.179 +16% | 1.494 +13% |
| 22,778 | 6 min 32 sec | 0.598 | 1.542 | 1.244 | 1.729 | 25,000 | 6 min 26 sec -1.5% | 0.722 +20% | 1.024 -33% | 1.180 -5% | 1.494 -13% |
| 50,000 | 13 min 48 sec | 0.671 | 1.953 | 1.51 | 2.09 | 50,000 | 12 min 16 sec | 0.777 +15% | 1.045 -46% | 1.265 | 1.550 -25% |
...
PTF - environment Quesnelia (qcp1)
- 10 db.r6g.xlarge EC2 instances located in US East (N. Virginia)us-east-1
1 database instances, writer
Name Memory GIB vCPUs db.r6g.xlarge
32 GiB 4 vCPUs - MSK ptf-mobius-testing2
- 4 m5.2xlarge brokers in 2 zones
Apache Kafka version 2.8.0
EBS storage volume per broker 300 GiB
- auto.create.topics.enable=true
- log.retention.minutes=480
- default.replication.factor=3
...
- The above files are all stored here - MARC Resources
- 22k file what was provided from MARC Resources does nor work, so 50k file was split to file with 25k records and used instead of 22k file. - At the time of the test run, Grafana was not available. As a result, response times for Check-In/Check-Out were parsed manually from a .jtl file, using the start and finish dates of the data import tests. These results were visualized in JMeter using a Listener (Response Times Over Time).
Test set
- Test 1: Manually tested 1k, 10k, 25k and 50k records files DI started on one tenant only.
- Test 2: Manually tested 1k, 10k, 25k and 50k records files DI started on one tenant only plus Check-in and Checkout (CICO) for 5 concurrent users.
- Test 3: Manually tested 1k, 10k, 25k and 50k records files DI started on 3 tenants concurrently. Order for load file without pause between files: 50k, 25k, 10k, 5k, and 1k for order tenants : Tenant 3(qcp1-02), Tenant 2(qcp1-01) and Tenant 1(qcp1-00)
- Test 4: Manually tested 1k, 10k, 25k and 50k records files DI started on 3 tenants concurrently. Order for load file with pause between files: 50k, 25k, 10k, 5k, and 1k for order tenants : Tenant 3(qcp1-02), Tenant 1(qcp1-00) and Tenant 2(qcp1-01)
- Test 5: Manually tested 1k, 10k, 25k and 50k records files DI started on 3 tenants concurrently. Order for load file without pause between files: 1k, 5k, 10k, 25k and 50k for order tenants : Tenant 3(qcp1-02), Tenant 2(qcp1-01) and Tenant 1(qcp1-00)
...