Data Import MARC BIB + CI/CO (Ramsons) [ECS]
- 1 Overview
- 2 Summary
- 3 Test Runs
- 4 Test Results
- 5 Comparison
- 6 Test №1
- 6.1 Response times
- 6.2 Cluster resource utilization
- 6.3 DB resources utilization
- 6.3.1 RDS CPU Utilizatoin
- 6.3.2 DB Connections
- 6.3.3 DB load
- 6.3.4 Top SQL-queries
- 6.3.5 Top applications
- 6.4 MSK resources utilization
- 7 Test №2
- 7.1 Response times
- 7.2 Cluster resource utilization
- 7.3 DB resources utilization
- 7.3.1 RDS CPU Utilization
- 7.3.2 DB Connections
- 7.3.3 DB load
- 7.3.4 Top SQL-queries
- 7.4 MSK resources utilization
- 8 Appendix
- 8.1 Infrastructure
- 9 Methodology/Approach
Overview
This document contains the results of testing Check-in/Check-out and Data Import for MARC Bibliographic records on the Ramsons[ECS] release environment.
https://folio-org.atlassian.net/browse/PERF-978
Summary
Data Import (DI) with Check In/Check Out tests finished successfully with PTF - Create 2 and PTF - Updates Success - 6 job profiles with files 5K, 10K, 25K, 50K, 100K records.
Comparison results:
DI create jobs duration with CI/CO degraded 25% in average. There's degradation without CI/CO 22% with 10k and 8% with 25k but there's improvement 14% with 50k and 17% with 100k.
DI update jobs durations with CI/CO degraded 20% for 25k, 50k, 100k and 50% for 5k, 10k. There's degradation without CI/CO 20% with 10k, 3% with 50k, 12% with 100k but there's 4% improvement with 25k.
DI with CI/CO degraded compared to DI without CI/CO. DI create jobs - 70%, DI update jobs - 30% in average.
CI/CO response times degraded with DI 20% in average.
DI create job with 100k file finished successfully but total duration in database was updated in 6 hours after it was finished. Additional 100k finished in 56 minutes (total duration on db side).
Memory showed some memory growing trend for mod-pubsub and it's possible that a memory leak has been detected on a module side.
Recommendations & Jiras
The previous results report:
The mod-pubsub module memory leak investigation ticket: https://folio-org.atlassian.net/browse/MODPUBSUB-311
New behaviour from the
mod_orders_storage.po_line table detected during data import create jobs. Consider also [tenant]_mod_orders_storage.internal_lock query. It should be investigated.
Test Runs
Test № | Scenario | Test Conditions | Results |
|---|---|---|---|
| DI MARC Bib Create | 5K, 10K, 25K, 50K, 100K consequentially (with 5 min pause) |
|
CICO | 8 users | ||
| DI MARC Bib Update | 5K, 10K, 25K, 50K, 100K consequentially (with 5 min pause) |
|
CICO | 8 users |
Test Results
This table contains durations for Data Import.
Profile | MARC File | DI Duration Ramsons (hh:mm:ss) | Check In, Check Out Response time (8 users) Ramsons | |
|---|---|---|---|---|
CI Average, sec | CO Average, sec | |||
DI MARC Bib Create (PTF - Create 2) | 5K | 0:04:11 | 1.01 | 2 |
10K | 0:06:39 | 0.95 | 1.88 | |
25K | 0:16:13 | 1.18 | 2.25 | |
50K | 0:29:47 | 1.11 | 2.29 | |
100K | 0:56:00 | 1.6 | 2.4 | |
DI MARC Bib Update (PTF - Updates Success - 6) | 5K | 0:06:19 | 0.99 | 2.2 |
10K | 0:12:10 | 1.1 | 2.5 | |
25K | 0:24:31 | 1.04 | 2.1 | |
50K | 0:49:53 | 1.03 | 2.2 | |
100K | 1:48:00 | 1 | 2.1 | |
Check-in/Check-out without DI
Scenario | Load level | Request | Response time, sec | |
|---|---|---|---|---|
95 perc | average | |||
Circulation Check-in/Check-out | 8 users | Check-in | 1 | 0.65 |
Check-out | 1.9 | 1.2 | ||
Comparison
This table contains DI durations with CICO comparison between Quesnelia and Ramsons releases.
Profile | MARC File | DI Duration | DI Delta without CI/CO, % | DI Delta with CI/CO, % | DI Delta Ramsons/Ramsons without/with CI/CO, % | Check In, Check Out | Check In, Check Out | Delta, % | ||||||
without CI/CO | with CI/CO | Quesnelia | Ramsons | Quesnelia/Ramsons | Quesnelia/Ramsons | |||||||||
Quesnelia | Ramsons | Quesnelia | Ramsons | CI Average sec | CO Average sec | CI Average sec | CO Average sec | CI | CO | |||||
DI MARC Bib Create (PTF - Create 2) | 5K.mrc |
| 00:04:11 | 00:03:21 | 00:04:11 |
| 24.88% | 0.00% | 0.831 | 1.357 | 1.01 | 2 | 21.54% | 47.38% |
10K.mrc | 00:04:14 | 00:05:10 | 00:06:51 | 00:06:39 | 22.05% | -2.92% | 28.71% | 0.845 | 1.41 | 0.95 | 1.88 | 12.43% | 33.33% | |
25K.mrc | 00:09:41 | 00:10:30 | 00:12:41 | 00:16:13 | 8.43% | 27.86% | 54.44% | 0.719 | 1.333 | 1.18 | 2.25 | 64.12% | 68.79% | |
50K.mrc | 00:18:18 | 00:15:43 | 00:23:19 | 00:29:47 | -14.12% | 27.73% | 89.50% | 0.691 | 1.327 | 1.11 | 2.29 | 60.64% | 72.57% | |
100K.mrc | 00:38:36 | 00:31:51 | 00:51:24 | 00:56:00 | -17.49% | 8.95% | 75.82% | 0.664 | 1.335 | 1.6 | 2.4 | 140.96% | 79.78% | |
DI MARC Bib Update (PTF - Updates Success - 6) | 5K.mrc |
|
| 00:04:12 | 00:06:19 |
| 50.40% |
| 0.764 | 1.458 | 0.99 | 2.2 | 29.58% | 50.89% |
10K.mrc | 00:05:59 | 00:07:10 | 00:08:15 | 00:12:10 | 19.78% | 47.47% | 69.77% | 0.779 | 1.377 | 1.1 | 2.5 | 41.21% | 81.55% | |
25K.mrc | 00:19:52 | 00:19:03 | 00:20:38 | 00:24:31 | -4.11% | 18.82% | 28.70% | 0.755 | 1.401 | 1.04 | 2.1 | 37.75% | 49.89% | |
50K.mrc | 00:37:53 | 00:38:53 | 00:43:06 | 00:49:53 | 2.64% | 15.74% | 28.29% | 0.75 | 1.444 | 1.03 | 2.2 | 37.33% | 52.35% | |
100K.mrc | 01:14:00 | 01:23:00 | 01:29:09 | 01:48:00 | 12.16% | 21.14% | 30.12% | 0.73 | 1.458 | 1 | 2.1 | 36.99% | 44.03% | |
Detailed CICO response time comparison without DI
Scenario | Load level | Request | Response time, sec | Response time, sec | ||
|---|---|---|---|---|---|---|
95 perc | average | 95 perc | average | |||
Circulation Check-in/Check-out (without Data import) | 8 users | Check-in | 0.64 | 0.49 | 1 | 0.65 |
Check-out | 1.24 | 1.08 | 1.9 | 1.2 | ||
Test №1
Response times
