Holdings with High Item Counts Performance Impact Analysis
- 1 Overview
- 2 Summary
- 3 Test runs
- 4 Results
- 4.1 Response times
- 4.2 Comparison tables
- 5 Resource utilization
- 5.1 Instance CPU Utilization
- 5.2 CPU Utilization
- 5.3 Memory Utilization
- 5.4 DB CPU Utilization
- 5.5 DB Connections
- 5.5.1 Single Holdings Editing
- 5.5.2 Continuous Holdings Editing
- 5.5.3 DB Load
- 5.6 OpenSearch
- 5.6.1 Data Nodes
- 5.6.2 Indexing Data Rate
- 5.6.3 Search rate
- 6 Appendix
- 6.1 Infrastructure
- 6.1.1 Infrastructure
- 6.2 Methodology
- 6.1 Infrastructure
Overview
Performance Testing Report Overview
This document presents the results of volume testing for Holding records with different item counts.
It includes the results of the impact on other LoC workflows response times when editing holdings records with 10K items attaching to them.
Ticket: https://folio-org.atlassian.net/browse/PERF-1289
Summary
Volume tests show that Holdings editing response time grows significantly after 1200 items (4,7 sec). 2000 items took almost 13 seconds, 10000 items ~ 1 minute.
Test editing a single Holdings record while all master script flows running in the background:
No observable LoC workflows response times degradation while editing of a Holdings record with 10k items
Editing holdings records with 10k under average LoC load increases the editing response time by 30%, rising from 60 seconds to 78 seconds.
No crashes or module restarts during tests observed.
CPU utilized in accordance with load.
No memory issues happened.
Test editing a Holdings record continuously while all other LoC flows are running in the background (not a realistic scenario)
Result: measurable degradation in 218 out of 365 transactions (59.7%). The impact is concentrated in the Finance/Ledger and ListApp operations because of contention on shared backend services like mod-inventory-storage.
Test runs
Volume testing experiments (only tested editing individual holdings records with various number of items attaching to them):
Test # | Item count | Tenant |
|---|---|---|
T1 | 700 | cs00000_int_0002 |
T2 | 1001 | cs00000_int_0002 |
T3 | 1230 | cs00000_int_0002 |
T4 | 1499 | cs00000_int_0002 |
T5 | 2002 | cs00000_int_0002 |
T6 | 2160 | cs00000_int_0004 |
T7 | 5861 | cs00000_int_0002 |
T8 | 9696 | cs00000_int_0002 |
T9 | 10795 | cs00000_int_0004 |
T10 | 27713 | cs00000_int_0004 |
Tests that measured performance of editing the holding records with LoC workfows running in the background:
Tests | Description | Duration |
|---|---|---|
Test #1 | All workflows, Data Export All, Data Import without editing big Holdings | 1 hour |
Test #2 | All workflows, Data Export All, Data Import with single editing of a big Holdings | 1 hour |
Test #3 | All workflows, Data Export All, Data Import with editing big Holdings records continously | 1 hour |
Test #4 | Repeat of test #1 | 1 hour |
Results
Response times
Volume testing experiments:
Test # | Holding Id | Item count | Tenant | Response times, sec |
|---|---|---|---|---|
T1 | 864f750e-e6f3-5b88-a456-b5b5f0b79797 | 700 | cs00000_int_0002 | 3 |
T2 | f26f4290-84cd-5b5c-9c48-299f207aded9 | 1001 | cs00000_int_0002 | 3,3 |
T3 | 3c9199ae-ce80-5f32-8c53-1024ae17f560 | 1230 | cs00000_int_0002 | 4,7 |
T4 | b6c49272-a85b-5b9a-b83d-e42b7bd38da7 | 1499 | cs00000_int_0002 | 6,3 |
T5 | f814c846-9f29-5a5f-a71f-4beba3044cd7 | 2002 | cs00000_int_0002 | 13 |
T6 | 1a0888b9-757d-5fad-814b-66b9f08d7b59 | 2160 | cs00000_int_0004 | 13 |
T7 | 94e59f5a-d0ec-5e17-904f-3702c6d7ec3b | 5861 | cs00000_int_0002 | 35 |
T8 | 52cf4cd0-307e-534b-90d0-ef5c6d721ce0 | 9696 | cs00000_int_0002 | 60 |
T9 | 4fdb5f47-c608-5da8-9bce-3d523deac200 | 10795 | cs00000_int_0004 | 87 |
T10 | 6136b24d-d95a-5091-9f11-cf072f270c24 | 27713 | cs00000_int_0004 | 194 |
The graph added just to visualize the response times results from T1 - T10, x-axis represents the number of items attached to a holdings record while the y-axis shows the response time in seconds.