Combined Bulk edit + Check-in-check-out Test Report (Nolana)
- Olga Kondratenko
Overview
This is a report for a series of Check-in-check-out test runs together with Bulk edits against the Nolana release.
Infrastructure
PTF -environment ncp3
- 10 m6i.2xlarge EC2 instances located in US West (Oregon)us-west-2 AWS region (comparing to 10 m6i.2xlarge EC2 instances located in US East (N. Virginia)us-east-1 for Morning Glory release)
- 2 instances of db.r6.xlarge database instances, one reader, and one writer
- MSK ptf-kakfa-3
- 4 m5.2xlarge brokers in 2 zones
Apache Kafka version 2.8.0
EBS storage volume per broker 300 GiB
- auto.create.topics.enable=truec
- log.retention.minutes=480
- default.replication.factor=3
Modules memory and CPU parameters
Modules | Version | Task Definition | Running Tasks | CPU | Memory | MemoryReservation | MaxMetaspaceSize | Xmx |
|---|---|---|---|---|---|---|---|---|
| mod-inventory | 19.0.1 | 1 | 2 | 1024 | 2880 | 2592 | 512m | 1814m |
mod-inventory-storage | 25.0.1 | 1 | 2 | 1024 | 2208 (1872 in MG) | 1952 (1684 in MG) | 512m | 1440m |
| okapi | 4.14.7 | 1 | 3 | 1024 | 1684 (1512 in MG) | 1440 (1360 in MG) | 512m | 922m |
| mod-feesfines | 18.1.0 | 1 | 2 | 128 | 1024 | 896 | 128 | 768 |
| mod-patron-blocks | 1.7.1 | 1 | 2 | 1024 | 1024 | 896 | 128 | 768 |
| mod-pubsub | 2.7.0 | 1 | 2 | 1024 | 1536 (1440 in MG) | 1440 (1296 in MG) | 512 | 922 |
| mod-authtoken | 2.12.0 | 1 | 2 | 512 (128 in MG) | 1440 (1024 in MG) | 1152 (896 in MG) | 128 | 922 (768 in MG) |
| mod-circulation-storage | 15.0.0 | 1 | 2 | 1024 | 1536 (1152 in MG) | 1440 (1024 in MG) | 512 | 896 |
| mod-circulation | 23.3.0 | 1 | 2 | 1024 | 1024 | 896 | 128 | 768 |
MG- Morning Glory release
Front End:
- Item Check-in (folio_checkin-7.2.0)
- Item Check-out (folio_checkout-8.2.0)
High-Level Summary
- In general, there are insignificant regressions in performance for Check-in and check-out during bulk editing. Bulk edit duration is shorter than without CI/CO.
- Memory usage is stable for all modules, except for mod-data-export-worker increased from 57% to 60% (Stable CPU utilization was after reaching 98% in Morning Glory).
- The relevant services overall seem to occupy CPU resources nominally. Only mod-authtoken seems to have the spikes but the processes did not crash. CPU usage of all modules did not exceed 41%, except for mod-users up to 75%.
- RDS CPU utilization did not exceed 41%.
Test Runs
Test | Virtual Users | Duration of CI/CO | Load generator size (recommended) | Load generator Memory(GiB) (recommended) |
1. | 8 users CI/CO + Items editing | 30 mins | t3.medium | 3 |
2. | 8 users CI/CO + Users editing | 10 mins | t3.medium | 3 |
3. | 8 users CI/CO + Holdings editing | 10 mins | t3.medium | 3 |
| 4. | 8 users CI/CO+ Items, Users & Holdings editing | 30 mins | t3.medium | 3 |
Results
Response Times (Average of all tests listed above, in seconds)
| Check-in-check-out | Bulk edit | ||||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|
| Average (seconds) | 50th %tile (seconds) | 75th %tile (seconds) | 95th %tile (seconds) | Items | Users | Holdings | |||||
| Check-in | Check-out | Check-in | Check-out | Check-in | Check-out | Check-in | Check-out | 10k records | 2500 records | 10k records | |
| Test 1 | 0.431 | 0.793 | 0.417 | 0.770 | 0.444 | 0.807 | 0.504 | 0.930 | 20 min | - | - |
| Test 2 | 0.451 | 0.805 | 0.428 | 0.767 | 0.456 | 0.807 | 0.580 | 1.001 | - | 1 min | - |
| Test 3 | 0.428 | 0.790 | 0.410 | 0.761 | 0.434 | 0.795 | 0.521 | 0.923 | - | - | 11 min 20 sec |
| Test 4 | 0.436 | 0.797 | 0.426 | 0.776 | 0.450 | 0.811 | 0.504 | 0.941 | 29 min | 1 min | 11min 20 sec |
| CI/CO 8 users to compare | 0.456 | 0.698 | 0.441 | 0.672 | 0.471 | 0.709 | 0.534 | 0.839 | - | - | - |
Response times are consistently good.
Memory Utilization
- Memory usage is stable for all modules, except for mod-data-export-worker increased from 57% to 60% (98% in Morning Glory).
Modules CPUs Utilization
The relevant services overall seem to occupy CPU resources nominally. Only mod-authtoken seems to have the spikes but the processes did not crash. CPU usage of all modules did not exceed 41%, except for mod-users up to 75%.
Database and network
RDS CPU usage did not exceed 46%
