Combined Bulk edit + Check-in-check-out Test Report (Nolana)
20-users tests | Avg | Max | ||
|---|---|---|---|---|
mod-users | 21% | 21% | ||
mod-pubsub | 5% | 5% | ||
okapi | 14% | 14% | ||
mod-circulation | 4% | 4% | ||
mod-circulation-storage | 5% | 5% | ||
mod-inventory | 7% | 7% | ||
mod-inventory-storage | 7% | 7% | ||
mod-patron-blocks | 1% | 1% | ||
mod-feesfines | 17% | 17% | ||
mod-authtoken | 31% | 20-users tests | Avg | Max |
mod-users | 21% | 21% | ||
mod-pubsub | 5% | 5% | ||
okapi | 14% | 14% | ||
mod-circulation | 4% | 4% | ||
mod-circulation-storage | 5% | 5% | ||
mod-inventory | 7% | 7% | ||
mod-inventory-storage | 7% | 7% | ||
mod-patron-blocks | 1% | 1% | ||
mod-feesfines | 17% | 17% | ||
mod-authtoken | 31% | 78% |
Overview
This is a report for a series of Check-in-check-out test runs together with Bulk edits against the Nolana release.
Infrastructure
PTF -environment ncp3
10 m6i.2xlarge EC2 instances located in US West (Oregon)us-west-2 AWS region (comparing to 10 m6i.2xlarge EC2 instances located in US East (N. Virginia)us-east-1 for Morning Glory release)
2 instances of db.r6.xlarge database instances, one reader, and one writer
MSK ptf-kakfa-3
4 m5.2xlarge brokers in 2 zones
Apache Kafka version 2.8.0
EBS storage volume per broker 300 GiB
auto.create.topics.enable=truec
log.retention.minutes=480
default.replication.factor=3
Modules memory and CPU parameters
Modules | Version | Task Definition | Running Tasks | CPU | Memory | MemoryReservation | MaxMetaspaceSize | Xmx |
|---|---|---|---|---|---|---|---|---|
mod-inventory | 19.0.1 | 1 | 2 | 1024 | 2880 | 2592 | 512m | 1814m |
mod-inventory-storage | 25.0.1 | 1 | 2 | 1024 | 2208 (1872 in MG) | 1952 (1684 in MG) | 512m | 1440m |
okapi | 4.14.7 | 1 | 3 | 1024 | 1684 (1512 in MG) | 1440 (1360 in MG) | 512m | 922m |
mod-feesfines | 18.1.0 | 1 | 2 | 128 | 1024 | 896 | 128 | 768 |
mod-patron-blocks | 1.7.1 | 1 | 2 | 1024 | 1024 | 896 | 128 | 768 |
mod-pubsub | 2.7.0 | 1 | 2 | 1024 | 1536 (1440 in MG) | 1440 (1296 in MG) | 512 | 922 |
mod-authtoken | 2.12.0 | 1 | 2 | 512 (128 in MG) | 1440 (1024 in MG) | 1152 (896 in MG) | 128 | 922 (768 in MG) |
mod-circulation-storage | 15.0.0 | 1 | 2 | 1024 | 1536 (1152 in MG) | 1440 (1024 in MG) | 512 | 896 |
mod-circulation | 23.3.0 | 1 | 2 | 1024 | 1024 | 896 | 128 | 768 |
MG- Morning Glory release
Front End:
Item Check-in (folio_checkin-7.2.0)
Item Check-out (folio_checkout-8.2.0)
High-Level Summary
In general, there are insignificant regressions in performance for Check-in and check-out during bulk editing. Bulk edit duration is shorter than without CI/CO.
Memory usage is stable for all modules, except for mod-data-export-worker increased from 57% to 60% (Stable CPU utilization was after reaching 98% in Morning Glory).
The relevant services overall seem to occupy CPU resources nominally. Only mod-authtoken seems to have the spikes but the processes did not crash. CPU usage of all modules did not exceed 41%, except for mod-users up to 75%.
RDS CPU utilization did not exceed 41%.
Test Runs
Test | Virtual Users | Duration of CI/CO | Load generator size (recommended) | Load generator Memory(GiB) (recommended) |
1. | 8 users CI/CO + Items editing | 30 mins | t3.medium | 3 |
2. | 8 users CI/CO + Users editing | 10 mins | t3.medium | 3 |
3. | 8 users CI/CO + Holdings editing | 10 mins | t3.medium | 3 |
4. | 8 users CI/CO+ Items, Users & Holdings editing | 30 mins | t3.medium | 3 |
Results
Response Times (Average of all tests listed above, in seconds)
Check-in-check-out | Bulk edit | ||||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|
Average (seconds) | 50th %tile (seconds) | 75th %tile (seconds) | 95th %tile (seconds) | Items | Users | Holdings | |||||
Check-in | Check-out | Check-in | Check-out | Check-in | Check-out | Check-in | Check-out | 10k records | 2500 records | 10k records | |
Test 1 | 0.431 | 0.793 | 0.417 | 0.770 | 0.444 | 0.807 | 0.504 | 0.930 | 20 min | - | - |
Test 2 | 0.451 | 0.805 | 0.428 | 0.767 | 0.456 | 0.807 | 0.580 | 1.001 | - | 1 min | - |