Check-in-check-out Test Report (Nolana)
20-users tests | Avg | Max | ||
|---|---|---|---|---|
mod-users | 21% | 21% | ||
mod-pubsub | 5% | 5% | ||
okapi | 14% | 14% | ||
mod-circulation | 4% | 4% | ||
mod-circulation-storage | 5% | 5% | ||
mod-inventory | 7% | 7% | ||
mod-inventory-storage | 7% | 7% | ||
mod-patron-blocks | 1% | 1% | ||
mod-feesfines | 17% | 17% | ||
mod-authtoken | 31% | 20-users tests | Avg | Max |
mod-users | 21% | 21% | ||
mod-pubsub | 5% | 5% | ||
okapi | 14% | 14% | ||
mod-circulation | 4% | 4% | ||
mod-circulation-storage | 5% | 5% | ||
mod-inventory | 7% | 7% | ||
mod-inventory-storage | 7% | 7% | ||
mod-patron-blocks | 1% | 1% | ||
mod-feesfines | 17% | 17% | ||
mod-authtoken | 31% | 78% |
Overview
This is a report for a series of Check-in-check-out test runs against the Nolana release.
Infrastructure
PTF -environment ncp3
9 m6i.2xlarge EC2 instances located in US East (N. Virginia)us-east-1
2 instances of db.r6.xlarge database instances, one reader, and one writer
MSK ptf-kakfa-3
4 m5.2xlarge brokers in 2 zones
Apache Kafka version 2.8.0
EBS storage volume per broker 300 GiB
auto.create.topics.enable=truec
log.retention.minutes=480
default.replication.factor=3
Modules memory and CPU parameters
Modules | Version | Task Definition | Running Tasks | CPU | Memory | MemoryReservation | MaxMetaspaceSize | Xmx |
|---|---|---|---|---|---|---|---|---|
mod-inventory | 19.0.1 | 1 | 2 | 1024 | 2880 | 2592 | 512m | 1814m |
mod-inventory-storage | 25.0.1 | 1 | 2 | 1024 | 2208 (1872 in MG) | 1952 (1684 in MG) | 512m | 1440m |
okapi | 4.14.7 | 1 | 3 | 1024 | 1684 (1512 in MG) | 1440 (1360 in MG) | 512m | 922m |
mod-feesfines | 18.1.0 | 1 | 2 | 128 | 1024 | 896 | 128 | 768 |
mod-patron-blocks | 1.7.1 | 1 | 2 | 1024 | 1024 | 896 | 128 | 768 |
mod-pubsub | 2.7.0 | 1 | 2 | 1024 | 1536 (1440 in MG) | 1440 (1296 in MG) | 512 | 922 |
mod-authtoken | 2.12.0 | 1 | 2 | 512 (128 in MG) | 1440 (1024 in MG) | 1152 (896 in MG) | 128 | 922 (768 in MG) |
mod-circulation-storage | 15.0.0 | 1 | 2 | 1024 | 1536 (1152 in MG) | 1440 (1024 in MG) | 512 | 896 |
mod-circulation | 23.3.0 | 1 | 2 | 1024 | 1024 | 896 | 128 | 768 |
mod-configuration | 5.9.0 | 1 | 2 | 128 | 1024 | 896 | 128m | 768m |
mod-users | 19.0.0 | 1 | 2 | 258 | 1024 | 896 | 128m | 768m |
mod-remote-storage | 1.7.0 | 1 | 2 | 128 | 1872 | 1684 | 512m | 1178m |
MG- Morning Glory release
Front End:
Item Check-in (folio_checkin-7.2.0)
Item Check-out (folio_checkout-8.2.0)
High-Level Summary
In general, there are insignificant regressions in performance in the Nolana release compared to Morning Glory. Check-in times and check-out times regressed by 10%, but are still about 20% better than the Lotus release. The 25 users' response times are very similar to the 5 users. This means that the system is very stable from 1 to 25 users. The 20 users test has the best response times and the smallest difference from the Morning Glory release.
Services' memory utilization slightly increases during the test runs. Probably it is a result of new cluster usage for the tests and memory utilization will grow over time until reaches some steady state. Added table for comparison of Nolana to Morning Glory memory utilization. To be sure of memory leak presence longevity test has to be performed.
The relevant services overall seem to occupy CPU resources nominally. Only mod-authtoken seems to have the spikes but the processes did not crash. CPU usage of all modules did not exceed 31%.
RDS CPU utilization did not exceed 20%.
Longevity test shows response times worsen over time.
The test with R/W Split enabled - 2 instances of the database (reader and writer) are working simultaneously while R/W Split is enabled. CPU usage of the writer instance decreases so potentially it can handle a higher load.
Response times of CICO are almost the same (about 1,5% better in total) while R/W Split is enabled.
Service CPU usage is similar for both R/W Split is enabled and disabled. For mod-configuration CPU usage increases with time (when memory usage reaches its max level) for both R/W Split is enabled and disabled tests. The ticket for investigation created https://folio-org.atlassian.net/browse/PERF-375
Also created tickets for WARN CQL2PgJSON Doing FT search without indexes:
mod-users https://folio-org.atlassian.net/browse/PERF-376
mod-feesfines https://folio-org.atlassian.net/browse/PERF-377
mod-circulation-storage https://folio-org.atlassian.net/browse/PERF-378
mod-configuration https://folio-org.atlassian.net/browse/PERF-380
Test Runs
Test | Virtual Users | Duration | Load generator size (recommended) | Load generator Memory(GiB) (recommended) |
1. | 5 users | 30 mins | t3.medium | 3 |
2. | 8 users | 30 mins | t3.medium | 3 |
3. | 20 users | 30 mins | t3.medium | 4 |
4. | 25 users | 30 mins | t3.medium | 4 |
Results
Response Times (Average of all tests listed above, in seconds)
Average (seconds) | 50th %tile (seconds) | 75th %tile (seconds) | 95th %tile (seconds) | |||||
Check-in | Check-out | Check-in | Check-out | Check-in | Check-out | |||