Overview
The test goal is to assess the performance of circulation Check-Out functionality for users with 2000 loans each.
Ticket: PERF-243 - Getting issue details... STATUS
Summary
Load tests showed that there is performance degradation of users with 2000 loans each compared to those without loans.
Test Runs
Test # | Test Conditions | Duration | Load generator size (recommended) | Load generator Memory(GiB) (recommended) | Notes |
|---|---|---|---|---|---|
1. | Checkout with 1, 5, 8, and 20 users | 30 min | t3.medium | 3 | ncp3 environment |
Results
Response Times
Baseline (users without loans)
Verification (users with 2000 loans)
Checkout response time comparison
Users quantity | Response time 95prc, sec | Degradation, sec | Degradation, % | Response time Avg, sec | Degradation, sec | Degradation, % | ||
|---|---|---|---|---|---|---|---|---|
Baseline (users without loans) | Verification (users with 2000 loans each) | Baseline (users without loans) | Verification (users with 2000 loans each) | |||||
| 1 user | 1.146 | 1.269 | 0.950 | 1.015 | ||||
| 5 users | 1.004 | 1.071 | 0.852 | 0.908 | ||||
| 8 users | 0.946 | 0.995 | 0.837 | 0.884 | ||||
| 20 users | 0.928 | 0.850 | ||||||
Instance CPU Utilization
Baseline (users without loans)
Verification (users with 2000 loans)
Service CPU Utilization
Baseline (users without loans)
Verification (users with 2000 loans)
Memory Utilization
Baseline (users without loans)
Verification (users with 2000 loans)
RDS CPU Utilization
Baseline (users without loans)
Verification (users with 2000 loans)
RDS DB connections
Baseline (users without loans)
Verification (users with 2000 loans)
Database Load
Baseline (users without loans)
Verification (users with 2000 loans)
Appendix
Infrastructure
PTF -environment ncp3
- 10 m6i.2xlarge EC2 instances located in US East (N. Virginia)us-east-1
- 2 instances of db.r6.xlarge database instances: Writer & reader instances
- MSK ptf-kakfa-3 [ kafka configurations]
- 4 kafka.m5.2xlarge brokers in 2 zones
Apache Kafka version 2.8.0
EBS storage volume per broker 300 GiB
- auto.create.topics.enable=true
- log.retention.minutes=480
- default.replication.factor=3
Modules memory and CPU parameters:
Modules | Version | Task Definition | Running Tasks | CPU | Memory (Soft/Hard limits) | MaxMetaspaceSize | Xmx |
|---|---|---|---|---|---|---|---|
| okapi | 4.14.7 | 1 | 3 | 1024 | 1440/1684 | 512 | 922 |
| mod-feesfines | 18.1.1 | 3 | 2 | 128 | 896/1024 | 128 | 768 |
| mod-patron-blocks | 1.7.1 | 4 | 2 | 1024 | 896/1024 | 128 | 768 |
| mod-pubsub | 2.7.0 | 4 | 2 | 1024 | 1440/1536 | 512 | 922 |
| mod-authtoken | 2.12.0 | 3 | 2 | 512 | 1152/1440 | 128 | 922 |
| mod-circulation-storage | 15.0.2 | 3 | 2 | 1024 | 1440/1536 | 512 | 896 |
| mod-circulation | 23.3.2 | 3 | 2 | 1024 | 896/1024 | 128 | 768 |
| mod-configuration | 5.9.0 | 3 | 2 | 128 | 896/1024 | 128 | 768 |
| mod-users | 19.0.0 | 4 | 2 | 128 | 896/1024 | 128 | 768 |
| mod-remote-storage | 1.7.1 | 3 | 2 | 128 | 1692/1872 | 512 | 1178 |
Methodology/Approach
- Conduct necessary commands to return the database to the initial state. Do this before each test run. Wait several minutes before the test start.
- Conduct check-out load tests with different numbers of users.
- Set a delay of all circulation_ and circulation-storage_ okapi timers to 0 (to avoid background jobs starting). Repeat tests with the same approach but before each test also generates 2000 loans for each user by running the command ./high_circ-data-load.sh psql_[environment name].conf [tenant].
- Compare test results.
Grafana dashboard
Verification (users with 2000 loans): http://carrier-io.int.folio.ebsco.com/grafana/d/elIt9zCnz/jmeter-performance-test-copy?orgId=1&from=1675857341711&to=1675878170222&var-percentile=95&var-test_type=baseline&var-test=circulation_CheckOut_nolana&var-env=int&var-grouping=1s&var-low_limit=250&var-high_limit=750&var-db_name=jmeter&var-sampler_type=All
API response comparison: https://docs.google.com/spreadsheets/d/1560SZ7vb9aNip0sBsUesXbaKhn0l5HzDd7_Ts8TxJGQ/edit#gid=1126827422
Please note that dashboards will expire in 6 weeks after test runs.