Overview
Test goal is to assess performance of circulation check-in functionality for users with 10 TLR (title-level requests) each.
Ticket: PERF-350 - Getting issue details... STATUS
Summary
Load tests showed that there is no significant difference in performance of users with TLRs and without it.
Test Runs
Test # | Test Conditions | Duration | Load generator size (recommended) | Load generator Memory(GiB) (recommended) | Notes |
|---|---|---|---|---|---|
1. | Checkin with 1, 8, 20 users | 30 min | t3.medium | 3 | ncp3 environment |
Results
Response Times
Baseline (users without TLR)
Verification (users with 10 TLRs)
Response time comparison
Users quantity | Response time 95prc, sec | Degradation, sec | Degradation, % | |
|---|---|---|---|---|
Baseline (users without TLR) | Verification (users with 10 TLR each) | |||
| 1 user | 0.838 | 0.742 | -0.096 | -11% |
| 8 users | 0.538 | 0.555 | 0.017 | 3% |
| 20 users | 0.495 | 0.531 | 0.036 | 7% |
Instance CPU Utilization
Baseline (users without TLR)
Verification (users with 10 TLRs)
Service CPU Utilization
Baseline (users without TLR)
Verification (users with 10 TLRs)
Memory Utilization
Baseline (users without TLR)
Verification (users with 10 TLRs)
RDS CPU Utilization
Baseline (users without TLR)
There are several spikes during test with 1 user load. This can be caused by background jobs, as such behaviour wasn't reproduced during other tests.
Verification (users with 10 TLRs)
RDS DB connections
Baseline (users without TLR)
Verification (users with 10 TLRs)
Database Load
Baseline (users without TLR)
Verification (users with 10 TLRs)
Appendix
Infrastructure
PTF -environment ncp3
- 10 m6i.2xlarge EC2 instances located in US East (N. Virginia)us-east-1
- 2 instances of db.r6.xlarge database instances: Writer & reader instances
- MSK ptf-kakfa-3 [ kafka configurations]
- 4 kafka.m5.2xlarge brokers in 2 zones
Apache Kafka version 2.8.0
EBS storage volume per broker 300 GiB
- auto.create.topics.enable=true
- log.retention.minutes=480
- default.replication.factor=3
Modules memory and CPU parameters:
Modules | Version | Task Definition | Running Tasks | CPU | Memory (Soft/Hard limits) | MaxMetaspaceSize | Xmx |
|---|---|---|---|---|---|---|---|
| okapi | 4.14.7 | 1 | 3 | 1024 | 1440/1684 | 512 | 922 |
| mod-feesfines | 18.1.1 | 3 | 2 | 128 | 896/1024 | 128 | 768 |
| mod-patron-blocks | 1.7.1 | 4 | 2 | 1024 | 896/1024 | 128 | 768 |
| mod-pubsub | 2.7.0 | 4 | 2 | 1024 | 1440/1536 | 512 | 922 |
| mod-authtoken | 2.12.0 | 3 | 2 | 512 | 1152/1440 | 128 | 922 |
| mod-circulation-storage | 15.0.2 | 3 | 2 | 1024 | 1440/1536 | 512 | 896 |
| mod-circulation | 23.3.2 | 3 | 2 | 1024 | 896/1024 | 128 | 768 |
| mod-configuration | 5.9.0 | 3 | 2 | 128 | 896/1024 | 128 | 768 |
| mod-users | 19.0.0 | 4 | 2 | 128 | 896/1024 | 128 | 768 |
| mod-remote-storage | 1.7.1 | 3 | 2 | 128 | 1692/1872 | 512 | 1178 |
Methodology/Approach
- Conduct necessary commands to return the database to the initial state. Do this before each test run. Wait several minutes before the test start.
- Conduct check-in load tests with different number of users.
- Repeat tests with the same approach but before each test also generate 10 TLR for each user by running JMeter script (Create_TLR.jmx).
- Compare test results.
Grafana dashboard
Verification (users with 10 TLRs): http://carrier-io.int.folio.ebsco.com/grafana/d/elIt9zCnz/jmeter-performance-test-copy?orgId=1&var-percentile=95&var-test_type=baseline&var-test=circulation_checkIn_nolana2&var-env=int&var-grouping=1s&var-low_limit=250&var-high_limit=750&var-db_name=jmeter&var-sampler_type=All&from=1675322822809&to=1675332023447
Please note that dashboards will expire in 6 weeks since test run.