Check-in-check-out Test Report (Honeysuckle)
Overview
This is a report for a series of Check-in-check-out test runs against the Honeysuckle release. - PERF-135Getting issue details... STATUS
Backend:
- mod-circulation-19.2.7
- mod-circulation-storage-12.1.4
- mod-inventory-16.1.3
- mod-inventory-storage-19.4.4
- mod-authtoken-2.6.0
- mod-pubsub-1.3.3
- okapi-4.3.3 (also with 4.2.2)
Frontend:
- folio_circulation-4.0.1
- Item Check-in (folio_checkin-4.0.1)
- Item Check-out (folio_checkout-5.0.1)
Environment:
- 61 back-end modules deployed in 110 ECS services
- 3 okapi ECS services
- 8 m5.large EC2 instances
- 2 db.r5.xlarge AWS RDS instance
- INFO logging level
High Level Summary
- Check-out: Honeysuckle is slower by 9%-28% than Goldenrod
- Check-in: 4%-22% slower than Goldenrod
- APIs turned slower in Honeysuckle: GET /automated-patron-blocks/{id} (150% slower) and GET /circulation/loans (60%). These are covered by MODPATBLK-70 and CIRC-1014, respectively
- Okapi v4.3.3 seem to be using 2x-3x CPU cycles than in v1.3.2 (Goldenrod). Potential issue found with the logging methods OKAPI-964
- mod-pubsub has a memory leak that would drag down performance under high loads (see section on longevity test): MODPUBSUB-136
- Caching Okapi tokens in Okapi reduced mod-authtoken's CPU usage by over 90%
- Database's memory usage improved dramatically from Goldenrod's - little memory consumptions observed.
Test Runs
Test | Virtual Users | Duration | OKAPI log level |
1. | 1 | 30 mins | INFO |
2. | 5 | 30 mins | INFO |
3. | 8 | 30 mins | INFO |
4. | 20 | 30 mins | INFO |
5. | 20 | 24 Hours | INFO |
Results
Response Times
Average (seconds) | 50th %tile (seconds) | 75th %tile (seconds) | 95th %tile (seconds) | |||||
Check-in | Check-out | Check-in | Check-out | Check-in | Check-out | Check-in | Check-out | |
1 user | 0.967 | 1.989 | 0.889 | 1.832 | 0.984 | 2.201 | 1.254 | 2.815 |
5 users | 1.053 | 2.171 | 0.981 | 1.969 | 1.114 | 2.253 | 1.528 | 3.370 |
8 users | 1.193 | 2.244 | 1.076 | 2.022 | 1.339 | 2.372 | 1.895 | 3.544 |
20 users | 2.391 | 3.901 | 1.639 | 3.073 | 2.263 | 4.12 | 4.811 | 8.784 |
The following table shows the slow 75th percentile APIs taking more than 100 ms to return, and also comparing them against Goldenrod's. Other than the 1-user test, starting with the 5-users test, all listed APIs are slower with GET automated-patron-blocks leading the way at 150% slower, while GET circulation/loans regressed up to 60%
Note: GR = Goldenrod build, HS = Honeysuckle build
API | 1 user GR (75th %tile) | 1 user HS (75th %tile) | 5 users GR (75th %tile) | 5 users HS (75th %tile) | 8 users GR (75th %tile) | 8 users HS (75th %tile) | 20 users GR (75th %tile) | 20 users HS (75th %tile) |
---|---|---|---|---|---|---|---|---|
GET circulation/loans | 0.345 | 0.349 | 0.365 | 0.406 | 0.075 | 0.122 | 0.654 | 0.784 |
GET inventory/items | 0.208 | 0.186 | 0.208 | 0.222 | 0.225 | 0.244 | 0.312 | 0.375 |
POST checkin-by-barcode | 0.682 | 0.593 | 0.631 | 0.664 | 0.815 | 0.874 | 1.296 | 1.467 |
POST checkout-by-barcode | 0.750 | 0.717 | 0.688 | 0.784 | 0.733 | 0.877 | 1.205 | 1.469 |
GET automated-patron-blocks | 0.069 | 0.163 | 0.085 | 0.180 | 0.079 | 0.197 | 0.118 | 0.296 |
Average | 50th Percentile | 75th percentile | 95th percentile | |||||||||||||
---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
Check-in GR | Check-in HS | Check-out GR | Check-out HS | Check-in GR | Check-in HS | Check-out GR | Check-out HS | Check-in GR | Check-in HS | Check-out GR | Check-out HS | Check-in GR | Check-in HS | Check-out GR | Check-out HS | |
1 user | 1.06 | 0.967 | 1.994 | 1.989 | 0.977 | 0.889 | 1.82 | 1.832 | 1.111 | 0.984 | 2.004 | 2.021 |