Check-in-check-out Test Report (Honeysuckle)

















Overview

This is a report for a series of Check-in-check-out test runs against the Honeysuckle release. PERF-135 - Getting issue details... STATUS

Backend:

  • mod-circulation-19.2.7
  • mod-circulation-storage-12.1.4
  • mod-inventory-16.1.3
  • mod-inventory-storage-19.4.4
  • mod-authtoken-2.6.0
  • mod-pubsub-1.3.3
  • okapi-4.3.3 (also with 4.2.2)

Frontend:

  • folio_circulation-4.0.1
  • Item Check-in (folio_checkin-4.0.1)
  • Item Check-out (folio_checkout-5.0.1)

Environment:

  • 61 back-end modules deployed in 110 ECS services
  • 3 okapi ECS services
  • 8 m5.large  EC2 instances
  • 2 db.r5.xlarge AWS RDS instance
  • INFO logging level

High Level Summary

  • Check-out: Honeysuckle is slower by 9%-28% than Goldenrod
  • Check-in: 4%-22% slower than Goldenrod
  • APIs turned slower in Honeysuckle: GET /automated-patron-blocks/{id} (150% slower) and  GET /circulation/loans (60%). These are covered by MODPATBLK-70 and CIRC-1014, respectively
  • Okapi v4.3.3 seem to be using 2x-3x CPU cycles than in v1.3.2 (Goldenrod).  Potential issue found with the logging methods OKAPI-964 
  • mod-pubsub has a memory leak that would drag down performance under high loads (see section on longevity test): MODPUBSUB-136
  • Caching Okapi tokens in Okapi reduced mod-authtoken's CPU usage by over 90%
  • Database's memory usage improved dramatically from Goldenrod's - little memory consumptions observed.

Test Runs

Test

Virtual Users

Duration

OKAPI log level

1.

130 minsINFO

2.

530 minsINFO

3.

830 minsINFO

4.

2030 minsINFO

5.

2024 HoursINFO

Results

Response Times


Average (seconds)50th %tile (seconds)75th %tile (seconds)95th %tile  (seconds)

Check-inCheck-outCheck-inCheck-outCheck-inCheck-outCheck-inCheck-out
1 user0.9671.9890.8891.8320.9842.2011.2542.815
5 users1.0532.1710.9811.9691.1142.2531.5283.370
8 users1.1932.2441.0762.0221.3392.3721.8953.544
20 users2.3913.9011.6393.0732.2634.124.8118.784

The following table shows the slow 75th percentile APIs taking more than 100 ms to return, and also comparing them against Goldenrod's.  Other than the 1-user test, starting with the 5-users test, all listed APIs are slower with GET automated-patron-blocks leading the way at 150% slower, while GET circulation/loans regressed up to 60%

Note: GR = Goldenrod build, HS = Honeysuckle build

API

1 user  GR (75th %tile)

1 user  HS (75th %tile)

5 users GR (75th %tile)

5 users HS (75th %tile)

8 users GR (75th %tile)8 users HS (75th %tile)

20 users GR (75th %tile)

20 users HS (75th %tile)
GET circulation/loans0.3450.3490.3650.4060.0750.1220.6540.784
GET inventory/items0.2080.1860.2080.2220.2250.2440.3120.375

POST checkin-by-barcode 

0.6820.5930.6310.6640.8150.8741.2961.467
POST checkout-by-barcode0.7500.7170.6880.7840.7330.8771.2051.469
GET automated-patron-blocks0.0690.1630.0850.1800.0790.1970.1180.296


Average50th Percentile75th percentile95th percentile 
Check-in GRCheck-in HSCheck-out GRCheck-out HSCheck-in GRCheck-in HSCheck-out GRCheck-out HSCheck-in GRCheck-in HSCheck-out GRCheck-out HSCheck-in GRCheck-in HSCheck-out GRCheck-out HS
1 user1.060.9671.9941.9890.9770.8891.821.8321.1110.9842.0042.021