Check-in-check-out Test Report (Juniper)



















Overview

This is a report for a series of Check-in-check-out test runs against the Juniper release. 

Backend:

  • mod-circulation-22.0.0
  • mod-circulation-storage-13.0.0
  • mod-inventory-17.0.1
  • mod-inventory-storage-21.0.0
  • mod-authtoken-2.8.0
  • mod-pubsub-2.3.1
  • mod-patron-blocks-1.3.0
  • mod-feesfines-16.1.0
  • okapi-4.8.2

Frontend:

  • folio_circulation-5.1.0
  • Item Check-in (folio_checkin-5.1.0)
  • Item Check-out (folio_checkout-6.1.0)

Environment:

  • 72 back-end modules deployed in 136 ECS services
  • 3 okapi ECS services
  • 12 m5.large  EC2 instances
  • 2 db.r5.xlarge AWS RDS instance
  • INFO logging level

High Level Summary

  • GET automated-patron-blocks response time gets worse even with 1 user (46%) and exponentially worse with 5 users (53%), 8 users (78%), 20 users (259%) MODPATBLK-94 - Getting issue details... STATUS
  • Database quickly runs out of memory, 2GB per 2 hours in a 8 users test, could be due to mod-patron-block queries.  RMB-863 - Getting issue details... STATUS
  • Check-in time improved compared to Iris.
  • See below for API calls consuming high CPU utilization percentage.
  • mod-inventory-storage appears to have a slow gradual memory leak MODINVSTOR-751 - Getting issue details... STATUS

Test Runs

Test

Virtual Users

Duration

1.

130 mins

2.

530 mins

3.

830 mins

4.

2030 mins

5.

130 mins

6.

530 mins

7.

830 mins

8.

2030 mins

9.

130 mins

10.

530 mins

11.

830 mins

12.

2030 mins
13.201 day

Results

Response Times (Average of all tests listed above, in seconds)


Average (seconds)50th %tile (seconds)75th %tile (seconds)95th %tile  (seconds)

Check-inCheck-outCheck-inCheck-outCheck-inCheck-outCheck-inCheck-out
1 user0.942.5820.852.401.1062.751.423.83
5 users1.002.980.912.721.133.361.684.9
8 users1.433.931.293.621.674.552.546.69
20 users4.7513.354.6312.535.9516.508.2123.67

The following table shows the slow 75th percentile APIs taking more than 100 ms to return, and also comparing them against Goldenrod's.  Other than the 1-user test, starting with the 5-users test, all listed APIs are slower with GET automated-patron-blocks leading the way at 150% slower, while GET circulation/loans regressed up to 60%. 

Average Response Time in Seconds. Note: IR = Iris build, JP = Juniper build.

API

1 user  IR (75th %tile) 

1 user  JP (75th %tile)

5 users IR (75th %tile)

5 users JP (75th %tile)

8 users IR (75th %tile)8 users JP (75th %tile)

20 users IR (75th %tile)

20 users JP (75th %tile)
POST checkout-by-barcode1.081.081.431.371.801.743.595.62
POST checkin-by-barcode 0.68s0.580.980.631.410.932.891.83

GET automated-patron-blocks

0.280.410.400.610.500.891.063.81
GET circulation/loans0.360.350.600.520.780.621.571.14
GET inventory/items0.350.240.440.290.550.421.133.97
  • POST checkout-by-barcode response time IMPROVES up to 8 users, more than 8 users (i.e., 20 users it's a lot worse)
  • POST checkin-by-barcode response time IMPROVES throughout
  • GET automated-patron-blocks response time gets worse even with 1 user (46%) and exponentially worse with 5 users (53%), 8 users (78%), 20 users (259%)
  • GET circulation/loans response time IMPROVES throughout
  • GET inventory/items response time IMPROVES up to 8 users, more than 8 users (i.e., 20 users it's a lot worse)


Average50th Percentile75th percentile95th percentile 
Check-in IRCheck-in JPCheck-out IRCheck-out JPCheck-in IR