Check-in-check-out Test Report (Morning Glory)

20-users testsAvgMax

mod-users21%21%

mod-pubsub5%5%

okapi14%14%

mod-circulation4%4%

mod-circulation-storage5%5%

mod-inventory7%7%

mod-inventory-storage7%7%

mod-patron-blocks1%1%

mod-feesfines17%17%

mod-authtoken31%20-users testsAvgMax


mod-users21%21%


mod-pubsub5%5%


okapi14%14%


mod-circulation4%4%


mod-circulation-storage5%5%


mod-inventory7%7%


mod-inventory-storage7%7%


mod-patron-blocks1%1%


mod-feesfines17%17%


mod-authtoken31%78%














Overview

This is a report for a series of Check-in-check-out test runs against the Lotus release. 

Back End:

  • mod-circulation-23.0.11
  • mod-circulation-storage-14.1.0
  • mod-inventory-18.2.0
  • mod-inventory-storage-24.0.0
  • mod-authtoken-2.11.0
  • mod-pubsub-2.6.0
  • mod-patron-blocks-1.6.0
  • mod-feesfines-18.0.0
  • okapi-4.14.2

Front End:

  • Item Check-in (folio_checkin-7.1.0)
  • Item Check-out (folio_checkout-8.1.0)

Infrastructure:

  • 79 back-end modules deployed in 153 ECS tasks (changed. In Lotus, it was75 back-end modules deployed in 159 ECS tasks)
  • 3 okapi ECS tasks
  • 10 m6i.2xlarge  EC2 instances (changed. In Lotus it was 9 m5.xlarge)
  • 2 db.r6g.xlarge AWS RDS instance
  • INFO logging level
  • mod-inventory (running tasks -2)
    • 1024 CPU units, 1684 MiB
  • mod-inventory-storage (running tasks -2)
    • 1024 CPU units, 1684MB mem
  • mod-circulation (running tasks -2)
    • 1024 CPU units, 896MB mem
  • mod-circulation-storage (running tasks -2)
    • 1024 CPU units, 1024MB mem
  • mod-pubsub (running tasks -2)
    • 1024 CPU units, 1296 MiB
  • mod-patron-blocks (running tasks -2)
    • 1024 CPU units, 896MB mem
  • mod-feesfines (running tasks -2)
    • 128 CPU units, 896MB mem
  • mod-authtoken (running tasks -2)
    • 128 CPU units, 896MB mem
  • okapi (running tasks -3)
    • 1024 CPU units, 1360 MB mem

High-Level Summary

  • Morning Glory performs much better than Lotus response times for check-ins are mid 400ms, checkout 600ms - 800ms, with little variations between 1 and 50 users
  • Worst-performing APIs are only POST /checkin-by-barcode and POST /checkout-by-barcode.  They have improved since Lotus, now averaging in the mid 200ms even in the 50 users scenario.
    • GET /circulation/loans take 107ms on average in the 20 users and 50 users cases.

Test Runs

Test

Virtual Users

Duration

Load generator size (recommended)Load generator Memory(GiB) (recommended)

1.

1 user30 minst3.medium3

2.

5 users30 minst3.medium3

3.

8 users30 minst3.medium3

4.

20 users30 minst3.medium4
5.50 users30 minst3.large6
9.20 users longevity16 hourst3.xlarge14

Results

Response Times (Average of all tests listed above, in seconds)


Average (seconds)50th %tile (seconds)75th %tile (seconds)95th %tile  (seconds)

Check-inCheck-outCheck-inCheck-outCheck-inCheck-outCheck-inCheck-out
1 user0.5160.7950.490.7580.5180.8390.6291.012
5 users0.4180.6460.4080.6240.4330.6720.4930.766
8 users0.4110.6270.40.6060.4240.6460.4780.741
20 users0.4010.6150.3930.60.4130.6320.4560.704
50 users0.3710.6010.3730.590.3950.6160.4360.682

Response times are consistently good, and over 1 second only for check-out in the 95th percentile. The 50 users test has the best response times.

Comparisons to Last Release

The following tables compare Lotus test results against Morning Glory.

Response Time Comparison

In the tables below, the Delta columns express the differences between Lotus and Morning Glory releases in percentage.

In general, there are no regressions in performance instead, there are improvements in response times across the board.  Check-in times improved by at least 30% and check-out times consistently around 35%.  The 50 users' response times are also very similar to the 5 users. This means that the system is very stable from 1 to 50 users.

Note: LT = Lotus build, MG = Morning Glory build


Average50th percentile 
Check-in MGCheck-in LTDeltaCheck-out MGCheck-out LTDeltaCheck-in MGCheck-in LTDeltaCheck-out MGCheck-out LTDelta
1 user0.5160.7329.32%0.7951.18833.08%0.490.68228.15%0.7581.08430.07%
5 users0.4180.6232.58%0.6460.97633.81%0.4080.58229.90%0.6240.91531.80%
8 users0.4110.61533.17%0.6270.97135.43%0.40.58131.15%0.6060.90432.96%
20 users0.4010.66139.33%0.6151.05241.54%0.3930.6236.61%0.60.96737.95%
50 users0.371--0.601--0.373--0.59--


75th percentile95th percentile 
Check-in MGCheck-in LT