/
Check-in-check-out Test Report (Morning Glory)

Check-in-check-out Test Report (Morning Glory)

Jul 27, 2022

20-users tests

Avg

Max

 

 

20-users tests

Avg

Max

 

 

mod-users

21%

21%

 

 

mod-pubsub

5%

5%

 

 

okapi

14%

14%

 

 

mod-circulation

4%

4%

 

 

mod-circulation-storage

5%

5%

 

 

mod-inventory

7%

7%

 

 

mod-inventory-storage

7%

7%

 

 

mod-patron-blocks

1%

1%

 

 

mod-feesfines

17%

17%

 

 

mod-authtoken

31%

20-users tests

Avg

Max

 

 

mod-users

21%

21%

 

 

mod-pubsub

5%

5%

 

 

okapi

14%

14%

 

 

mod-circulation

4%

4%

 

 

mod-circulation-storage

5%

5%

 

 

mod-inventory

7%

7%

 

 

mod-inventory-storage

7%

7%

 

 

mod-patron-blocks

1%

1%

 

 

mod-feesfines

17%

17%

 

 

mod-authtoken

31%

78%

 

 

 

 

 

 

 

 

 

 

 

 

 

 

Overview

This is a report for a series of Check-in-check-out test runs against the Lotus release. 

Back End:

  • mod-circulation-23.0.11

  • mod-circulation-storage-14.1.0

  • mod-inventory-18.2.0

  • mod-inventory-storage-24.0.0

  • mod-authtoken-2.11.0

  • mod-pubsub-2.6.0

  • mod-patron-blocks-1.6.0

  • mod-feesfines-18.0.0

  • okapi-4.14.2

Front End:

  • Item Check-in (folio_checkin-7.1.0)

  • Item Check-out (folio_checkout-8.1.0)

Infrastructure:

  • 79 back-end modules deployed in 153 ECS tasks (changed. In Lotus, it was75 back-end modules deployed in 159 ECS tasks)

  • 3 okapi ECS tasks

  • 10 m6i.2xlarge  EC2 instances (changed. In Lotus it was 9 m5.xlarge)

  • 2 db.r6g.xlarge AWS RDS instance

  • INFO logging level

  • mod-inventory (running tasks -2)

    • 1024 CPU units, 1684 MiB

  • mod-inventory-storage (running tasks -2)

    • 1024 CPU units, 1684MB mem

  • mod-circulation (running tasks -2)

    • 1024 CPU units, 896MB mem

  • mod-circulation-storage (running tasks -2)

    • 1024 CPU units, 1024MB mem

  • mod-pubsub (running tasks -2)

    • 1024 CPU units, 1296 MiB

  • mod-patron-blocks (running tasks -2)

    • 1024 CPU units, 896MB mem

  • mod-feesfines (running tasks -2)

    • 128 CPU units, 896MB mem

  • mod-authtoken (running tasks -2)

    • 128 CPU units, 896MB mem

  • okapi (running tasks -3)

    • 1024 CPU units, 1360 MB mem

High-Level Summary

  • Morning Glory performs much better than Lotus response times for check-ins are mid 400ms, checkout 600ms - 800ms, with little variations between 1 and 50 users

  • Worst-performing APIs are only POST /checkin-by-barcode and POST /checkout-by-barcode.  They have improved since Lotus, now averaging in the mid 200ms even in the 50 users scenario.

    • GET /circulation/loans take 107ms on average in the 20 users and 50 users cases.

Test Runs

Test

Virtual Users

Duration

Load generator size (recommended)

Load generator Memory(GiB) (recommended)

1.

1 user

30 mins

t3.medium

3

2.

5 users

30 mins

t3.medium

3

3.

8 users

30 mins

t3.medium

3

4.

20 users

30 mins

t3.medium

4

5.

50 users

30 mins

t3.large

6

9.

20 users longevity

16 hours

t3.xlarge

14

Results

Response Times (Average of all tests listed above, in seconds)

 

Average (seconds)

50th %tile (seconds)

75th %tile (seconds)

95th %tile  (seconds)

 

Check-in

Check-out

Check-in

Check-out

Check-in

Check-out

Check-in

Check-out

1 user

0.516

0.795

0.49

0.758

0.518

0.839

0.629

1.012

5 users

0.418

0.646

0.408

0.624

0.433

0.672

0.493

0.766

8 users

0.411

0.627

0.4

0.606

0.424

0.646

0.478

0.741

20 users

0.401

0.615

0.393

0.6

0.413

0.632

0.456

0.704

50 users

0.371

0.601

0.373

0.59

0.395

0.616

0.436

0.682

Response times are consistently good, and over 1 second only for check-out in the 95th percentile. The 50 users test has the best response times.

Comparisons to Last Release

The following tables compare Lotus test results against Morning Glory.

Response Time Comparison

In the tables below, the Delta columns express the differences between Lotus and Morning Glory releases in percentage.

In general, there are no regressions in performance instead, there are improvements in response times across the board.  Check-in times improved by at least 30% and check-out times consistently around 35%.  The 50 users' response times are also very similar to the 5 users. This means that the system is very stable from 1 to 50 users.

Note: LT = Lotus build, MG = Morning Glory build

 

Average

50th percentile 

Check-in MG

Check-in LT

Delta

Check-out MG

Check-out LT

Delta

Check-in MG

Check-in LT

Delta

Check-out MG

Check-out LT

Delta

1 user

0.516

0.73

29.32%

0.795

1.188

33.08%

0.49

0.682

28.15%

0.758

1.084

30.07%

5 users

0.418

0.62

32.58%

0.646

0.976

33.81%

0.408

0.582

29.90%

0.624

0.915

31.80%

8 users

0.411

0.615

33.17%

0.627

0.971

35.43%

0.4

0.581

31.15%

0.606

0.904

32.96%

20 users

0.401

0.661