Overview

This is a report for a series of Check-in-check-out test runs against the Lotus release.

Back End:

mod-circulation-23.0.11
mod-circulation-storage-14.1.0
mod-inventory-18.2.0
mod-inventory-storage-24.0.0
mod-authtoken-2.11.0
mod-pubsub-2.6.0
mod-patron-blocks-1.6.0
mod-feesfines-18.0.0
okapi-4.14.2

Front End:

Item Check-in (folio_checkin-7.1.0)
Item Check-out (folio_checkout-8.1.0)

Infrastructure:

79 back-end modules deployed in 153 ECS tasks (changed. In Lotus, it was75 back-end modules deployed in 159 ECS tasks)
3 okapi ECS tasks
10 m6i.2xlarge EC2 instances (changed. In Lotus it was 9 m5.xlarge)
2 db.r6g.xlarge AWS RDS instance
INFO logging level
mod-inventory (running tasks -2)
- 1024 CPU units, 1684 MiB
mod-inventory-storage (running tasks -2)
- 1024 CPU units, 1684MB mem
mod-circulation (running tasks -2)
- 1024 CPU units, 896MB mem
mod-circulation-storage (running tasks -2)
- 1024 CPU units, 1024MB mem
mod-pubsub (running tasks -2)
- 1024 CPU units, 1296 MiB
mod-patron-blocks (running tasks -2)
- 1024 CPU units, 896MB mem
mod-feesfines (running tasks -2)
- 128 CPU units, 896MB mem
mod-authtoken (running tasks -2)
- 128 CPU units, 896MB mem
okapi (running tasks -3)
- 1024 CPU units, 1360 MB mem

High-Level Summary

Morning Glory performs much better than Lotus response times for check-ins are mid 400ms, checkout 600ms - 800ms, with little variations between 1 and 50 users
Worst-performing APIs are only POST /checkin-by-barcode and POST /checkout-by-barcode. They have improved since Lotus, now averaging in the mid 200ms even in the 50 users scenario.
- GET /circulation/loans take 107ms on average in the 20 users and 50 users cases.

Test Runs

Test	Virtual Users	Duration	Load generator size (recommended)	Load generator Memory(GiB) (recommended)
1.	1 user	30 mins	t3.medium	3
2.	5 users	30 mins	t3.medium	3
3.	8 users	30 mins	t3.medium	3
4.	20 users	30 mins	t3.medium	4
5.	50 users	30 mins	t3.large	6
9.	20 users longevity	16 hours	t3.xlarge	14

Results

Response Times (Average of all tests listed above, in seconds)

	Average (seconds)		50th %tile (seconds)		75th %tile (seconds)		95th %tile (seconds)
	Check-in	Check-out	Check-in	Check-out	Check-in	Check-out	Check-in	Check-out
1 user	0.516	0.795	0.49	0.758	0.518	0.839	0.629	1.012
5 users	0.418	0.646	0.408	0.624	0.433	0.672	0.493	0.766
8 users	0.411	0.627	0.4	0.606	0.424	0.646	0.478	0.741
20 users	0.401	0.615	0.393	0.6	0.413	0.632	0.456	0.704
50 users	0.371	0.601	0.373	0.59	0.395	0.616	0.436	0.682

Response times are consistently good, and over 1 second only for check-out in the 95th percentile. The 50 users test has the best response times.

Comparisons to Last Release

The following tables compare Lotus test results against Morning Glory.

Response Time Comparison

In the tables below, the Delta columns express the differences between Lotus and Morning Glory releases in percentage.

In general, there are no regressions in performance instead, there are improvements in response times across the board. Check-in times improved by at least 30% and check-out times consistently around 35%. The 50 users' response times are also very similar to the 5 users. This means that the system is very stable from 1 to 50 users.

Note: LT = Lotus build, MG = Morning Glory build

	Average						50th percentile
	Check-in MG	Check-in LT	Delta	Check-out MG	Check-out LT	Delta	Check-in MG	Check-in LT	Delta	Check-out MG	Check-out LT	Delta
1 user	0.516	0.73	29.32%	0.795	1.188	33.08%	0.49	0.682	28.15%	0.758	1.084	30.07%
5 users	0.418	0.62	32.58%	0.646	0.976	33.81%	0.408	0.582	29.90%	0.624	0.915	31.80%
8 users	0.411	0.615	33.17%	0.627	0.971	35.43%	0.4	0.581	31.15%	0.606	0.904	32.96%
20 users	0.401	0.661	39.33%	0.615	1.052	41.54%	0.393	0.62	36.61%	0.6	0.967	37.95%
50 users	0.371	-	-	0.601	-	-	0.373	-	-	0.59	-	-

	75th percentile						95th percentile
	Check-in MG	Check-in LT

Folio Development Teams

Check-in-check-out Test Report (Morning Glory)