FameFlower Test Results

Overview

PERF-40 - Getting issue details... STATUS

In this workflow we are checking the performance of the loading items on instance record (in holdings accordion) workflow running in the Fameflower release. We will test it with 1, 5, 8, and 20 virtual users for 30 minutes. A longevity test will also be executed to see if there were memory issues.

Backend:
- mod-inventory-storage-19.1.2
- mod-inventory-14.1.3
- mod-authtoken-2.4.0
- mod-permissions-5.9.0
- okapi-2.38.0

- mod-circulation-18.0.9

- mod-circulation-storage-11.0.8

Frontend:
- folio_inventory-2.0.2

Environment:

55 back-end modules deployed in 110 ECS services
3 okapi ECS services
8 m5.large EC2 instances
1 db.r5.xlarge AWS RDS instance
INFO logging level

High Level Summary

Overall check in, check out time in seconds

	Average (seconds)		50th %tile (seconds)		75th %tile (seconds)		95th %tile (seconds)
	Check-in	Check-out	Check-in	Check-out	Check-in	Check-out	Check-in	Check-out
1 user	1.015	1.234	0.96	1.277	1.071	1.409	1.322	1.653
5 users	1.236	1.488	1.156	1.393	1.464	1.869	1.704	2.219
8 users	1.512	1.751	1.403	1.852	1.741	2.031	2.02	2.274
20 users	1.649	1.898	1.535	1.996	1.896	2.211	2.252	2.539

Slow APIs taking more than 100 ms to return

API	1 user (75th %tile)	5 users (75th %tile)	8 users (75th %tile)	20 Users (75th %tile)
GET /source-storage/formattedRecords/{id}	5.24 s	905 ms	906 ms	988 ms
GET_/inventory/instances keyword all "aba"
POST_/circulation/check-out-by-barcode	283 ms	346 ms	449 ms	479 ms
Get inventory/items	217 ms	232 ms	237 ms	281 ms
GET_/inventory/instances/{id}
GET_/circulation/loans

Excess logging of missing indexes - 64K lines in 45 minutes run. Logging level could be reduced to WARNING or INFO, but at the cost of having less data to work with should there be a need to troubleshoot.
JVM profiling shows JSON de/serialization operations one of the slowest operations.

Test Runs

30-min Runs for export instance UUIDs workflow:

Test	Virtual Users	Duration	OKAPI log level	Profiled	Ramp up (total time in seconds)
1.	1	30 min	INFO	No	1
2.	5	30 min	INFO	No	50
3.	8	30 min	INFO	No	80
4.	20	30 min	INFO	No	200
5.	1	30 min	INFO	Yes	1
6.	5	30 min	INFO	Yes	50
7.	8	30 min	INFO	Yes	80
8.	20	30 min	INFO	Yes	200