Overview

In the scope of https://folio-org.atlassian.net/browse/PERF-598

Perform a fixed load test on the VIH_View instance holdings details (folio-cache) functionality with the same average and high loads that are used for NLA testing to get baseline numbers.

Summary

Response time for VIH_View instance holdings details (folio-cache) separate workflow is 2.805 sec and it is higher than for high load (1.591) and normal load tests (1.456).
Average CPU usage of mod-inventory - up to 11% for all other modules did not exceed 5 %. Instance CPU Utilization did not exceed 7,5%.
Service memory utilization was stable, and no memory leaks were suspected during tests.
DB CPU usage was at the same level as without load- about 7,2%. 342 connections count without load was increased to 346 connections during the tests.

Test Runs & Results

Test #	# configuration	Test duration	comments

Test #	# configuration	Test duration	comments
1	Throughput for VIH_View instance holdings details (folio-cache) operations is 1 per minute or about 60 per hour *	1 hour	NO errors
2	Throughput for VIH_View instance holdings details (folio-cache) operations is 1 per minute or about 60 per hour *	1 hour	NO errors

* - For both cases normal and stress usage load for VIH_View instance holdings details (folio-cache) operations should be every 5 minutes or 12/hour according to NLA workflow document. That is why was assumed that load 1 operation per minute or about 60 per hour is enough for both cases.

Test results from 1st test run (1st, 2nd and 3rd test run results are similar):

Test #	Workflow name	Total time it takes to complete workflow										Comment
		Avg (sec) Baseline	95th pct (sec) Baseline	Avg With DI (sec) normal load	95th pct with DI (sec) normal load	Avg (sec) high load with FYR	95th pct (sec) high load with FYR	Avg (sec) high load	95th pct (sec) high load	Avg (sec) normal load	95th pct (sec) normal load	Comment
1	VIH_View instance holdings details (folio-cache)	2.805	3.443	21.241	31.698	1.959	2.521	1.591	1.837	1.456	1.572	The most time consuming request is VIH_GET /inventory/instances?offset=0&limit=1000 average baseline time is 2.668 s

Throughput graphs

For Test#1, and Test#2 graphs are almost the same.

Memory Utilization

This graph represents memory usage of test runs and shows that no memory leak is suspected for all of the modules.

Service CPU Utilization

Average CPU usage of mod-inventory - up to 11% for all other modules did not exceed 5 %.

Detailed modules (mod-inventory, mod-inventory-storage, mod-source-record-storage, mod-source-record-manager)

Instance CPU Utilization

Instance CPU Utilization did not exceed 7,5%.

RDS CPU Utilization

DB CPU usage was at the same level as without load- about 7,2%.

RDS Database Connections

Test# 1 - Test# 3 - 342 connections count without load was increased to 346 connections during the test.

Appendix

Infrastructure

Load generator

Instance Type: t3.2xlarge (Ram memory in GB available per load generator - 30 GB needed for the test with high load)

PTF -environment ncp3

9 m6i.2xlarge EC2 instances located in US East (N. Virginia)us-east-1
2 database instances, one reader, and one writer
MSK ptf-kakfa-3
- 4 m5.2xlarge brokers in 2 zones
- Apache Kafka version 2.8.0
- EBS storage volume per broker 300 GiB
- auto.create.topics.enable=true
- log.retention.minutes=480
- default.replication.factor=3
Kafka topics partitioning: - 2 partitions for DI topics

Modules memory and CPU parameters

Modules	Version	Task Definition	Running Tasks	CPU	Memory	MemoryReservation	MaxMetaspaceSize	Xmx

Modules	Version	Task Definition	Running Tasks	CPU	Memory	MemoryReservation	MaxMetaspaceSize	Xmx
mod-inventory-storage	26.0.0	1	2	1024	2208	1952	384	1440
mod-inventory	20.0.4	1	2	1024	2880	2592	512	1814
mod-tags	2.0.1	1	2	128	1024	896	128	768
mod-gobi	2.6.0	1	2	128	1024	896	128	700
mod-remote-storage	2.0.2	1	2	1024	4920	4472	512	3960
mod-invoice-storage	5.6.0	1	2	128	1024	896	128	700
edge-sip2	3.0.0	1	2	128	1024	896	128	768
mod-users-bl	7.5.0	1	2	512	1440	1152	128	922
edge-rtac	2.6.0	1	2	128	1024	896	128	768
mod-feesfines	18.2.1	1	2	128	1024	896	128	768
mod-rtac	3.5.0	1	2	128	1024	896	128	768
mod-erm-usage-harvester	4.3.0	1	2	128	1024	896	128	768
mod-search	2.0.1	1	2	400	2592	2480	1024	1440
mod-service-interaction	2.2.2	1	2	256	2048	1844	512	1290
edge-ncip	1.8.1	1	2	128	1024	896	128	768
mod-authtoken	2.13.0	1	2	512	1440	1152	128	922
mod-permissions	6.3.1	2	2	512	1684	1544	512	1024
mod-circulation-storage	16.0.0	1	2	1024	1536	1440	512	896
mod-ncip	1.13.1	1	2	128	1024	896	128	768
mod-pubsub	2.9.1	1	2	1024	1536	1440	512	922
edge-orders	2.8.1	1	2	1024	1536	1440	512	922
mod-circulation	23.5.4	1	2	1536	2880	2592	128	700
edge-caiasoft	2.0.0	1	2	128	1024	896	-	-
mod-data-export

VIH_View instance holdings details (folio-cache) - NLA report