Table of Contents

outline	true

...

Check-In/Check-Out tests on Quesnelia carried out with 8, 20, 30, 75 users virtual users during 45 minutes and with 30 30 virtual users during longevity test (24 hours) to detect any trends in resource utilization

...

Response times in tests with 8, 20, 30 virtual users didn't differ a lot but the lowest response times were with 20 virtual users. Average in 20 vUsers test Check-In - 404 ms, average Check-Out - 729 ms
The highest response times results were with 75 virtual users. Average Check-In - 580 ms, Check-Out - 1039 ms

...

Memory consumption
- 45 minute and longevity tests didn't reveal some problems with memory usage by modules. So no memory leaks detected.

RDS CPU utilization average
- 8 vUsers - 13%, 20 vUsers - 22%, 30 vUsers - 30%, 75 vUsers - 63% During longevity test CPU grew from 30% to 45%. So it has growing trend during longevity test that can be explained by absent dcb-system-user in mod-dcb module.

CPU (User) usage by broker
- Common CPU utilization by broker during all tests was 15% with equal distribution between brokers

Common notes

Longevity test revealed CPU Utilization growing trend for mod-dcb-b module.

...

Control mod-dcb-b module related system user creation. For some reason dcb-system-user was not created after environment creation, additional redeploy of module, enabling resolved the issue. It may lead to a huge performance degradation.
Add more CPU to mod-users to exclude module fluctuations

Expand

title	Summary for tests with absent dcb-system-user

45 minute tests

Response times in tests with 8, 20, 30 virtual users didn't differ a lot but the lowest response times were with 8 virtual users. Average Check-In - 470 ms, average Check-Out - 838 ms
The highest response times results were with 75 virtual users. Average Check-In - 720 ms, Check-Out - 1301 ms
Additional test was carried out with changed configuration for mod-dcb-b (raised from 128 to 256). It helped to decrease CPU utilization for mod-dcb-b twice from 407% to 202% and nginx-okapi from 102% to 70% which helped to decrease response times more than 20% if to compare results with 128 CPU and 256 CPU. For this test average Check-In - 587 ms, Check-Out - 1055 ms. which is better than Poppy release - 5%.

Longevity test

Average Check-In - 454 ms, Check-Out - 1060 ms
No memory leaks during longevity test

Comparison with Poppy Results:

CI/CO response times increased by 4-7% for 20 and 30 vUsers tests.
For 75 vUsers test, the increase was 15%.
For 75 vUsers test CO response time was 1116 ms in Poppy and 1301 ms in Quesnelia

CPU utilization

45 minute and longevity tests used CPU in correspondence with number of vUsers but there are some modules that spiked during 75 vUsers test - mod-dcb-b - 407% with 128 CPU, mod-authtoken-b - 120%, nginx-okapi - 102%.
Additional test was carried out with changed configuration for mod-dcb-b (raised from 128 to 256). It helped to decrease CPU utilization for mod-dcb-b twice from 407% to 202% and nginx-okapi from 102% to 70% which helped to decrease response times more than 20% if to compare results with 128 CPU and 256 CPU.

Memory consumption

45 minute and longevity tests didn't reveal some problems with memory usage by modules. So no memory leaks detected.

RDS CPU utilization

8 vUsers - 13%, 20 vUsers - 22%, 30 vUsers - 30%, 75 vUsers - 50% During longevity test CPU grew from 30% to 45%. So it has growing trend during longevity test.

CPU (User) usage by broker

Common CPU utilization by broker during all tests was 15% with equal distribution between brokers

Common notes

Longevity test revealed CPU Utilization growing trend for mod-dcb-b module.

Recommendations

Investigate mod-dcb-b module behaviour. After 75 vUsers test modules require some time to stabilize CPU consumption.

...

The following table contains tests configuration information

Test #	Test artefact	vUsers	Ramp-up, sec	Duration, sec
1	circulation_checkInCheckOut_qcp1	8	80	2700
2	20	200	2700
3	30	300	2700
4	75	750	2700
5	30	300	86400

...

Error messages: POST_circulation/check-out-by-barcode (Submit_barcode_checkout)_POST_422. 422/Unprocessable Entity. Happen expectedly if instance was checked out already. Error rate - 0.06% which is acceptable.

Response time

The table contains results of Check-in, Check-out tests in Quesnelia release.

45 minute tests

	8 vUsers			20 vUsers			30 vUsers			75 vUsers
Requests	Response Times (ms)			Response Times (ms)			Response Times (ms)			Response Times (ms)
Label	#Samples	Average	95th pct	#Samples	Average	95th pct	#Samples	Average	95th pct	#Samples	Average	95th pct
Check-Out Controller	1857	741.82	904.1	4614	729.26	859	6637	767.83	935	14637	1039.21	1323
Check-In Controller	1394	408.2	484	3378	404.24	475	5063	427.22	527	11136	580.9	752
POST_circulation/check-out-by-barcode (Submit_barcode_checkout)	1858	256.34	337	4615	247.44	297	6639	252.35	310	14642	343.5	457
POST_circulation/check-in-by-barcode (Submit_barcode_checkin)	1395	172.66	225.2	3382	170.31	205	5080	174.75	224.95	11159	242.61	323
GET_circulation/loans (Submit_barcode_checkout)	1856	136.69	185.15	4613	133.16	161	6636	138.46	176	14628	186.43	259

Longevity test (with absent dcb-system-user)

30 vUsers Longevity
Requests	Samples, Response Times
Label	#Samples	95th pct	Average
Check-Out Controller	135591	1247	1065
Check-In Controller	101990	525	454
POST_circulation/check-out-by-barcode (Submit_barcode_checkout)	135593	410	340
POST_circulation/check-in-by-barcode (Submit_barcode_checkin)	101981	228	193

Comparisons

This table has comparison between average values of response times of Quesnelia and Poppy releases

	8 vUsers				20 vUsers				30 vUsers				75 vUsers
Requests	Response Times				Response Times				Response Times				Response Times
	Poppy	Quesnelia			Poppy	Quesnelia			Poppy	Quesnelia			Poppy	Quesnelia
Label	Average		Delta,ms	Difference,%	Average		Delta,ms	Difference,%	Average		Delta,ms	Difference,%	Average		Delta,ms	Difference,%
Check-Out Controller	832	741	-91	-10.94%	805	729	-76	-9.44%	807	767	-40	-4.96%	1116	1039	-77	-6.90%
Check-In Controller	479	408	-71	-14.82%	450	404	-46	-10.22%	449	427	-22	-4.90%	622	580	-42	-6.75%

Comparison of Longevity test (absent dcb-system-user in Quesnelia)

30 vUsers Longevity
Response Times
Poppy	Quesnelia
Average	Average	Delta,ms	Difference,%
1025	1065	40	3.90%
471	454	-17	-3.61%

API requests where response times >= 100 ms

...

Expand

title	Results for tests with absent dcb-system-user

Response time

The table contains results of Check-in, Check-out tests in Quesnelia release.

45 minute tests

	8 vUsers			20 vUsers			30 vUsers			75 vUsers
Requests	Samples, Response Times			Samples, Response Times			Samples, Response Times			Samples, Response Times
Label	#Samples	95th pct	Average	#Samples	95th pct	Average	#Samples	95th pct	Average	#Samples	95th pct	Average
Check-Out Controller	1785	1007.7	837.94	4576	993	842.65	6496	1020	870.2	14379	1876	1301.92
Check-In Controller	1437	554	470.2	3285	550.7	467.71	5045	562	480.62	10607	1047	720.2
POST_circulation/check-out-by-barcode (Submit_barcode_checkout)	1785	378	290.32	4577	334	282.21	6499	342	288.07	14383	665	426.66
POST_circulation/check-in-by-barcode (Submit_barcode_checkin)	1439	266	207.48	3293	237	195.47	5056	235	200.12	10629	464	295.4

Longevity test

30 vUsers Longevity
Requests	Samples, Response Times
Label	#Samples	95th pct	Average
Check-Out Controller	135591	1247	1065
Check-In Controller	101990	525	454
POST_circulation/check-out-by-barcode (Submit_barcode_checkout)	135593	410	340
POST_circulation/check-in-by-barcode (Submit_barcode_checkin)	101981	228	193

Comparisons

This table has comparison between average values of response times of Quesnelia and Poppy releases

	8 vUsers				20 vUsers				30 vUsers				75 vUsers
Requests	Response Times				Response Times				Response Times				Response Times
	Poppy	Quesnelia			Poppy	Quesnelia			Poppy	Quesnelia			Poppy	Quesnelia
Label	Average		Delta,ms	Difference,%	Average		Delta,ms	Difference,%	Average		Delta,ms	Difference,%	Average		Delta,ms	Difference,%
Check-Out Controller	832	837.94	5.94	0.71%	805	842.65	37.65	4.68%	807	870.2	63.2	7.83%	1116	1301.92	185.92	16.66%
Check-In Controller	479	470.2	-8.8	-1.84%	450	467.71	17.71	3.94%	449	480.62	31.62	7.04%	622	720.2	98.2	15.79%

Comparison table for tests after mod-dcb-b CPU 128 as default raised to 256 (Task definition: revision: qcp1-mod-dcb:3). Additional test without mod-dcb-b (# of tasks = 0)

	75 vUsers mod-dcb-b CPU 128				75 vUsers mod-dcb-b CPU 256				75 vUsers mod-dcb-b - Tasks = 0
Requests	Response Times				Response Times				Response Times
	Poppy	Quesnelia			Poppy	Quesnelia			Poppy	Quesnelia
Label	Average	Average	Delta,ms	Difference,%	Average	Average	Delta,ms	Difference,%	Average	Average	Delta,ms	Difference,%
Check-Out Controller	1116	1301.92	185.92	16.66%	1116	1055	-61	-5.47%	1116	1117	1	0.09%
Check-In Controller	622	720.2	98.2	15.79%	622	587	-35	-5.63%	622	629	7	1.13%

Comparison of Longevity test

30 vUsers Longevity
Response Times
Poppy	Quesnelia
Average	Average	Delta,ms	Difference,%
1025	1065	40	3.90%
471	454	-17	-3.61%

API requests where response times >= 100 ms

API

30 vUsers Poppy

Average, ms

30 vUsers Quesnelia

Average, ms

POST checkout-by-barcode

281

288

POST checkin-by-barcode

173

200

GET circulation/loans

135

158

...

Expand

title	CPU utilization by modules

Service Name	CPU 8 vUsers	CPU 20 vUsers	CPU 30 vUsers	CPU 75 vUsers
mod-users-b	21.08	38.95	60.75	132.25
mod-inventory-b	14.75	16.84	18.23	18.04
okapi-b	14.54	33.37	48.86	82.97
mod-configuration-b	11.74	24.4	36.58	79.42
mod-authtoken-b	9.89	23.43	31.37	98.15
mod-pubsub-b	7.88	9.08	11.25	15.37
mod-dcb-b	7.76	11.25	10.33	16.89
mod-feesfines-b	7.66	14.71	21.54	47.69
mod-quick-marc-b	6.68	5.96	6.13	6.41
mod-data-import-b	4.07	4.22	4.17	4.67
pub-okapi	3.94	9.33	13.32	32.52
mod-inventory-storage-b	3.64	7.59	11.93	14.2
mod-oa-b	2.69	2.41	2.46	3.58
nginx-okapi	2.28	5.03	7.99	84.13
mod-di-converter-storage-b	2.23	2.37	2.42	2.51
mod-patron-b	2.22	2.38	2.38	2.95
mod-source-record-manager-b	1.92	1.89	1.91	1.98
mod-source-record-storage-b	1.86	1.88	1.91	1.95
mod-circulation-storage-b	1.49	2.56	3.52	7.36
mod-search-b	1.48	1.17	3.64	1.25
mod-circulation-b	1.33	3.01	4.02	9.26
edge-patron-b	0.97	1	1.06	1.11
mod-patron-blocks-b	0.84	1.41	1.98	3.47
mod-users-bl-b	0.44	0.45	0.45	0.5
mod-entities-links-b	0.39	0.32	0.35	0.37
pub-edge	0.08	0.09	0.09	0.09

Longevity test (with absent dcb-system-user)

...

Expand

title	Memory usage by modules

Service Name	Memory 75 vUsers
mod-search-b	94.2
mod-oa-b	78.38
mod-inventory-b	72.3
mod-dcb-b	70.98
mod-users-b	63.74
okapi-b	48.33
mod-pubsub-b	47.6
mod-source-record-storage-b	45.07
mod-data-import-b	40.66
mod-di-converter-storage-b	39.51
mod-source-record-manager-b	35.98
mod-inventory-storage-b	35.84
mod-feesfines-b	34.69
mod-quick-marc-b	31.16
mod-configuration-b	30.76
mod-patron-blocks-b	30.69
mod-users-bl-b	30.24
mod-circulation-b	29.17
mod-entities-links-b	26.99
mod-patron-b	24
mod-authtoken-b	21.89
mod-circulation-storage-b	21.31
edge-patron-b	18.19
nginx-okapi	5.3
pub-okapi	4.63
pub-edge	4.46

45 minute tests

Longevity test

...

45 minute tests

Longevity test

RDS Database Connections

For both 45 minute and longevity tests RDS used max 615-630 connections. Without test it was 550 connections.

45 minute tests

Longevity test

Image Modified

CPU (User) usage by broker

Common CPU utilization by broker during all tests was 15% with equal distribution between brokers

45 minute tests

Longevity test

Image Modified

Database load

...

Expand

title	Resources Utilization

CPU Utilization

45 minute and longevity tests used CPU in correspondence with number of vUsers but there are some modules that spiked during 75 vUsers test - mod-dcb-b - 407% with 128 CPU, mod-authtoken-b - 120%, nginx-okapi - 102%.
Additional test was carried out with changed configuration for mod-dcb-b (raised from 128 to 256). It helped to decrease CPU utilization for mod-dcb-b twice from 407% to 202% and nginx-okapi from 102% to 70% which helped to decrease response times more than 20% if to compare results with 128 CPU and 256 CPU.

45 minute tests

Expand

title	CPU utilization by modules

Service Name	CPU 8 vUsers	CPU 20 vUsers	CPU 30 vUsers	CPU 75 vUsers, 128	CPU 30 vUsers Longevity	CPU 75 vUsers, 256
mod-users-b	29.31	65.11	89.61	206.63	96.57	199.92
mod-dcb-b	21.53	46.25	69.61	407.79	155.82	201.69
okapi-b	16.52	38.85	55.66	90.6	56.77	93
mod-inventory-b	14.28	18.17	17.97	15.14	15.58	15.77
mod-authtoken-b	12.08	20.81	32.34	120.62	30.98	115.95
mod-configuration-b	10.73	21.98	33.95	71.33	30.02	70.5
mod-pubsub-b	9.17	11.67	13.61	18.09	12.39	26.5
mod-feesfines-b	8.26	17.1	23.88	54.79	28.87	54.37
mod-quick-marc-b	7.2	7.63	7.57	8.19	7.93	7.88
pub-okapi	3.91	9.45	13.99	33.44	12.47	27.68
mod-inventory-storage-b	3.44	7.65	11.99	13.24	10.67	12.16
nginx-okapi	2.56	6.35	9.35	102.62	7.69	69.58
mod-patron-b	2.31	2.52	2.52	2.3	2.28	2.44
mod-oa-b	2.28	2.17	2.67	2.75	3.17	3.52
mod-di-converter-storage-b	2.2	2.19	2.29	2.39	2.12	1.97
mod-source-record-manager-b	1.93	1.9	1.87	1.88	1.81	1.95
mod-data-import-b	1.7	1.02	1.02	0.96	1.57	1.59
mod-source-record-storage-b	1.68	1.72	1.73	1.77	1.76	1.83
mod-circulation-storage-b	1.46	2.59	3.64	7.43	3.53	7.06
mod-circulation-b	1.46	3.54	4.58	10.25	5.62	11.21
mod-search-b	1.21	2.66	2.34	3.28	2.94	3.94
edge-patron-b	0.92	0.96	1	1.06	0.95	0.97
mod-patron-blocks-b	0.75	1.38	1.8	2.74	2.38	2.97
mod-users-bl-b	0.51	0.54	0.53	0.5	0.58	0.66
mod-entities-links-b	0.35	0.36	0.44	0.42	0.4	0.39
pub-edge	0.14	0.06	0.1	0.08	0.07	0.11

45 minute test with 75 vUsers (revision with 256 CPU for mod-dcb-b)

Longevity test

Memory Consumption

45 minute and longevity tests didn't reveal some problems with memory usage by modules. So no memory leaks detected.

Expand

title	Memory usage by modules

Service Name	Memory 75 vUsers	Memory 30 vUsers Longevity
mod-oa-b	81	79.24
mod-dcb-b	68.63	69.16
mod-users-b	63.65	62.98
mod-search-b	55.34	40.83
mod-pubsub-b	54.79	61.98
okapi-b	48.94	60.38
mod-inventory-b	47.49	50.79
mod-source-record-manager-b	45.06	45.76
mod-quick-marc-b	43.51	29.92
mod-di-converter-storage-b	35.02	29.58
mod-feesfines-b	34.18	34.12
mod-source-record-storage-b	33.76	31.37
mod-inventory-storage-b	33.12	40.06
mod-entities-links-b	32.3	25.08
mod-configuration-b	32.07	30.21
mod-data-import-b	31.31	39.73
mod-patron-blocks-b	31.13	30.92
mod-users-bl-b	28.95	25.82
mod-circulation-b	27.41	26.71
mod-patron-b	26.62	25.17
mod-circulation-storage-b	26.43	28.83
mod-authtoken-b	21.84	22.1
edge-patron-b	18.3	18.14
nginx-okapi	4.85	4.8
pub-okapi	4.69	4.69
pub-edge	4.58	4.63

45 minute tests

Longevity test

RDS CPU Utilization

RDS CPU utilized:

8 vUsers - 13%, 20 vUsers - 22%, 30 vUsers - 30%, 75 vUsers - 50% During longevity test CPU grew from 30% to 45%. So it has growing trend during longevity test.

45 minute tests

Longevity test

RDS Database Connections

For both 45 minute and longevity tests RDS used max 615 connections. Without test it was 550 connections.

45 minute tests

Image Modified

Longevity test

Image Modified

CPU (User) usage by broker

Common CPU utilization by broker during all tests was 15% with equal distribution between brokers

45 minute tests

Image Modified

Longevity test

Image Modified

Database load

Image Modified

Appendix

Infrastructure

PTF -environment qcp1

11 m6i.2xlarge EC2 instances located in US East (N. Virginia)us-east-1 [Number of ECS instances, instance type, location region]
1 instance of db.r6.xlarge database instance: Writer instance
MSK ptf-kakfa-3 MSK tenant [ kafka configurations]
- 4 kafka.m5.2xlarge brokers in 2 zones
- Apache Kafka version 2.8.0
- EBS storage volume per broker 300 GiB
- auto.create.topics.enable=true
- log.retention.minutes=480
- default.replication.factor=3

DB table records size:

fs09000000
- instances - 24493326
- items - 25662862
- holding records - 24459683

Modules

Expand

title	Check-In / Check-Out related modules

Module	Task Def. Revision	Module Version	Task Count	Mem Hard Limit	Mem Soft limit	CPU units	Xmx	MetaspaceSize	MaxMetaspaceSize
qcp1-pvt
Mon Mar 25 12:41:11 UTC 2024
mod-data-import	4	mod-data-import:3.1.0-SNAPSHOT.174	1	2048	1844	256	1292	384	512
mod-search	2	mod-search:3.2.0-SNAPSHOT.199	2	2592	2480	2048	1440	512	1024
mod-configuration	2	mod-configuration:5.11.0-SNAPSHOT.348	2	1024	896	128	768	88	128
mod-entities-links	3	mod-entities-links:3.0.0-SNAPSHOT.78	2	2592	2480	2096	1440	0	1024
pub-edge	2	pub-edge:2023.06.14	2	1024	896	128	768	0	0
edge-patron	2	edge-patron:5.1.0-SNAPSHOT.153	2	1024	896	256	768	88	128
mod-users-bl	2	mod-users-bl:7.6.1-SNAPSHOT.282	2	1440	1152	512	922	88	128
mod-inventory-storage	2	mod-inventory-storage:27.1.0-SNAPSHOT.719	2	4096	3690	2048	3076	384	512
mod-rtac	2	mod-rtac:3.6.0-SNAPSHOT.166	2	1024	896	128	768	88	128
mod-circulation-storage	2	mod-circulation-storage:17.2.0-SNAPSHOT.384	2	2880	2592	1536	1814	384	512
mod-circulation-item	2	mod-circulation-item:1.0.0-SNAPSHOT.16	2	1024	896	128	768	88	128
mod-pubsub	2	mod-pubsub:2.13.0-SNAPSHOT.196	2	1536	1440	1024	922	384	512
mod-di-converter-storage	2	mod-di-converter-storage:2.2.0-SNAPSHOT.108	2	1024	896	128	768	88	128
mod-patron-blocks	2	mod-patron-blocks:1.10.0-SNAPSHOT.152	2	1024	896	1024	768	88	128
mod-source-record-manager	2	mod-source-record-manager:3.8.0-SNAPSHOT.306	2	5600	5000	2048	3500	384	512
mod-quick-marc	2	mod-quick-marc:5.1.0-SNAPSHOT.134	1	2288	2176	128	1664	384	512
nginx-okapi	2	nginx-okapi:2023.06.14	2	1024	896	1024	0	0	0
okapi-b	2	okapi:5.1.2	3	1684	1440	1024	922	384	512
mod-feesfines	2	mod-feesfines:19.1.0-SNAPSHOT.189	2	1024	896	128	768	88	128
mod-dcb	2	mod-dcb:1.1.0-SNAPSHOT.18	2	1024	896	128	768	88	128
mod-oai-pmh	2	mod-oai-pmh:3.13.0-SNAPSHOT.393	2	4096	3690	2048	3076	384	512
pub-okapi	2	pub-okapi:2023.06.14	2	1024	896	128	768	0	0

Expand

title	All modules

Module	Task Def. Revision	Module Version	Task Count	Mem Hard Limit	Mem Soft limit	CPU units	Xmx	MetaspaceSize	MaxMetaspaceSize
qcp1-pvt
Mon Mar 25 12:41:11 UTC 2024
mod-remote-storage	2	mod-remote-storage:3.2.0-SNAPSHOT.162	2	4920	4472	1024	3960	512	512
mod-ncip	2	mod-ncip:1.14.5-SNAPSHOT.224	2	1024	896	128	768	88	128
mod-finance-storage	2	mod-finance-storage:8.6.0-SNAPSHOT.159	2	1024	896	1024	700	88	128
mod-agreements	2	mod-agreements:6.1.0-SNAPSHOT.195	2	1592	1488	128	0	0	0
mod-ebsconet	2	mod-ebsconet:2.2.0-SNAPSHOT.74	2	1248	1024	128	700	128	256
mod-organizations	2	mod-organizations:1.9.0-SNAPSHOT.84	2	1024	896	128	768	88	128
edge-sip2	2	edge-sip2:3.2.0-SNAPSHOT.209	2	1024	896	128	768	88	128
mod-serials-management	2	mod-serials-management:1.0.0-SNAPSHOT.25	2	2480	2312	128	1792	384	512
mod-settings	2	mod-settings:1.0.3-SNAPSHOT.63	2	1024	896	200	768	88	128
mod-data-import	4	mod-data-import:3.1.0-SNAPSHOT.174	1	2048	1844	256	1292	384	512
edge-dematic	2	edge-dematic:2.1.2-SNAPSHOT.130	1	1024	896	128	768	88	128
mod-search	2	mod-search:3.2.0-SNAPSHOT.199	2	2592	2480	2048	1440	512	1024
mod-inn-reach	2	mod-inn-reach:3.2.0-SNAPSHOT.86	2	3600	3240	1024	2880	512	1024
mod-tags	2	mod-tags:2.1.1-SNAPSHOT.132	2	1024	896	128	768	88	128
edge-courses	2	edge-courses:1.3.2-SNAPSHOT.1138	2	1024	896	128	768	88	128
mod-authtoken	2	mod-authtoken:2.15.0-SNAPSHOT.290	2	1440	1152	512	922	88	128
mod-inventory-update	2	mod-inventory-update:3.2.2-SNAPSHOT.82	2	1024	896	128	768	88	128
mod-notify	2	mod-notify:3.1.1-SNAPSHOT.262	2	1024	896	128	768	88	128
mod-configuration	2	mod-configuration:5.11.0-SNAPSHOT.348	2	1024	896	128	768	88	128
mod-orders-storage	2	mod-orders-storage:13.7.0-SNAPSHOT.215	2	1024	896	512	700	88	128
edge-caiasoft	2	edge-caiasoft:2.1.1-SNAPSHOT.82	2	1024	896	128	768	88	128
mod-login-saml	2	mod-login-saml:2.8.0-SNAPSHOT.280	2	1024	896	128	768	88	128
mod-erm-usage-harvester	2	mod-erm-usage-harvester:4.5.0-SNAPSHOT.123	2	1024	896	128	768	88	128
mod-licenses	2	mod-licenses:5.1.0-SNAPSHOT.147	2	2480	2312	128	1792	384	512
mod-gobi	2	mod-gobi:2.8.0-SNAPSHOT.244	2	1024	896	128	768	88	128
mod-password-validator	2	mod-password-validator:3.1.1-SNAPSHOT.137	2	1440	1298	128	768	384	512
mod-bulk-operations	2	mod-bulk-operations:2.0.0-SNAPSHOT.81	2	3072	2600	1024	1536	384	512
mod-fqm-manager	2	mod-fqm-manager:1.1.0-SNAPSHOT.1083	2	1024	896	128	768	88	128
edge-dcb	2	edge-dcb:1.1.0-SNAPSHOT.16	2	1024	896	128	768	88	128
mod-graphql	2	mod-graphql:1.12.1.401	2	1024	896	128	768	88	128
mod-finance	2	mod-finance:4.9.0-SNAPSHOT.160	2	1024	896	128	768	88	128
mod-erm-usage	2	mod-erm-usage:4.7.0-SNAPSHOT.99	2	1024	896	128	768	88	128
mod-batch-print	3	mod-batch-print:1.2.0-SNAPSHOT.10	2	1024	896	128	768	88	128
mod-copycat	2	mod-copycat:1.6.0-SNAPSHOT.197	2	1024	896	128	768	88	128
mod-lists	2	mod-lists:1.1.0-SNAPSHOT.1266	2	1024	896	128	768	88	128
mod-entities-links	3	mod-entities-links:3.0.0-SNAPSHOT.78	2	2592	2480	2096	1440	0	1024
mod-permissions	4	mod-permissions:6.5.0-SNAPSHOT.374	2	1684	1544	512	1024	384	512
pub-edge	2	pub-edge:2023.06.14	2	1024	896	128	768	0	0
mod-orders	2	mod-orders:12.8.0-SNAPSHOT.332	2	2048	1440	1024	1024	384	512
edge-patron	2	edge-patron:5.1.0-SNAPSHOT.153	2	1024	896	256	768	88	128
edge-ncip	2	edge-ncip:1.9.3-SNAPSHOT.103	2	1024	896	128	768	88	128
edge-inn-reach	2	edge-inn-reach:3.1.1-SNAPSHOT.45	2	1024	896	128	768	88	128
mod-users-bl	2	mod-users-bl:7.6.1-SNAPSHOT.282	2	1440	1152	512	922	88	128
mod-oa	2	mod-oa:2.1.0-SNAPSHOT.62	2	1024	896	128	768	88	128
mod-inventory-storage	2	mod-inventory-storage:27.1.0-SNAPSHOT.719	2	4096	3690	2048	3076	384	512
mod-invoice	3	mod-invoice:5.8.0-SNAPSHOT.187	2	1440	1152	512	922	88	128
mod-user-import	2	mod-user-import:3.9.0-SNAPSHOT.192	2	1024	896	128	768	88	128
mod-sender	3	mod-sender:1.11.1-SNAPSHOT.94	2	1024	896	128	768	88	128
edge-oai-pmh	2	edge-oai-pmh:2.8.0-SNAPSHOT.155	2	1512	1360	1024	1440	384	512
mod-data-export-worker	2	mod-data-export-worker:3.2.0-SNAPSHOT.195	2	3072	2800	1024	2048	384	512
mod-rtac	2	mod-rtac:3.6.0-SNAPSHOT.166	2	1024	896	128	768	88	128
mod-circulation-storage	2	mod-circulation-storage:17.2.0-SNAPSHOT.384	2	2880	2592	1536	1814	384	512
mod-calendar	2	mod-calendar:2.6.0-SNAPSHOT.218	2	1024	896	128	768	88	128
mod-source-record-storage	2	mod-source-record-storage:5.8.0-SNAPSHOT.353	2	5600	5000	2048	3500	384	512
mod-event-config	2	mod-event-config:2.6.1-SNAPSHOT.100	2	1024	896	128	768	88	128
mod-courses	2	mod-courses:1.4.9-SNAPSHOT.143	2	1024	896	128	768	88	128
mod-circulation-item	2	mod-circulation-item:1.0.0-SNAPSHOT.16	2	1024	896	128	768	88	128
mod-inventory	2	mod-inventory:20.2.0-SNAPSHOT.501	2	2880	2592	1024	1814	384	512
mod-email	2	mod-email:1.16.1-SNAPSHOT.122	2	1024	896	128	768	88	128
mod-pubsub	2	mod-pubsub:2.13.0-SNAPSHOT.196	2	1536	1440	1024	922	384	512
mod-circulation	2	mod-circulation:24.2.0-SNAPSHOT.649	2	2880	2592	1536	1814	384	512
mod-di-converter-storage	2	mod-di-converter-storage:2.2.0-SNAPSHOT.108	2	1024	896	128	768	88	128
edge-rtac	2	edge-rtac:2.7.0-SNAPSHOT.212	2	1024	896	128	768	88	128
edge-orders	2	edge-orders:2.10.0-SNAPSHOT.123	2	1024	896	128	768	88	128
mod-users	2	mod-users:19.3.0-SNAPSHOT.679	2	1024	896	128	768	88	128
mod-template-engine	2	mod-template-engine:1.19.2-SNAPSHOT.111	2	1024	896	128	768	88	128
mod-patron-blocks	2	mod-patron-blocks:1.10.0-SNAPSHOT.152	2	1024	896	1024	768	88	128
mod-audit	2	mod-audit:2.9.0-SNAPSHOT.207	2	1024	896	128	768	88	128
edge-fqm	2	edge-fqm:1.1.0-SNAPSHOT.84	2	1024	896	128	768	88	128
mod-source-record-manager	2	mod-source-record-manager:3.8.0-SNAPSHOT.306	2	5600	5000	2048	3500	384	512
nginx-edge	2	nginx-edge:2023.06.14	2	1024	896	128	0	0	0
mod-quick-marc	2	mod-quick-marc:5.1.0-SNAPSHOT.134	1	2288	2176	128	1664	384	512
nginx-okapi	2	nginx-okapi:2023.06.14	2	1024	896	1024	0	0	0
okapi-b	2	okapi:5.1.2	3	1684	1440	1024	922	384	512
mod-feesfines	2	mod-feesfines:19.1.0-SNAPSHOT.189	2	1024	896	128	768	88	128
mod-invoice-storage	2	mod-invoice-storage:5.8.0-SNAPSHOT.123	2	1872	1536	1024	1024	384	512
mod-dcb	2	mod-dcb:1.1.0-SNAPSHOT.18	2	1024	896	128	768	88	128
mod-service-interaction	2	mod-service-interaction:3.1.0-SNAPSHOT.117	2	2048	1844	256	1290	384	512
mod-patron	2	mod-patron:6.1.0-SNAPSHOT.147	2	1024	896	128	768	88	128
mod-oai-pmh	2	mod-oai-pmh:3.13.0-SNAPSHOT.393	2	4096	3690	2048	3076	384	512
edge-connexion	2	edge-connexion:1.2.0-SNAPSHOT.72	2	1024	896	128	768	88	128
mod-kb-ebsco-java	2	mod-kb-ebsco-java:4.1.0-SNAPSHOT.162	2	1024	896	128	768	88	128
mod-notes	2	mod-notes:5.2.0-SNAPSHOT.306	2	1024	896	128	952	384	512
mod-data-export-spring	2	mod-data-export-spring:3.2.0-SNAPSHOT.149	1	2048	1844	256	1536	384	512
mod-organizations-storage	2	mod-organizations-storage:4.7.0-SNAPSHOT.125	2	1024	896	128	768	88	128
mod-login	2	mod-login:7.11.0-SNAPSHOT.314	2	1440	1298	1024	768	384	512
pub-okapi	2	pub-okapi:2023.06.14	2	1024	896	128	768	0	0
mod-eusage-reports	2	mod-eusage-reports:2.1.0-SNAPSHOT.65	2	1024	896	128	768	88	128

Methodology/Approach

Run data preparation script before each CI/CO test
Update .jmx file script for Quesnelia release
Create artefact and upload to carrier-io
Use Jenkins job to change parameters and run tests
Test CI/CO with 8, 20, 30, 75 concurrent users for 45 minutes each.
Test CI/CO with 30 users for 24 hours to detect any trends in memory.

...

Versions Compared

Old Version 16

New Version Current

Key

Comparisons

RDS Database Connections

CPU (User) usage by broker

Database load

Appendix

Infrastructure

Methodology/Approach

Page Comparison

Versions Compared

Old Version 16

New Version Current

Key

Comparisons

RDS Database Connections

CPU (User) usage by broker

Database load

Appendix

Infrastructure

Methodology/Approach