Table of Contents

Overview

...

For a single user, performance is better in the Eureka environment. However, when scaled to 20 usersvusers, there is a performance degradation of approximately 7.8%
Very high load for the OpenSearch cluster with more than 52 search operations per second. ERROR: "Too Many Requests" observed during the test with 20 users vusers - non-Eureka.
Service CPU utilization was up to 6% 14% for mod-search and 5% for mod-inventory-storage, for all other services CPU did not exceed 4%6%.
Memory utilization was stable and no memory leaks or OOM issues were observed.
RDS CPU utilization was about up to 6%10% for all tests.
Non-Eureka environment uses 50 fewer database connections than Eureka.
Open search CPU utilization was up to 100% with 20 user tests.

Recommendations & Jiras

Errors during 20 users vusers - non-Eureka test are due to a very high load for the OpenSearch cluster. For this test we have a throughput of about 55 ops/s and the maximal Elastic Search throughput is 52 search operations (https://repost.aws/knowledge-center/opensearch-resolve-429-error) according to our cluster configurations.

Use the following formula to calculate maximum active threads for search requests:int ((# of available_processors * 3) / 2) + 1.
For an r6g.2xlarge.search node, you can perform a maximum of 13 search operations:
(8 VCPUs * 3) / 2 + 1 = 13 operations
For an OpenSearch Service cluster with four nodes, you can perform a maximum of 52 search operations:
4 nodes * 13 = 52 operations

...

Test Runs

60 minutes tests

Test#	Test name	results	Comments
1	1

user

vuser - non-Eureka	Success	0 errors
2	20

users

vusers - non-Eureka	Completed with errors	3224 errors out of 199179 API calls ERROR: "Too Many Requests"
3	20

users

vusers - non-Eureka retest	Completed with errors	41894 errors out of 252249 API calls ERROR: "Too Many Requests"
4	1

user

vuser - Eureka	Success	0 errors
5	20

users

vusers - Eureka

Success

0 errors

Results and Comparison Eureka and non-Eureka

...

Show total requests processed increase as we increase the number of users concurrently.

	1

user

vuser - non-Eureka(avg)	1

user

vuser - Eureka (avg)	%	20

users

vusers - non-Eureka(avg)	20

users

vusers - Eureka (avg)	%
Total requests processed in 60 minutes test run	19657	20811	5.9%	199179 (3224 errors)	183864	-7.6%
Average throughput	5.44 ops/s	5.77 ops/s	6%	55.2 ops/s	50.9 ops/s	-7.8%
Average response time	194	185	-4.6%	365	382	4.7%

	1

user

vuser - non-Eureka		1

user

vuser - Eureka			20

users

vusers - non-Eureka		20

users

vusers - Eureka
transaction	Number Of Samples	Average	Number Of Samples	Average	%	Number Of Samples	Average	Number Of Samples	Average	%
MSF_GET /search/authorities keyword <> random sentence	393	207	416	200	-3.38164	3943	540	3682	477	-11.6667
MSF_GET /search/authorities keyword <> randomword	393	202	417	194	-3.9604	4104	516	3684	462	-10.4651
MSF_GET /search/authorities keyword = random sentence	393	1224	416	1142	-6.69935	3933	2120	3682	1545	-27.1226
MSF_GET /search/authorities keyword = random sentence	393	30	416	23	-23.3333	3937	95	3682	245	157.8947
MSF_GET /search/authorities keyword = randomword	393	44	417	32	-27.2727	4246	127	3684	234	84.25197
MSF_GET /search/authorities keyword == random sentence	393	29	416	22	-24.1379	3949	92	3683	225	144.5652
MSF_GET /search/authorities keyword == randomword	393	43	417	31	-27.907	4041	138	3683	244	76.81159
MSF_GET /search/authorities keyword all randomword	393	891	417	821	-7.85634	3997	1626	3683	1191	-26.7528
MSF_GET /search/authorities keyword all random sentence	393	32	417	25	-21.875	3979	104	3683	243	133.6538
MSF_GET /search/authorities keyword all randomword	394	67	417	54	-19.403	5481	141	3686	262	85.8156
MSF_GET /search/authorities keyword any random sentence	393	99	416	86	-13.1313	3958	226	3683	272	20.35398
MSF_GET /search/authorities keyword any randomword	394	55	417	43	-21.8182	4618	129	3685	232	79.84496
MSF_GET /search/instances: contributors <> 2random words	393	158	416	152	-3.79747	3877	379	3679	408	7.651715
MSF_GET /search/instances: contributors = *randomword	393	198	416	192	-3.0303	3889	429	3680	441	2.797203
MSF_GET /search/instances: contributors = randomword*	393	57	416	46	-19.2982	3892	101	3680	227	124.7525
MSF_GET /search/instances: contributors == 2random words	393	23	416	17	-26.087	3880	78	3679	217	178.2051
MSF_GET /search/instances: contributors all 2random words	393	23	416	16	-30.4348	3876	85	3677	231	171.7647
MSF_GET /search/instances: contributors all randomword	393	58	416	47	-18.9655	3896	109	3680	232	112.844
MSF_GET /search/instances: contributors any 2random words	393	65	416	51	-21.5385	3885	115	3680	244	112.1739
MSF_GET /search/instances: keyword all random sentence AND languages == randomlang	786	40	832	35	-12.5	7747	98	7354	233	137.7551
MSF_GET /search/instances: languages == (lang) AND items.status.name == ("Available" OR "In transit")	786	119	832	113	-5.04202	7743	273	7350	346	26.73993
MSF_GET /search/instances: languages == lang	786	110	832	104	-5.45455	7743	260	7349	344	32.30769
MSF_GET /search/instances: languages == lang AND items.status.name == "Available"	786	123	832	114	-7.31707	7744	287	7350	353	22.99652
MSF_GET /search/instances: subjects <> 1randomword	393	70	416	59	-15.7143	3867	126	3670	338	168.254
MSF_GET /search/instances: subjects = (randomword OR randomword)	393	67	416	52	-22.3881	3905	129	3682	247	91.47287
MSF_GET /search/instances: subjects = *1randomword	393	314	416	318	1.273885	3868	645	3672	601	-6.82171
MSF_GET /search/instances: subjects = 1randomword*	393	70	416	58	-17.1429	3869	113	3672	235	107.9646
MSF_GET /search/instances: subjects == 1randomword	393	40	416	32	-20	3868	123	3670	232	88.61789
MSF_GET /search/instances: subjects all 1randomword	393	47	416	37	-21.2766	3869	124	3672	241	94.35484
MSF_GET /search/instances: subjects all randomword AND title all randomword	393	36	416	27	-25	3930	98	3682	237	141.8367
MSF_GET /search/instances: subjects all randomword NOT title all randomword	393	81	416	67	-17.284	3900	125	3681	235	88
MSF_GET /search/instances: subjects all randomword OR title all randomword	393	96	416	83	-13.5417	3913	165	3682	261	58.18182
MSF_GET /search/instances: subjects any 2random words	393	63	416	52	-17.4603	3868	133	3670	255	91.72932
MSF_GET /search/instances: title <> 3 word sentence	393	167	416	167	0	3863	403	3670	415	2.977667
MSF_GET /search/instances: title == 3 word sentence	393	36	416	29	-19.4444	3866	93	3670	237	154.8387
MSF_GET /search/instances: title all *1randomword	393	3403	416	3465	1.821922	3859	5471	3670	3984	-27.1797
MSF_GET /search/instances: title all 1randomword*	393	94	416	78	-17.0213	3860	144	3670	264	83.33333
MSF_GET /search/instances: title all 3 word sentence	393	40	416	38	-5	3867	101	3670	236	133.6634
MSF_GET /search/instances: title any 3 word sentence	393	408	416	395	-3.18627	3867	735	3670	653	-11.1565
Total	16901	194.1087	17895	185.1522	-4.61418	170397	365.1304	158131	382.1522	4.661824

Test Details

1

...

vuser - non-Eureka

transaction	Number Of Samples	Average	Median	pct90	pct95	pct99	Min	Max
MSF_GET /search/authorities keyword <> random sentence	393	207	206	241	252	266	171	266
MSF_GET /search/authorities keyword <> randomword	393	202	202	241	250	311	165	311
MSF_GET /search/authorities keyword = random sentence	393	1224	1220	1468	1481	1509	921	1509
MSF_GET /search/authorities keyword = random sentence	393	30	30	39	53	115	22	116
MSF_GET /search/authorities keyword = randomword	393	44	44	91	134	201	19	201
MSF_GET /search/authorities keyword == random sentence	393	29	28	40	46	64	21	65
MSF_GET /search/authorities keyword == randomword	393	43	44	112	136	175	19	175
MSF_GET /search/authorities keyword all randomword	393	891	895	1331	1343	1382	500	1382
MSF_GET /search/authorities keyword all random sentence	393	32	30	43	65	196	23	196
MSF_GET /search/authorities keyword all randomword	394	67	64	173	332	383	19	383
MSF_GET /search/authorities keyword any random sentence	393	99	83	218	305	310	23	310
MSF_GET /search/authorities keyword any randomword	394	55	55	140	152	244	18	244
MSF_GET /search/instances: contributors <> 2random words	393	158	158	183	188	225	131	225
MSF_GET /search/instances: contributors = *randomword	393	198	189	394	646	668	157	668
MSF_GET /search/instances: contributors = randomword*	393	57	48	210	221	263	17	264
MSF_GET /search/instances: contributors == 2random words	393	23	23	34	51	144	16	145
MSF_GET /search/instances: contributors all 2random words	393	23	23	32	44	103	17	103
MSF_GET /search/instances: contributors all randomword	393	58	58	159	225	396	17	398
MSF_GET /search/instances: contributors any 2random words	393	65	66	134	187	434	18	437
MSF_GET /search/instances: keyword all random sentence AND languages == randomlang	786	40	29	131	257	294	22	294
MSF_GET /search/instances: languages == (lang) AND items.status.name == ("Available" OR "In transit")	786	119	119	264	271	319	41	319
MSF_GET /search/instances: languages == lang	786	110	110	248	252	294	40	294
MSF_GET /search/instances: languages == lang AND items.status.name == "Available"	786	123	121	262	270	331	41	331
MSF_GET /search/instances: subjects <> 1randomword	393	70	69	115	159	228	18	229
MSF_GET /search/instances: subjects = (randomword OR randomword)	393	67	67	118	192	565	19	565
MSF_GET /search/instances: subjects = *1randomword	393	314	309	475	543	546	259	546
MSF_GET /search/instances: subjects = 1randomword*	393	70	63	208	309	322	17	322
MSF_GET /search/instances: subjects == 1randomword	393	40	40	73	94	258	16	259
MSF_GET /search/instances: subjects all 1randomword	393	47	46	91	124	442	18	444
MSF_GET /search/instances: subjects all randomword AND title all randomword	393	36	33	76	124	136	19	136
MSF_GET /search/instances: subjects all randomword NOT title all randomword	393	81	77	162	302	407	19	409
MSF_GET /search/instances: subjects all randomword OR title all randomword	393	96	96	199	298	670	30	673
MSF_GET /search/instances: subjects any 2random words	393	63	60	125	228	380	19	381
MSF_GET /search/instances: title <> 3 word sentence	393	167	167	197	207	218	138	218
MSF_GET /search/instances: title == 3 word sentence	393	36	29	134	200	218	21	218
MSF_GET /search/instances: title all *1randomword	393	3403	3401	4602	7838	9273	2276	9273
MSF_GET /search/instances: title all 1randomword*	393	94	87	230	270	722	22	727
MSF_GET /search/instances: title all 3 word sentence	393	40	31	135	153	186	20	186
MSF_GET /search/instances: title any 3 word sentence	393	408	384	1498	1611	1820	39	1823
MSF_TC: mod search by auth query fs09000000	393	2923	2895	3552	3735	3778	2040	3778
MSF_TC: mod search by boolean query fs09000000	393	280	274	550	616	1012	112	1014
MSF_TC: mod search by contributors query fs09000000	393	583	567	998	1149	1687	392	1696
MSF_TC: mod search by filter query fs09000000	393	396	377	886	1066	1080	150	1080
MSF_TC: mod search by keyword query fs09000000	393	394	378	864	974	990	151	990
MSF_TC: mod search by subject query fs09000000	393	603	587	903	944	1578	365	1583
MSF_TC: mod search by title query fs09000000	393	4149	4084	5985	8930	10027	2617	10027

1

...

vuser - Eureka

transaction	Number Of Samples	Average	Median	pct90	pct95	pct99	Min	Max
MSF_GET /search/authorities keyword <> random sentence	416	200	199	233	240	276	167	276
MSF_GET /search/authorities keyword <> randomword	417	194	192	227	268	293	161	293
MSF_GET /search/authorities keyword = random sentence	416	1142	1144	1352	1381	1412	913	1412
MSF_GET /search/authorities keyword = random sentence	416	23	22	26	84	98	18	98
MSF_GET /search/authorities keyword = randomword	417	32	31	79	104	127	14	127
MSF_GET /search/authorities keyword == random sentence	416	22	21	26	79	92	18	92
MSF_GET /search/authorities keyword == randomword	417	31	29	66	90	174	14	174
MSF_GET /search/authorities keyword all randomword	417	821	824	1216	1236	1345	492	1346
MSF_GET /search/authorities keyword all random sentence	417	25	23	31	151	181	19	181
MSF_GET /search/authorities keyword all randomword	417	54	48	152	234	544	15	548
MSF_GET /search/authorities keyword any random sentence	416	86	67	182	563	581	18	581
MSF_GET /search/authorities keyword any randomword	417	43	41	134	229	249	14	249
MSF_GET /search/instances: contributors <> 2random words	416	152	151	173	184	195	129	195
MSF_GET /search/instances: contributors = *randomword	416	192	181	271	670	676	158	676
MSF_GET /search/instances: contributors = randomword*	416	46	36	203	245	379	12	379
MSF_GET /search/instances: contributors == 2random words	416	17	16	34	58	224	12	225
MSF_GET /search/instances: contributors all 2random words	416	16	15	19	23	41	13	41
MSF_GET /search/instances: contributors all randomword	416	47	44	132	227	336	12	338
MSF_GET /search/instances: contributors any 2random words	416	51	51	110	177	407	13	410
MSF_GET /search/instances: keyword all random sentence AND languages == randomlang	832	35	22	178	372	510	17	510
MSF_GET /search/instances: languages == (lang) AND items.status.name == ("Available" OR "In transit")	832	113	111	259	373	375	38	375
MSF_GET /search/instances: languages == lang	832	104	103	242	247	255	36	255
MSF_GET /search/instances: languages == lang AND items.status.name == "Available"	832	114	113	252	259	314	37	314
MSF_GET /search/instances: subjects <> 1randomword	416	59	55	121	151	182	13	183
MSF_GET /search/instances: subjects = (randomword OR randomword)	416	52	51	98	121	309	14	310
MSF_GET /search/instances: subjects = *1randomword	416	318	306	481	843	870	264	870
MSF_GET /search/instances: subjects = 1randomword*	416	58	47	239	269	538	12	538
MSF_GET /search/instances: subjects == 1randomword	416	32	32	56	72	243	12	245
MSF_GET /search/instances: subjects all 1randomword	416	37	35	78	114	239	13	240
MSF_GET /search/instances: subjects all randomword AND title all randomword	416	27	22	87	108	141	14	141
MSF_GET /search/instances: subjects all randomword NOT title all randomword	416	67	57	164	261	277	15	278
MSF_GET /search/instances: subjects all randomword OR title all randomword	416	83	76	197	399	686	22	690
MSF_GET /search/instances: subjects any 2random words	416	52	47	102	182	361	14	362
MSF_GET /search/instances: title <> 3 word sentence	416	167	163	204	410	448	136	448
MSF_GET /search/instances: title == 3 word sentence	416	29	22	88	241	258	16	258
MSF_GET /search/instances: title all *1randomword	416	3465	3402	4696	7599	7650	2261	7650
MSF_GET /search/instances: title all 1randomword*	416	78	66	272	363	701	16	707
MSF_GET /search/instances: title all 3 word sentence	416	38	23	155	251	263	17	263
MSF_GET /search/instances: title any 3 word sentence	416	395	373	1534	1684	1792	32	1795
MSF_TC: mod search by auth query fs09000000	416	2673	2633	3179	3948	4137	2021	4137
MSF_TC: mod search by boolean query fs09000000	416	229	207	495	769	1002	89	1005
MSF_TC: mod search by contributors query fs09000000	416	521	497	807	1311	1566	361	1575
MSF_TC: mod search by filter query fs09000000	416	375	348	854	1303	1357	133	1357
MSF_TC: mod search by keyword query fs09000000	416	369	348	820	1289	1336	133	1336
MSF_TC: mod search by subject query fs09000000	416	555	525	933	1383	1427	353	1427
MSF_TC: mod search by title query fs09000000	416	4172	4034	6379	9095	9209	2558	9209

20

...

vusers - non-Eureka

transaction	NumberOf Samples	Average	Median	pct90	pct95	pct99	Min	Max	Error Count
MSF_GET /search/authorities keyword <> random sentence	3943	540	553	982	1317	2853	28	2868	5
MSF_GET /search/authorities keyword <> randomword	4104	516	532	934	1283	3170	27	3177	61
MSF_GET /search/authorities keyword = random sentence	3933	2120	2166	2965	3249	4537	37	4539	2
MSF_GET /search/authorities keyword = random sentence	3937	95	96	283	513	1499	20	1510	4
MSF_GET /search/authorities keyword = randomword	4246	127	130	422	666	2186	17	2199	142
MSF_GET /search/authorities keyword == random sentence	3949	92	95	285	485	1442	21	1452	6
MSF_GET /search/authorities keyword == randomword	4041	138	140	439	844	2044	18	2049	43
MSF_GET /search/authorities keyword all randomword	3997	1626	1668	2593	2876	4135	24	4148	17
MSF_GET /search/authorities keyword all random sentence	3979	104	106	338	565	2167	22	2173	21
MSF_GET /search/authorities keyword all randomword	5481	141	146	466	760	2589	16	2606	863
MSF_GET /search/authorities keyword any random sentence	3958	226	233	620	953	2603	22	2618	8
MSF_GET /search/authorities keyword any randomword	4618	129	132	426	758	2308	18	2324	372
MSF_GET /search/instances: contributors <> 2random words	3877	379	389	677	953	2098	75	2108	1
MSF_GET /search/instances: contributors = *randomword	3889	429	439	760	1099	2051	25	2062	1
MSF_GET /search/instances: contributors = randomword*	3892	101	104	343	666	1841	15	1852	3
MSF_GET /search/instances: contributors == 2random words	3880	78	79	268	475	1580	15	1592	2
MSF_GET /search/instances: contributors all 2random words	3876	85	87	289	511	1806	16	1816	2
MSF_GET /search/instances: contributors all randomword	3896	109	111	376	612	2466	15	2481	4
MSF_GET /search/instances: contributors any 2random words	3885	115	117	378	635	1942	17	1951	4
MSF_GET /search/instances: keyword all random sentence AND languages == randomlang	7747	98	100	305	530	2357	20	2374	3
MSF_GET /search/instances: languages == (lang) AND items.status.name == ("Available" OR "In transit")	7743	273	278	699	963	2421	40	2424
MSF_GET /search/instances: languages == lang	7743	260	265	677	919	2186	40	2190	1
MSF_GET /search/instances: languages == lang AND items.status.name == "Available"	7744	287	292	738	986	2456	41	2470
MSF_GET /search/instances: subjects <> 1randomword	3867	126	121	388	732	2354	17	2358
MSF_GET /search/instances: subjects = (randomword OR randomword)	3905	129	129	422	663	2617	18	2631	5
MSF_GET /search/instances: subjects = *1randomword	3868	645	659	1014	1405	3126	261	3133
MSF_GET /search/instances: subjects = 1randomword*	3869	113	114	361	642	2184	16	2192	1
MSF_GET /search/instances: subjects == 1randomword	3868	123	124	410	707	2189	16	2195	1
MSF_GET /search/instances: subjects all 1randomword	3869	124	127	392	607	2507	16	2519
MSF_GET /search/instances: subjects all randomword AND title all randomword	3930	98	99	326	590	2195	17	2211	17
MSF_GET /search/instances: subjects all randomword NOT title all randomword	3900	125	127	374	615	1971	18	1980	4
MSF_GET /search/instances: subjects all randomword OR title all randomword	3913	165	169	465	761	2590	27	2604	8
MSF_GET /search/instances: subjects any 2random words	3868	133	138	399	663	2313	19	2325
MSF_GET /search/instances: title <> 3 word sentence	3863	403	411	737	1022	2582	30	2596	3
MSF_GET /search/instances: title == 3 word sentence	3866	93	93	301	541	1697	18	1705	3
MSF_GET /search/instances: title all *1randomword	3859	5471	5549	8145	8699	10240	26	10241	4
MSF_GET /search/instances: title all 1randomword*	3860	144	147	410	654	2262	20	2278	1
MSF_GET /search/instances: title all 3 word sentence	3867	101	102	336	535	1924	19	1932
MSF_GET /search/instances: title any 3 word sentence	3867	735	741	2713	3093	4191	34	4203
MSF_TC: mod search by auth query fs09000000	5475	5724	6037	7625	8208	10745	24	10772	1544
MSF_TC: mod search by boolean query fs09000000	3930	515	536	1199	1765	3572	27	3587	34
MSF_TC: mod search by contributors query fs09000000	3891	1295	1339	2194	2721	4735	30	4750	17
MSF_TC: mod search by filter query fs09000000	3874	917	944	2108	2543	4522	147	4540	1
MSF_TC: mod search by keyword query fs09000000	3872	913	937	2115	2485	4516	35	4530	3
MSF_TC: mod search by subject query fs09000000	3869	1262	1291	2177	2853	5627	379	5640	2
MSF_TC: mod search by title query fs09000000	3866	6939	7072	10201	11284	14160	355	14171	11

20

...

vusers - Eureka

transaction	Number Of Samples	Average	Median	pct90	pct95	pct99	Min	Max
MSF_GET /search/authorities keyword <> random sentence	3682	477	487	862	1148	2214	175	2400
MSF_GET /search/authorities keyword <> randomword	3684	462	472	850	1108	2423	166	2848
MSF_GET /search/authorities keyword = random sentence	3682	1545	1555	2153	2548	4332	941	4350
MSF_GET /search/authorities keyword = random sentence	3682	245	252	653	993	2623	19	2635
MSF_GET /search/authorities keyword = randomword	3684	234	236	608	973	3428	16	3451
MSF_GET /search/authorities keyword == random sentence	3683	225	233	591	885	1783	20	1792
MSF_GET /search/authorities keyword == randomword	3683	244	252	625	965	3221	16	3239
MSF_GET /search/authorities keyword all randomword	3683	1191	1206	1816	2117	4118	518	4139
MSF_GET /search/authorities keyword all random sentence	3683	243	251	631	936	3322	21	3341
MSF_GET /search/authorities keyword all randomword	3686	262	266	660	1035	2643	17	2652
MSF_GET /search/authorities keyword any random sentence	3683	272	279	603	902	2910	20	2920
MSF_GET /search/authorities keyword any randomword	3685	232	237	592	947	3309	16	3332
MSF_GET /search/instances: contributors <> 2random words	3679	408	418	786	1073	2685	133	2702
MSF_GET /search/instances: contributors = *randomword	3680	441	451	844	1064	2958	162	2969
MSF_GET /search/instances: contributors = randomword*	3680	227	234	561	882	3076	14	3094
MSF_GET /search/instances: contributors == 2random words	3679	217	226	594	902	2062	14	2073
MSF_GET /search/instances: contributors all 2random words	3677	231	239	613	952	3320	14	3338
MSF_GET /search/instances: contributors all randomword	3680	232	239	592	973	3142	14	3157
MSF_GET /search/instances: contributors any 2random words	3680	244	252	622	984	2020	15	2030
MSF_GET /search/instances: keyword all random sentence AND languages == randomlang	7354	233	237	614	901	3484	19	3511
MSF_GET /search/instances: languages == (lang) AND items.status.name == ("Available" OR "In transit")	7350	346	356	772	1084	3477	39	3501
MSF_GET /search/instances: languages == lang	7349	344	350	797	1114	3435	38	3458
MSF_GET /search/instances: languages == lang AND items.status.name == "Available"	7350	353	360	775	1085	3426	39	3449
MSF_GET /search/instances: subjects <> 1randomword	3670	338	255	926	1177	2866	14	2880
MSF_GET /search/instances: subjects = (randomword OR randomword)	3682	247	256	609	918	2879	16	2896
MSF_GET /search/instances: subjects = *1randomword	3672	601	610	1032	1365	2939	268	2940
MSF_GET /search/instances: subjects = 1randomword*	3672	235	241	589	877	3489	14	3509
MSF_GET /search/instances: subjects == 1randomword	3670	232	238	595	1004	2564	15	2573
MSF_GET /search/instances: subjects all 1randomword	3672	241	248	635	898	3062	16	3082
MSF_GET /search/instances: subjects all randomword AND title all randomword	3682	237	243	620	976	2919	13	2935
MSF_GET /search/instances: subjects all randomword NOT title all randomword	3681	235	242	580	853	2493	16	2508
MSF_GET /search/instances: subjects all randomword OR title all randomword	3682	261	270	637	998	2581	26	2582
MSF_GET /search/instances: subjects any 2random words	3670	255	264	640	1031	2757	15	2759
MSF_GET /search/instances: title <> 3 word sentence	3670	415	424	770	1087	2284	140	2295
MSF_GET /search/instances: title == 3 word sentence	3670	237	244	634	1073	2361	18	2367
MSF_GET /search/instances: title all *1randomword	3670	3984	3988	5754	6226	7451	2260	7453
MSF_GET /search/instances: title all 1randomword*	3670	264	273	678	977	2829	17	2839
MSF_GET /search/instances: title all 3 word sentence	3670	236	243	628	891	3082	18	3091
MSF_GET /search/instances: title any 3 word sentence	3670	653	642	1974	2483	5288	36	5311
MSF_TC: mod search by auth query fs09000000	3682	5627	5750	7399	8206	10533	2217	10551
MSF_TC: mod search by boolean query fs09000000	3681	980	1015	1870	2341	4271	107	4272
MSF_TC: mod search by contributors query fs09000000	3677	1998	2069	3160	3756	6881	423	6898
MSF_TC: mod search by filter query fs09000000	3677	1267	1310	2360	2812	5026	144	5043
MSF_TC: mod search by keyword query fs09000000	3672	1273	1315	2334	2785	6416	142	6444
MSF_TC: mod search by subject query fs09000000	3670	1900	1881	3226	3637	5916	391	5920
MSF_TC: mod search by title query fs09000000	3670	5787	5851	8198	9116	11512	2621	11530

Resource Utilization

Memory Utilization

1

...

vuser - non-Eureka

1

...

vuser - Eureka

20

...

vusers - non-Eureka

20

...

vusers - Eureka

Service CPU Utilization

1

...

vuser - non-Eureka

1

...

vuser - Eureka

20

...

vusers - non-Eureka

20

...

vusers - Eureka

RDS CPU Utilization

1

...

vuser - non-Eureka

1

...

vuser - Eureka

20

...

vusers - non-Eureka

20

...

vusers - Eureka

RDS Connections count

1

...

vuser - non-Eureka

1

...

vuser - Eureka

20

...

vusers - non-Eureka

20

...

vusers - Eureka

OpenSearch metrics

Additional information

environments: qcp1-non-Eureka, qecp1-Eureka

Different permutations of random words and sentences were generated from https://www.mit.edu/~ecprice/wordlist.10000

...

Appendix

Infrastructure

PTF environment qecp1
11 m6i.2xlarge EC2 instances located in US East (N. Virginia)us-east-1 1 instance of db.r6.xlarge database instance: Writer instance MSK tenant [ kafka configurations] 4 kafka.m5.2xlarge brokers in 2 zones Apache Kafka version 2.8.0 EBS storage volume per broker 300 GiB auto.create.topics.enable=true log.retention.minutes=480 default.replication.factor=3

DB table records size:

fs09000000
- instances - 24493326
- items - 25662862
- holding records - 24459683

Files

Modules

Expand

title	All modules

Module	Task Definition Revision	Module Version	Task Count	Mem Hard Limit	Mem Soft Limit	CPU Units	Xmx	Metaspace Size	Max Metaspace Size
mod-remote-storage	3	mod-remote-storage:3.2.0	2	4920	4472	0	3960	512	512
mod-remote-storage - Sidecar 1	N/A	folio-module-sidecar:1.0.0.312	N/A	665	512	0	128	0	0
mod-ncip	3	mod-ncip:1.14.5	2	1024	896	0	768	88	128
mod-ncip - Sidecar 1	N/A	folio-module-sidecar:1.0.0.312	N/A	665	512	0	128	0	0
mod-finance-storage	3	mod-finance-storage:8.6.1	2	1024	896	0	700	88	128
mod-finance-storage - Sidecar 1	N/A	folio-module-sidecar:1.0.0.312	N/A	665	512	0	128	0	0
mod-agreements	3	mod-agreements:7.0.6	2	1592	1488	0	0	0	0
mod-agreements - Sidecar 1	N/A	folio-module-sidecar:1.0.0.312	N/A	665	512	0	128	0	0
mod-ebsconet	3	mod-ebsconet:2.2.0	2	1248	1024	0	700	128	256
mod-ebsconet - Sidecar 1	N/A	folio-module-sidecar:1.0.0.312	N/A	665	512	0	128	0	0
mod-organizations	3	mod-organizations:1.9.2	2	1024	896	0	768	88	128
mod-organizations - Sidecar 1	N/A	folio-module-sidecar:1.0.0.312	N/A	665	512	0	128	0	0
edge-sip2	2	edge-sip2:3.2.5	2	1024	896	0	768	88	128
mod-settings	3	mod-settings:1.0.3	2	1024	896	0	768	88	128
mod-settings - Sidecar 1	N/A	folio-module-sidecar:1.0.0.312	N/A	665	512	0	128	0	0
mod-serials-management	3	mod-serials-management:1.0.3	2	2480	2312	0	1792	384	512
mod-serials-management - Sidecar 1	N/A	folio-module-sidecar:1.0.0.312	N/A	665	512	0	128	0	0
mod-search	3	mod-search:3.2.6	2	2592	2480	0	1440	512	1024
mod-search - Sidecar 1	N/A	folio-module-sidecar:1.0.0.312	N/A	665	512	0	128	0	0
mod-data-import	3	mod-data-import:3.1.1	1	2048	1844	0	1292	384	512
mod-data-import - Sidecar 1	N/A	folio-module-sidecar:1.0.0.312	N/A	665	512	0	128	0	0
edge-dematic	2	edge-dematic:2.2.3	1	1024	896	0	768	88	128
mod-tags	3	mod-tags:2.2.0	2	1024	896	0	768	88	128
mod-tags - Sidecar 1	N/A	folio-module-sidecar:1.0.0.312	N/A	665	512	0	128	0	0
edge-courses	2	edge-courses:1.4.3	2	1024	896	0	768	88	128
mod-notify	3	mod-notify:3.2.0	2	1024	896	0	768	88	128
mod-notify - Sidecar 1	N/A	folio-module-sidecar:1.0.0.312	N/A	665	512	0	128	0	0
mod-inventory-update	3	mod-inventory-update:3.3.1	2	1024	896	0	768	88	128
mod-inventory-update - Sidecar 1	N/A	folio-module-sidecar:1.0.0.312	N/A	665	512	0	128	0	0
mod-configuration	3	mod-configuration:5.10.0	2	1024	896	0	768	88	128
mod-configuration - Sidecar 1	N/A	folio-module-sidecar:1.0.0.312	N/A	665	512	0	128	0	0
mod-orders-storage	3	mod-orders-storage:13.7.4	2	1024	896	0	700	88	128
mod-orders-storage - Sidecar 1	N/A	folio-module-sidecar:1.0.0.312	N/A	665	512	0	128	0	0
folio-keycloak	3	folio-keycloak:25.0.3	1	2400	2048	0	2048	0	0
edge-caiasoft	2	edge-caiasoft:2.2.4	2	1024	896	0	768	88	128
mod-licenses	3	mod-licenses:6.0.3	2	2480	2312	0	1792	384	512
mod-licenses - Sidecar 1	N/A	folio-module-sidecar:1.0.0.312	N/A	665	512	0	128	0	0
mod-gobi	3	mod-gobi:2.8.1	2	1024	896	0	768	88	128
mod-gobi - Sidecar 1	N/A	folio-module-sidecar:1.0.0.312	N/A	665	512	0	128	0	0
mod-password-validator	3	mod-password-validator:3.2.1	2	1440	1298	0	768	384	512
mod-password-validator - Sidecar 1	N/A	folio-module-sidecar:1.0.0.312	N/A	665	512	0	128	0	0
mod-fqm-manager	3	mod-fqm-manager:2.0.5	0	1024	896	0	768	88	128
mod-fqm-manager - Sidecar 1	N/A	folio-module-sidecar:1.0.0.312	N/A	665	512	0	128	0	0
mod-bulk-operations	3	mod-bulk-operations:2.0.2	2	3072	2600	0	1536	384	512
mod-bulk-operations - Sidecar 1	N/A	folio-module-sidecar:1.0.0.312	N/A	665	512	0	128	0	0
mod-graphql	3	mod-graphql:1.12.1	2	1024	896	0	768	88	128
mod-graphql - Sidecar 1	N/A	folio-module-sidecar:1.0.0.312	N/A	665	512	0	128	0	0
mod-finance	3	mod-finance:4.9.0	2	1024	896	0	768	88	128
mod-finance - Sidecar 1	N/A	folio-module-sidecar:1.0.0.312	N/A	665	512	0	128	0	0
mod-roles-keycloak	3	mod-roles-keycloak:1.4.5	2	1024	896	0	512	88	256
mod-roles-keycloak - Sidecar 1	N/A	folio-module-sidecar:1.0.0.312	N/A	665	512	0	128	0	0
mod-batch-print	3	mod-batch-print:1.1.0	2	1024	896	0	768	88	128
mod-batch-print - Sidecar 1	N/A	folio-module-sidecar:1.0.0.312	N/A	665	512	0	128	0	0
mod-copycat	3	mod-copycat:1.6.0	2	1024	896	0	768	88	128
mod-copycat - Sidecar 1	N/A	folio-module-sidecar:1.0.0.312	N/A	665	512	0	128	0	0
mod-lists	4	mod-lists:2.0.6	2	1024	896	0	768	88	128
mod-lists - Sidecar 1	N/A	folio-module-sidecar:1.0.0.312	N/A	665	512	0	128	0	0
mod-entities-links	3	mod-entities-links:3.0.2	2	2592	2480	0	1440	0	1024
mod-entities-links - Sidecar 1	N/A	folio-module-sidecar:1.0.0.312	N/A	665	512	0	128	0	0
mod-permissions	6	mod-permissions:6.5.0	2	1684	1544	512	1024	384	512
mod-permissions - Sidecar 1	N/A	folio-module-sidecar:1.0.0.312	N/A	665	512	0	128	0	0
pub-edge	1	pub-edge:2023.06.14	2	1024	896	0	768	0	0
mod-orders	3	mod-orders:12.8.8	2	2048	1740	0	1024	384	512
mod-orders - Sidecar 1	N/A	folio-module-sidecar:1.0.0.312	N/A	665	512	0	128	0	0
kong	9	kong:3.7.1	1	6144	6144	2048	768	88	128
edge-patron	2	edge-patron:5.1.2	2	1024	896	256	768	88	128
mod-users-keycloak	3	mod-users-keycloak:1.5.3	2	1024	896	0	512	88	256
mod-users-keycloak - Sidecar 1	N/A	folio-module-sidecar:1.0.0.312	N/A	665	512	0	128	0	0
edge-ncip	2	edge-ncip:1.10.1	2	1024	896	0	768	88	128
mgr-applications	2	mgr-applications:1.3.3	1	1024	896	512	512	88	256
mod-users-bl	3	mod-users-bl:7.8.0	2	1440	1152	0	922	88	128
mod-users-bl - Sidecar 1	N/A	folio-module-sidecar:1.0.0.312	N/A	665	512	0	128	0	0
mgr-tenants	2	mgr-tenants:1.3.3	1	1024	896	512	512	88	256
mod-invoice	3	mod-invoice:5.8.2	2	1440	1152	0	922	88	128
mod-invoice - Sidecar 1	N/A	folio-module-sidecar:1.0.0.312	N/A	665	512	0	128	0	0
mod-inventory-storage	4	mod-inventory-storage:27.1.4	2	4096	3690	0	3076	384	512
mod-inventory-storage - Sidecar 1	N/A	folio-module-sidecar:1.0.0.312	N/A	665	512	0	256	0	0
mod-user-import	3	mod-user-import:3.8.0	2	1024	896	0	768	88	128
mod-user-import - Sidecar 1	N/A	folio-module-sidecar:1.0.0.312	N/A	665	512	0	128	0	0
mod-sender	3	mod-sender:1.12.0	2	1024	896	0	768	88	128
mod-sender - Sidecar 1	N/A	folio-module-sidecar:1.0.0.312	N/A	665	512	0	128	0	0
edge-oai-pmh	2	edge-oai-pmh:2.9.2	2	1512	1360	1024	1440	384	512
mod-login-keycloak	8	mod-login-keycloak:1.4.4	2	1024	896	0	512	88	256
mod-login-keycloak - Sidecar 1	N/A	folio-module-sidecar:1.0.0.312	N/A	665	512	0	128	0	0
mgr-tenant-entitlements	2	mgr-tenant-entitlements:1.2.4	1	1024	896	512	512	88	256
mod-data-export-worker	3	mod-data-export-worker:3.2.4	2	3072	2800	0	2048	384	512
mod-data-export-worker - Sidecar 1	N/A	folio-module-sidecar:1.0.0.312	N/A	665	512	0	128	0	0
mod-rtac	4	mod-rtac:3.6.0	2	1024	896	0	768	88	128
mod-rtac - Sidecar 1	N/A	folio-module-sidecar:1.0.0.312	N/A	665	512	0	256	0	0
mod-circulation-storage	4	mod-circulation-storage:17.2.3	2	2880	2592	0	1814	384	512
mod-circulation-storage - Sidecar 1	N/A	folio-module-sidecar:1.0.0.312	N/A	665	512	0	256	0	0
mod-source-record-storage	4	mod-source-record-storage:5.8.6	2	5600	5000	0	3500	384	512
mod-source-record-storage - Sidecar 1	N/A	folio-module-sidecar:1.0.0.312	N/A	665	512	0	128	0	0
mod-scheduler	3	mod-scheduler:1.2.3	2	1024	896	0	512	88	256
mod-scheduler - Sidecar 1	N/A	folio-module-sidecar:1.0.0.312	N/A	665	512	0	128	0	0
mod-calendar	4	mod-calendar:3.1.0	2	2048	1536	0	768	88	128
mod-calendar - Sidecar 1	N/A	folio-module-sidecar:1.0.0.317	N/A	665	512	0	128	0	0
mod-event-config	3	mod-event-config:2.7.1	2	1024	896	0	768	88	128
mod-event-config - Sidecar 1	N/A	folio-module-sidecar:1.0.0.312	N/A	665	512	0	128	0	0
mod-courses	3	mod-courses:1.4.10	2	1024	896	0	768	88	128
mod-courses - Sidecar 1	N/A	folio-module-sidecar:1.0.0.312	N/A	665	512	0	128	0	0
mod-inventory	5	mod-inventory:20.2.7	2	2881	2592	0	1814	384	512
mod-inventory - Sidecar 1	N/A	folio-module-sidecar:1.0.0.312	N/A	665	512	0	128	0	0
mod-email	3	mod-email:1.17.0	2	2800	2550	0	1800	384	512
mod-email - Sidecar 1	N/A	folio-module-sidecar:1.0.0.312	N/A	665	512	0	128	0	0
mod-di-converter-storage	3	mod-di-converter-storage:2.2.2	2	1024	896	0	768	88	128
mod-di-converter-storage - Sidecar 1	N/A	folio-module-sidecar:1.0.0.312	N/A	665	512	0	128	0	0
mod-pubsub	3	mod-pubsub:2.14.0	2	1536	1440	0	922	384	512
mod-pubsub - Sidecar 1	N/A	folio-module-sidecar:1.0.0.312	N/A	665	512	0	128	0	0
mod-circulation	4	mod-circulation:24.2.5	2	2880	2592	0	1814	384	512
mod-circulation - Sidecar 1	N/A	folio-module-sidecar:1.0.0.312	N/A	665	512	0	256	0	0
edge-orders	2	edge-orders:3.0.3	2	1024	896	0	768	88	128
edge-rtac	2	edge-rtac:2.7.3	2	1024	896	0	768	88	128
mod-template-engine	3	mod-template-engine:1.20.0	2	1024	896	0	768	88	128
mod-template-engine - Sidecar 1	N/A	folio-module-sidecar:1.0.0.312	N/A	665	512	0	128	0	0
mod-users	3	mod-users:19.3.1	2	1024	896	0	768	88	128
mod-users - Sidecar 1	N/A	folio-module-sidecar:1.0.0.312	N/A	665	512	0	128	0	0
mod-patron-blocks	3	mod-patron-blocks:1.10.0	2	1024	896	0	768	88	128
mod-patron-blocks - Sidecar 1	N/A	folio-module-sidecar:1.0.0.312	N/A	665	512	0	128	0	0
mod-audit	3	mod-audit:2.9.0	2	1024	896	0	768	88	128
mod-audit - Sidecar 1	N/A	folio-module-sidecar:1.0.0.312	N/A	665	512	0	128	0	0
edge-fqm	2	edge-fqm:2.0.2	2	1024	896	0	768	88	128
mod-source-record-manager	3	mod-source-record-manager:3.8.6	2	5600	5000	0	3500	384	512
mod-source-record-manager - Sidecar 1	N/A	folio-module-sidecar:1.0.0.312	N/A	665	512	0	128	0	0
nginx-edge	1	nginx-edge:2023.06.14	1	1024	896	0	0	0	0
mod-quick-marc	3	mod-quick-marc:5.1.1	1	2288	2176	0	1664	384	512
mod-quick-marc - Sidecar 1	N/A	folio-module-sidecar:1.0.0.312	N/A	665	512	0	128	0	0
mod-feesfines	3	mod-feesfines:19.1.0	2	1024	896	0	768	88	128
mod-feesfines - Sidecar 1	N/A	folio-module-sidecar:1.0.0.312	N/A	665	512	0	128	0	0
mod-invoice-storage	3	mod-invoice-storage:5.8.2	2	1872	1536	0	1024	384	512
mod-invoice-storage - Sidecar 1	N/A	folio-module-sidecar:1.0.0.312	N/A	665	512	0	128	0	0
mod-service-interaction	3	mod-service-interaction:4.0.2	2	2048	1844	0	1290	384	512
mod-service-interaction - Sidecar 1	N/A	folio-module-sidecar:1.0.0.312	N/A	665	512	0	128	0	0
mod-data-export	11	mod-data-export:5.0.4	1	2592	2480	0	1440	88	1024
mod-data-export - Sidecar 1	N/A	folio-module-sidecar:1.0.0.312	N/A	665	512	0	128	0	0
mod-patron	3	mod-patron:6.1.0	2	1024	896	0	768	88	128
mod-patron - Sidecar 1	N/A	folio-module-sidecar:1.0.0.312	N/A	665	512	0	128	0	0
mod-oai-pmh	3	mod-oai-pmh:3.13.2	2	4096	3690	0	3076	384	512
mod-oai-pmh - Sidecar 1	N/A	folio-module-sidecar:1.0.0.312	N/A	665	512	0	128	0	0
edge-connexion	2	edge-connexion:1.3.1	2	1024	896	0	768	88	128
mod-kb-ebsco-java	3	mod-kb-ebsco-java:4.0.0	2	1024	896	0	768	88	128
mod-kb-ebsco-java - Sidecar 1	N/A	folio-module-sidecar:1.0.0.312	N/A	665	512	0	128	0	0
mod-notes	3	mod-notes:5.2.0	2	1024	896	0	952	384	512
mod-notes - Sidecar 1	N/A	folio-module-sidecar:1.0.0.312	N/A	665	512	0	128	0	0
mod-organizations-storage	3	mod-organizations-storage:4.7.0	2	1024	896	0	768	88	128
mod-organizations-storage - Sidecar 1	N/A	folio-module-sidecar:1.0.0.312	N/A	665	512	0	128	0	0
mod-data-export-spring	3	mod-data-export-spring:3.2.2	1	2048	1844	0	1536	384	512
mod-data-export-spring - Sidecar 1	N/A	folio-module-sidecar:1.0.0.312	N/A	665	512	0	128	0	0
edge-erm	2	edge-erm:1.2.1	2	1024	896	0	768	88	128

Additional information

environments: qcp1-non-Eureka, qecp1-Eureka

Different permutations of random words and sentences were generated from https://www.mit.edu/~ecprice/wordlist.10000

Artifact stored http://github.com/folio-org/perf-testing/mod-search
test performed for 1 vuser ramp-up 1 sec
test performed for 20 vusers ramp-up 200 sec
runtime 60 minutes

Ran Run mod-search load testing for the following variant of queries:

...

Page Comparison

Versions Compared

Old Version 12

New Version 13

Key

Overview

Recommendations & Jiras

Test Runs

Results and Comparison Eureka and non-Eureka

Test Details

1

vuser - non-Eureka

1

vuser - Eureka

20

vusers - non-Eureka

20

vusers - Eureka

Resource Utilization

Memory Utilization

1

vuser - non-Eureka

1

vuser - Eureka

20

vusers - non-Eureka

20

vusers - Eureka

Service CPU Utilization

1

vuser - non-Eureka

1

vuser - Eureka

20

vusers - non-Eureka

20

vusers - Eureka

RDS CPU Utilization

1

vuser - non-Eureka

1

vuser - Eureka

20

vusers - non-Eureka

20

vusers - Eureka

RDS Connections count

1

vuser - non-Eureka

1

vuser - Eureka

20

vusers - non-Eureka

20

vusers - Eureka

OpenSearch metrics

Additional information

Appendix

Infrastructure

Additional information