Overview

This document contains performance testing results comparing new version OpenSearch 3.5 with OpenSearch 2.19 in the Sunflower Eureka consortium (SECON) environment.
Jira PERF-1380: Verify FOLIO Performance with OpenSearch v3Closed

Summary

OpenSearch v3.5 delivers equal or better performance compared to v2.19 with no stability or resource concerns.
Performance verification was conducted on OpenSearch version 3.5, with baseline tests executed on OpenSearch version 2.19 for comparison.
The inventory search workflow was tested with 1, 10, and 20 concurrent users.
Data Import tested with 25K instances (Marc BIB Create and Update).
Comparison of results between OpenSearch v3.5 and OpenSearch v2.19 shows minor differences for the inventory search:
- Response times on OpenSearch v3.5 are equal to or faster across all load levels, with improvements in average 8% for most query types;
- mod-search CPU was slightly lower on v3.5: ~1% (1 user), ~4% (10 users), ~20% (20 users);
- Search latency averaged ~4 ms on both versions across all tests; minor spikes were observed only during 1-user tests — up to 8 ms on v2.19 vs up to 5 ms on v3.5.
Data Import showed reduced execution time on OpenSearch v3.5:
- 25K Creates: 1 min 15 sec faster;
- 25K Updates: 4 min 13 sec faster;
- While maintaining a similar or slightly higher resource profile;
- In both versions, indexing latency remained around ~1 ms during DI.

Test	Test description	Status
Test 1-2	Inventory search test with 1 user	Completed
Test 3-4	Inventory search test with 10 users	Completed
Test 5-6	Inventory search test with 20 users	Completed
Test 7	Data Import Create using “PTF - Create 2“ profile	Completed
Test 8	Data Import Update using “PTF - Update Success - 6“ profile	Completed

This table contains inventory search tests Response Times:

OpenSearch 2.19
	1 user - Test 1	1 user - Test 2	10 users - Test 1	10 users - Test 2	20 users - Test 1	20 users - Test 2
Transaction	Average Response Times (ms)
TC_main_MSF: mod search by title query	467	463	397	384	402	396
TC_main_MSF: mod search by subject query	407	324	269	259	289	266
TC_main_MSF: mod search by notes query	527	512	504	503	510	509
TC_main_MSF: mod search by keyword query	314	311	317	313	320	322
TC_main_MSF: mod search by identifier query	50	53	45	49	48	48
TC_main_MSF: mod search by filter query	321	311	314	321	321	321
TC_main_MSF: mod search by contributors query	359	313	287	282	295	294
TC_main_MSF: mod search by classification query	43	42	41	42	42	44
TC_main_MSF: mod search by boolean query	241	187	156	151	159	157
TC_main_MSF: mod search by auth query	781	715	682	662	693	695

OpenSearch v3.5
	1 user - Test 1	1 user - Test 2	10 users - Test 1	10 users - Test 2	20 users - Test 1	20 users - Test 2
Transaction	Average Response Times (ms)
TC_main_MSF: mod search by title query	372	352	339	323	337	322
TC_main_MSF: mod search by subject query	337	284	254	289	258	267
TC_main_MSF: mod search by notes query	495	477	483	491	490	484
TC_main_MSF: mod search by keyword query	250	251	270	255	262	256
TC_main_MSF: mod search by identifier query	61	47	49	47	52	48
TC_main_MSF: mod search by filter query	257	258	253	253	263	257
TC_main_MSF: mod search by contributors query	295	298	256	256	267	255
TC_main_MSF: mod search by classification query	39	44	37	39	40	38
TC_main_MSF: mod search by boolean query	203	179	144	138	142	140
TC_main_MSF: mod search by auth query	825	784	792	788	820	792

This table contains DI durations:

Comparison between tests with OpenSearch version 2.19 and OpenSearch version 3.5:

	1 user - Test 1	1 user - Test 2	10 users - Test 1	10 users - Test 2	20 users - Test 1	20 users - Test 2
Transaction	Average Response Times (ms)
TC_main_MSF: mod search by title query	-95	-111	-58	-61	-65	-74
TC_main_MSF: mod search by subject query	-70	-40	-15	30	-31	1
TC_main_MSF: mod search by notes query	-32	-35	-21	-12	-20	-25
TC_main_MSF: mod search by keyword query	-64	-60	-47	-58	-58	-66
TC_main_MSF: mod search by identifier query	11	-6	4	-2	4	0
TC_main_MSF: mod search by filter query	-64	-53	-61	-68	-58	-64
TC_main_MSF: mod search by contributors query	-64	-15	-31	-26	-28	-39
TC_main_MSF: mod search by classification query	-4	2	-4	-3	-2	-6
TC_main_MSF: mod search by boolean query	-38	-8	-12	-13	-17	-17
TC_main_MSF: mod search by auth query	44	69	110	126	127	97

Number of records	DI Create, delta	DI Update, delta
25,000	- 1 min 15 sec	- 4 min 13 sec

Resource utilization comparison for inventory search tests:

OpenSearch v2.19	OpenSearch v3.5

OpenSearch v2.19	OpenSearch v3.5
Service CPU Utilization mod-search is the top CPU consumer. OpenSearch v3.5 showed slightly lower CPU usage compared to v2.19 across all load levels: ~37% vs 38% for 1 user, ~116% vs 120% for 10 users, and ~190% vs 210% for 20 concurrent users.