Jira Legacy

server	System JiraJIRA
serverId	01505d01-b853-3c2e-90f1-ee9b165564fc
key	MSEARCH-300

5 users call number browsing with random words.

Test

...

10 users call number browsing with random words

...

.

Test 20 users call number browsing with random words.

Test 50 users call number browsing with

...

random words.

Test with long ramp-up period, 1 minute per user from 0 to 25 users

Test 50 users call number browsing + DI + CICO

Summary

ALL tests (1-50 users) ended without errors. Average response time . most resource intensive is mod-search. During test 50 users call number browsing CPU usage reached more than 300%.
During tests 50 users with background activities (CICO+ Data Import) response time increased up to 10%-15%.

...

Image Removed

Anchortest01test011 user Image Removed

Anchor

	env
	env

Environment

mod-search 1.6.4
OpenSearch service
- nodes type r6g.large.search (X4) * (4 data nodes 3 dedicated master nodes);
- version OpenSearch 1.2 (latest);
kafka 2.8.0

Anchor

	summary
	summary

Summary

ALL tests (1-50 users) ended without errors. Average response time . most resource intensive is mod-search. During test 50 users call number browsing CPU usage reached more than 300%.
During tests 50 users with background activities (CICO+ Data Import) response time increased up to 10%-15%.

number of users	average rt	% CPU mod-search
1	0.148	12
5	0.168	42
10	0.191	75
20	0.321	122
50	1.821	158

Image Added Image Added

Anchor

	test01
	test01

1 user

Image Added

Requests	Total	OK	KO	% KO	Req/s	Min	50th pct	75th pct	95th pct	99th pct	Max	Average	Latency
GET_browse _call_numbers	1747	1747	0	0	2.220	0.053	0.142	0.166	0.255	0.308	0.687	0.148	0.251

Image Added

Anchor

	test05
	test05

5 users

Requests	Total	OK	KO	% KO	Req/s	Min	50th pct	75th pct	95th pct	99th pct	Max	Average	Latency
GET_browse _call_numbers	8219	8219	0	0	10.693	0.048	0.150	0.188	0.295	0.419	0.920	0.168	0.292

Image Added

Anchor

	test10
	test10

10 users

Requests	Total	OK	KO	% KO	Req/s	Min	50th pct	75th pct	95th pct	99th pct	Max	Average	Latency
GET_browse _call_numbers	12741	12741	0	0	20.313	0.051	0.161	0.233	0.387	0.584	1.437	0.191	0.384

Image Added

20 users

Requests	Total	OK	KO	% KO	Req/s	Min	50th pct	75th pct	95th pct	99th pct	Max	Average	Latency
GET_browse _call_numbers	12193	12193	0	0	32.169	0.052	0.245	0.441	0.739	1.025	2.441	0.321	0.736

Image Added

50 users

Requests	Total	OK	KO	% KO	Req/s	Min	50th pct	75th pct	95th pct	99th pct	Max	Average	Latency
GET_browse _call_numbers	7849	7849	0	0	38.485	0.081	0.963	1.259	1.823	2.182	3.275	0.996	1.821

Image Added

Anchor

	test1
	test1

50 users 30 min (10 min ramp up)

Image Added

Anchor

	test2
	test2

50 users + DI in background

Image Added

Anchor

	test3
	test3

50 users +DI 5K in background

Image Added

RTT without background activity

Requests	Total	OK	KO	% KO	Req/s	Min	50th pct	75th pct	95th pct	99th pct	Max	Average	Latency
GET_browse _call_numbers

17471747002.2200.0530.1420.1660.2550.3080.6870.1480.251

Image Removed

5 users

RequestsTotalOKKO% KOReq/sMin50th pct75th pct95th pct99th pctMaxAverageLatency

"BT83.57 .N625 2013"	10276	10276	0	0	9.349	0.046	0.161	0.234	0.401	0.779	1.547	0.192	0.401
GET_browse _call_numbers

82198219

"CC76.4 .C66 2013"

10275

0

10

9.

693

352

0.

048

046

0.

150

169

0.

188

243

0.

295

402

0.

419

806

0

1.

920

736

0.

168

199

0.

292

Image Removed

10 users

RequestsTotalOKKO% KOReq/sMin50th pct75th pct95th pct99th pctMaxAverageLatency

402

GET_browse _call_numbers

1274112741

"DT107.827.L32 1987"

10277

0

20

9.

313

346

0.

051

050

0.

161

169

0.

233

242

0.

387

400

0.

584

810

1.

437

721

0.

191

198

0.

384

Image Removed

20 users

RequestsTotalOKKO% KOReq/sMin50th pct75th pct95th pct99th pctMaxAverageLatency

399
GET_browse _call_numbers "G3893.M5 1923 .W4"	10275	10275	0	0	9.336	0.053	0.173	0.248	0.407	0.821	1.753	0.203	0.407
GET_browse _call_numbers

12193121930032.169

"LC4019 .H83 2012"

10275

0

9.355

0.056

0.

052

181

0.

245

253

0.

441

418

0.

739

823

1.

025

606

2

0.

441

210

0.

3210.736

Image Removed

...

50 users 30 min (10 min ramp up)

Image Removed

...

50 users + DI in background

Image Removed

...

50 users +DI 5K in background

Image Removed

RTT without background activity

Latency

Requests	Total	OK	KO	% KO	Req/s	Min	50th pct	75th pct	95th pct	99th pct	Max	Average
418
GET_browse _call_numbers "N6370 .C585 2013eb"	10276	10276	0	0	9.343	0.054	0.170	0.243	0.397	0.763	1.847	0.199	0.397
GET_browse _call_numbers "P302.77 .P67 2013"	10273	10273	0	0	9.349	0.062	0.192	0.267	0.433	0.808	1.880	0.222	0.433
GET_browse _call_numbers "BT83.57 .N625 2013PR2274 .P7 1633"	102761027710276	10277	0	0	9.349334	0.046047	0.161168	0.234245	0.401411	0.779811	1.547815	0.192201	0.401411
GET_browse _call_numbers "CC76.4 .C66 2013TL220 .B85 2012"	102751027610275	10276	0	0	9.352339	0.046043	0.169151	0.243224	0.402383	0.806734	1.736765	0.199182	0.402383
GET_browse _call_numbers "DT107.827.L32 1987Z711 .C88 2013"	102771027510277	10275	0	0	9.346350	0.050093	0.169260	0.242339	0.400507	0.810995	12.721074	0.198289	0.399	GET_browse _call_numbers "G3893.M5 1923 .W4"	10275	10275	0	0	9.336	0.053	0.173	0.248	0.407	0.821	1.753	0.203	0.407507

RTT with DI in the background.

Requests	Total	OK	Req/s	Min	50th pct	75th pct	95th pct	99th pct	Max	Average	Latency
GET_browse _call_numbers "LC4019 .H83 2012BT83.57 .N625 2013"	10275207210275	2072	9.355280	0.056054	0.181211	0.253293	0.418460	0.823755	1.606423	0.210239	0.418460
GET_browse _call_numbers "N6370 .C585 2013ebCC76.4 .C66 2013"	10276207310276	2073	9.343282	0.054051	0.170224	0.243306	0.397469	0.763717	1.847419	0.199249	0.397469
GET_browse _call_numbers "P302DT107.77 827.P67 2013L32 1987"	10273207310273	2073	9.349277	0.062053	0.192216	0.267297	0.433461	0.808684	1.880359	0.222241	0.433460
GET_browse _call_numbers "PR2274 .P7 1633G3893.M5 1923 .W4"	10277207310277	2073	9.334270	0.047058	0.168217	0.245302	0.411462	0.811725	1.815332	0.201244	0.411462
GET_browse _call_numbers "TL220 LC4019 .B85 H83 2012"	10276207110276	2071	9.339283	0.043067	0.151234	0.224314	0.383484	0.734742	1.765330	0.182261	0.383484
GET_browse _call_numbers "Z711 N6370 .C88 2013C585 2013eb"	10275207310275	2073	9.350275	0.093060	0.260218	0.339298	0.507464	0.995690	21.074454	0.289246	0.507

RTT with DI in the background.

Latency0.484

Requests	Total	OK	KO	% KO	Req/s	Min	50th pct	75th pct	95th pct	99th pct	Max	Average
464
GET_browse _call_numbers "P302.77 .P67 2013"	2071	2071	0	0	9.278	0.068	0.240	0.325	0.500	0.752	1.484	0.268	0.500
GET_browse _call_numbers "BT83.57 .N625 2013PR2274 .P7 1633"	207220742072	2074	0	0	9.280266	0.054049	0.211221	0.293304	0.460464	0.755691	1.423485	0.239245	0.460464
GET_browse _call_numbers "CC76.4 .C66 2013TL220 .B85 2012"	207320722073	2072	0	0	9.282273	0.051044	0.224198	0.306274	0.469438	0.717671	1.419449	0.249225	0.469437
GET_browse _call_numbers "DT107.827.L32 1987"	2073	2073	0	0	9.277	0.053	0.216	0.297	0.461	0.684	1.359	0.241	0.460
GET_browse _call_numbers "G3893.M5 1923 .W4"	2073	2073	0	0	9.270	0.058	0.217	0.302	0.462	0.725	1.332	0.244	0.462
GET_browse _call_numbers "LC4019 .H83 2012"	2071	2071	0	0	9.283	0.067	0.234	0.314	0.484	0.742	1.330	0.261	Z711 .C88 2013"	2075	2075	0	0	9.282	0.103	0.315	0.406	0.591	0.871	1.604	0.340	0.591


average real call numbers from database	0.2095
average random words	0.326

Anchor
test4
test4

Comparison chart - call number browsing real keys + data import 5K in background

Image Added

Call number browsing with background CICO activity.

Image Added

A large usage of processor resources is observed in the mod-search module during call number browsing. I guess need to increase the processor resources in task definition of this module.

Grafana chart from test - random keywords.

Image Added

Grafana chart from test - real keywords from database.

SELECT jsonb -> effectiveCallNumberComponents →> 'callNumber') FROM fs09000000_mod_inventory_storage.item WHERE jsonb -> effectiveCallNumberComponents →'callNumber' is not null limit 10

Image Added

50 users call number browsing random words + 10 user CICO in background

Image Added

Requests	Total	OK	KO	% KO	Req/s	Min	50th pct	75th pct	95th pct	99th pct	Max	Average	Latency

Image Removed

Call number browsing with background CICO activity.

Image Removed

A large usege of processor resources is observed in the mod-search module during call number browsing. I guess need to increase the processor resources in tack definition of this module.

Grafana chart from test - random keywords.

Image Removed

Grafana chart from test - real keywords from database.

SELECT jsonb -> effectiveCallNumberComponents →> 'callNumber') FROM fs09000000_mod_inventory_storage.item WHERE jsonb -> effectiveCallNumberComponents →'callNumber' is not null limit 10

Image Removed

50 users call number browsing random words + 10 user CICO in background

Image Removed

Latency2073207420722075average real call numbers from database2095average random words326

Requests	Total	OK	KO	% KO	Req/s	Min	50th pct	75th pct	95th pct	99th pct	Max	Average
GET_browse _call_numbers "N6370 .C585 2013eb"	2073	BT83.57 .N625 2013	5299	5299	0	0	98.275566	0.060048	0.218172	0.298268	0.464475	0.690802	12.454981	0.246214	0.464475
GET_browse _call_numbers "P302CC76.77 4 .P67 C66 2013"	207152972071	5297	0	0	98.278569	0.068050	0.240178	0.325282	0.500486	0.752884	12.484249	0.268224	0.500485
GET_browse _call_numbers "PR2274 .P7 1633"	2074	DT107.827.L32 1987	5297	5297	0	0	98.266561	0.049052	0.221181	0.304281	0.464502	0.691922	12.485	0.245226	0.464502
GET_browse _call_numbers "TL220 .B85 2012"	2072	G3893.M5 1923 .W4	5297	5297	0	0	98.273549	0.044054	0.198182	0.274289	0.438511	0.671882	12.449742	0.225230	0.437511
GET_browse _call_numbers "Z711 .C88 2013"	2075	LC4019 .H83 2012	5298	5298	0	0	98.282576	0.103058	0.315193	0.406290	0.591513	0.871879	12.604337	0.340237	0.591	513
GET_browse _call_numbers N6370 .C585 2013eb	5296	5296	0	0	8.556	0.	054	0.178	0.	272

...

5299

0.491	0.869	2.302	0.223	0.491
GET_browse _call_numbers P302.77 .P67 2013	5297	5297	0	0	8.562	0.066	0.204	0.310	0.527	0.862	2.916	0.250	0.527
GET_browse _call_numbers PR2274 .P7 1633	5299	5299	0	0	8.547	0.048	0.182	0.280	0.504	0.903	2.142	0.228	0.504
GET_browse _call_numbers BT83.57 .N625 2013	5299	TL220 .B85 2012	5297	5297	0	0	8.566552	0.048043	0.172162	0.268259	0.475466	0.802847	2.981224	0.214207	0.475466
GET_browse _call_numbers CC76Z711 .4 .C66 C88 2013	529753005297	5300	0	0	8.569572	0.050091	0.178262	0.282387	0.4866470	1.884157	2.249992	0.2240.485318	0.647

call number browsing W/O background process

Image Added

Latency5298529652975297

Requests	Total	OK	KO	% KO	Req/s	Min	50th pct	75th pct	95th pct	99th pct	Max	Average	Latency
Requests	Total	OK	KO	% KO	Req/s	Min	50th pct	75th pct	95th pct	99th pct	Max	Average
GET_browse _call_numbers DT107BT83.82757 .L32 1987N625 2013	529727735297	2773	0	0	87.561757	0.052047	0.181144	0.281223	0.502378	0.922778	2.485292	0.226181	0.502378
GET_browse _call_numbers G3893CC76.M5 1923 4 .W4C66 2013	529727725297	2772	0	0	87.549762	0.054050	0.182159	0.289240	0.511409	0.882804	21.742594	0.230195	0.511409
GET_browse _call_numbers LC4019 .H83 2012	5298	DT107.827.L32 1987	2774	2774	0	0	87.576747	0.058050	0.193157	0.290240	0.513402	0.879792	2.337029	0.237193	0.513401
GET_browse _call_numbers N6370 .C585 2013eb	5296	G3893.M5 1923 .W4	2770	2770	0	0	87.556726	0.054053	0.178168	0.272253	0.491415	0.869712	21.302655	0.223201	0.491415
GET_browse _call_numbers P302.77 .P67 2013	5297	LC4019 .H83 2012	2773	2773	0	0	87.562772	0.066059	0.204168	0.310255	0.527415	0.862889	21.916485	0.250206	0.527415
GET_browse _call_numbers PR2274 N6370 .P7 1633C585 2013eb	529927745299	2774	0	0	87.547742	0.048054	0.182160	0.280235	0.504402	0.903758	21.142403	0.228195	0.504402
GET_browse _call_numbers TL220 .B85 2012	5297	P302.77 .P67 2013	2773	2773	0	0	87.552751	0.043067	0.162186	0.259270	0.466431	0.847797	21.224744	0.207221	0.466431
GET_browse _call_numbers Z711 PR2274 .C88 2013P7 1633	530027715300	2771	0	0	87.572720	0.091050	0.262160	0.387240	0.647408	10.157809	21.992845	0.318198	0.647408

call number browsing W/O background process

Image Removed

0.401

GET_browse _call_numbers TL220 .B85 2012	2776	2776	7.736	0.041	0.140	0.213	0.381	0.816	1.294	0.175	0.381
GET_browse _call_numbers BT83.57 .N625 2013	2773	2773	7.757	0.047	0.144	0.223	0.378	0.778	2.292	0.181	0.378
GET_browse _call_numbers CC76.4 .C66 2013	2772	2772	7.762	0.050	0.159	0.240	0.409	0.804	1.594	0.195	0.409
GET_browse _call_numbers DT107.827.L32 1987	2774	2774	7.747	0.050	0.157	0.240	0.402	0.792	2.029	0.193	Z711 .C88 2013	2774	2774	0	0	7.768	0.090	0.250	0.348	0.526	0.934	1.775	0.284	0.526

Average response times chart (blue- callnumber browsing random words, orange - call number browsing random words + CICO 10 users)_

Image Added

it can be seen that the response time slightly (15%) increases when working with CICO in background.

Anchor

	test5
	test5

long ramp-up period, 1 minute per user from 0 to 25 users

Image Added

Anchor

	test6
	test6

Call number browsing + DI + CICO

Image Added

Call number browsing with old cluster

1 user

0.415

Requests	Total	OK	KO	% KO	Req/s	Min	50th pct	75th pct	95th pct	99th pct	Max	Average	Latency
GET_browse _call_numbers G3893.M5 1923 .W4	2770	2770	0	0	7.726	0.053	0.168	0.253	0.415	0.712	1.655	0.201		2643	2643	0	0	2.233	0.044	0.125	0.164	0.285	0.494	0.948	0.145	0.278

10 users

Image Added

Requests	Total	OK	KO	% KO	Req/s	Min	50th pct	75th pct	95th pct	99th pct	Max	Average	Latency
GET_browse _call_numbers LC4019 .H83 2012	2773255692773	25569	0	0	721.772363	0.059043	0.168138	0.255194	0.415350	0.889611	1.485937	0.206167	0.415	GET_browse _call_numbers N6370 .C585 2013eb	2774	2774	0	0	7.742	0.054	0.160	0.235	0.402	0.758	1.403	0.195	0.402346

20 users

Image Added

2773

Requests	Total	OK	KO	% KO	Req/s	Min	50th pct	75th pct	95th pct	99th pct	Max	Average	Latency
GET_browse _call_numbers P302.77 .P67 2013	2773		41155	41155	0	0	7.751	0.06735.026	0.186041	0.270196	0.431355	0.797671	1.078	2.744625	0.221271	0.431669

50 users

Requests	Total	OK	KO	% KO	Req/s	Min	50th pct	75th pct	95th pct	99th pct	Max	Average	Latency
GET_browse _call_numbers PR2274 .P7 1633	27711033982771	103398	0	0	789.720437	0.050041	0.160211	0.240313	0.408534	0.809896	12.845146	0.198251	0.408	GET_browse _call_numbers TL220 .B85 2012	2776	2776	0	0	7.736	0.041	0.140	0.213	0.381	0.816	1.294	0.175	0.381530

Image Added

100 users

...

long ramp-up period, 1 minute per user from 0 to 25 users

Image Removed

...

Call number browsing + DI + CICO

Image Removed

...

Requests	Total	OK	KO	% KO	Req/s	Min	50th pct	75th pct	95th pct	99th pct	Max	Average	Latency
GET_browse _call_numbers Z711 .C88 2013	27741068652774	106865	0	0	793.768836	0.090044	0.250727	0.3480.526.978	1.394	2.073	4.713	0.934758	1.775	0.284	0.526

Average response times chart (blue- callnumber browsing random words, orange - call number browsing random words + CICO 10 users)_

Image Removed

it can be seen that the response time slightly (15%) increases when working with CICO in background.

...

389

Image Added

users	new cluster	old cluster
1	0.148	0.145
10	0.191	0.167
20	0.321	0.271
50	1.821	0.251

Versions Compared

Old Version 42

New Version Current

Key

Environment

Summary

Test 1 user call number browsing with random words.

Test

5 users call number browsing with random words.

Test

10 users call number browsing with random words

.

Test 20 users call number browsing with random words.

Test 50 users call number browsing with

random words.

Summary

Environment

Summary

Anchor
test4
test4

Comparison chart - call number browsing real keys + data import 5K in background

Call number browsing with background CICO activity.

Grafana chart from test - random keywords.

Grafana chart from test - real keywords from database.

50 users call number browsing random words + 10 user CICO in background

Call number browsing with background CICO activity.

Grafana chart from test - random keywords.

Grafana chart from test - real keywords from database.

50 users call number browsing random words + 10 user CICO in background

call number browsing W/O background process

Image Added

call number browsing W/O background process

Image Removed

Average response times chart (blue- callnumber browsing random words, orange - call number browsing random words + CICO 10 users)_

Image Added

long ramp-up period, 1 minute per user from 0 to 25 users

long ramp-up period, 1 minute per user from 0 to 25 users

Average response times chart (blue- callnumber browsing random words, orange - call number browsing random words + CICO 10 users)_

Image Removed

Page Comparison

Versions Compared

Key

Summary

Environment

Summary

Anchortest4test4Comparison chart - call number browsing real keys + data import 5K in background

Call number browsing with background CICO activity.

Grafana chart from test - random keywords.

Grafana chart from test - real keywords from database.

50 users call number browsing random words + 10 user CICO in background

Call number browsing with background CICO activity.

Grafana chart from test - random keywords.

Grafana chart from test - real keywords from database.

50 users call number browsing random words + 10 user CICO in background

call number browsing W/O background process

Image Added

call number browsing W/O background process

Image Removed

Average response times chart (blue- callnumber browsing random words, orange - call number browsing random words + CICO 10 users)_

Image Added

long ramp-up period, 1 minute per user from 0 to 25 users

long ramp-up period, 1 minute per user from 0 to 25 users

Average response times chart (blue- callnumber browsing random words, orange - call number browsing random words + CICO 10 users)_

Image Removed

Anchor
test4
test4

Comparison chart - call number browsing real keys + data import 5K in background