PERF-131 - Getting issue details... STATUS

Overview

The purpose of this test is to define baselines of request search flow to be able compare test metrics in the future

Version

okapi:4.2.2
mod-inventory-storage:19.5.0-SNAPSHOT.460
mod-circulation:19.2.2"
mod-users:17.2.1"
mod-circulation-storage:12.1.0

Tests

Test #	Virtual Users	Duration (sec)
1	5	1800
2	8	1800
3	10	1800
4	20	1800

Test Summary

All tests shows stable work, and pass whit 0% of errors. However high CPU utilisation from RDS side may cause an issues with bigger loads in future.

The more load causes more CPU usage on database and it leads to higher average response times.

There is two query types that causes most of CPU usage, it's count.estimate and select jsonb,id from circulation_storage.requests.

Results

Test 1

Requests	Total	OK	Min	50th pct	75th pct	95th pct	99th pct	Max
Requests	Total	OK	Response times (seconds)
GET_addresstypes	4141	4141	0.013	0.032	0.040	0.072	0.106	0.418
GET_cancellation-reason-storage/cancellation-reasons	4141	4141	0.014	0.031	0.038	0.059	0.093	0.762
GET_circulation/pick-slips/servicePointId	4141	4141	0.028	0.089	0.105	0.159	0.246	0.489
GET_circulation/requests	16560	16560	0.079	0.256	0.534	0.876	1.039	2.054
GET_circulation/requests-reports/hold-shelf-clearance/servicePointId	8282	8282	0.026	0.066	0.080	0.122	0.185	0.800
GET_groups	4141	4141	0.013	0.031	0.039	0.068	0.104	0.778
GET_service-points	4141	4141	0.015	0.031	0.038	0.061	0.106	0.373
GET_staff-slips-storage/staff-slips	4141	4141	0.014	0.031	0.038	0.062	0.114	0.766
GET_tags	4141	4141	0.014	0.031	0.038	0.060	0.098	0.375
POST_authn/login	1	1	0.445	0.445	0.445	0.445	0.445	0.445
Search request by open status transaction workflow	4137	4137	1.294	1.893	2.017	2.417	2.825	3.898

Brief overview of resource usage below:

Service CPU usage:

As you can see, there's only one module that uses around 250% CPU usage - it's okapi.

Service Memory usage:

There's no visible memory usage for all relevant modules.

RDS CPU usage:

As you can see above, CPU usage on 50% rate was made by count estimate queries that takes 1.14 seconds in average.

Test 2

Requests	Total	OK	Min	50th pct	75th pct	95th pct	99th pct	Max
Requests	Total	OK	Response times (seconds)
GET_addresstypes	5048	5048	0.012	0.036	0.055	0.117	0.192	0.603
GET_cancellation-reason-storage/cancellation-reasons	5048	5048	0.013	0.034	0.047	0.108	0.201	0.610
GET_circulation/pick-slips/servicePointId	5047	5047	0.027	0.104	0.140	0.272	0.418	1.581
GET_circulation/requests	20172	20172	0.078	0.477	0.812	1.161	1.480	3.079
GET_circulation/requests-reports/hold-shelf-clearance/servicePointId	10096	10096	0.025	0.076	0.104	0.206	0.330	1.456
GET_groups	5048	5048	0.012	0.034	0.049	0.110	0.193	0.706
GET_service-points	5048	5048	0.014	0.035	0.046	0.104	0.191	1.211
GET_staff-slips-storage/staff-slips	5048	5048	0.013	0.034	0.047	0.106	0.187	0.703
GET_tags	5046	5046	0.014	0.034	0.046	0.101	0.187	0.790
POST_authn/login	1	1	0.331	0.331	0.331	0.331	0.331	0.331
Search request by open status transaction workflow	5040	5040	1.783	2.535	2.938	4.061	4.732	6.336

Service CPU usage:

Most used modules:

OKAPI - 415%;
Mod-Inventory storage - 73%-75%;
Mod-users - 60-70
Mod-circulation - 60-70
Mod circulation storage - 40;

Services memory usage ;

RDS CPU usage

there is high CPU utilisation from DB side during test. Perhaps select mentioned on screen above may be optimised to reduce this. because this select taking most of CPU time.

RDS wait is 1.85 seconds in average;

Test 3

Requests	Total	OK	Min	50th pct	75th pct	95th pct	99th pct	Max
Requests	Total	OK	Response times (seconds)
GET_addresstypes	5477	5477	0.014	0.038	0.058	0.143	0.214	0.459
GET_cancellation-reason-storage/cancellation-reasons	5477	5477	0.014	0.037	0.050	0.118	0.219	0.990
GET_circulation/pick-slips/servicePointId	5476	5476	0.031	0.110	0.141	0.260	0.427	1.325
GET_circulation/requests	21899	21899	0.082	0.530	0.896	1.234	1.588	601.342
GET_circulation/requests-reports/hold-shelf-clearance/servicePointId	10954	10954	0.028	0.078	0.102	0.207	0.330	1.048
GET_groups	5477	5477	0.012	0.037	0.051	0.133	0.208	0.602
GET_service-points	5477	5477	0.014	0.037	0.048	0.103	0.189	0.454
GET_staff-slips-storage/staff-slips	5477	5477	0.013	0.036	0.049	0.113	0.209	0.452
GET_tags	5476	5476	0.014	0.036	0.047	0.107	0.207	0.878
POST_authn/login	1	1	0.317	0.317	0.317	0.317	0.317	0.317
Search request by open status transaction workflow	5473	5473	2.016	2.785	3.131	4.236	5.235	603.154

Service CPU usage:

Okapi- 440 -450;
Users - 100-115
Mod inventory storage 101 - 110;
Circulation - 60-80;
Circulation storage 40-50

Memory usage:

There is still no visible memory impact on all relevant modules

RDS wait is 2.80 seconds wait;

Test 4

Requests	Total	OK	Min	50th pct	75th pct	95th pct	99th pct	Max
Requests	Total	OK	Response times (seconds)
GET_addresstypes	6279	6279	0.015	0.050	0.092	0.241	0.505	1.479
GET_cancellation-reason-storage/cancellation-reasons	6279	6279	0.015	0.159	0.313	0.518	0.671	1.211
GET_circulation/pick-slips/servicePointId	6278	6278	0.030	0.154	0.227	0.601	0.965	3.259
GET_circulation/requests	25090	25089	0.114	0.995	1.615	2.289	3.282	6.656
GET_circulation/requests-reports/hold-shelf-clearance/servicePointId	12562	12561	0.034	0.098	0.151	0.448	0.760	600.003
GET_groups	6279	6279	0.015	0.046	0.080	0.242	0.506	1.219
GET_service-points	6279	6279	0.016	0.045	0.068	0.219	0.468	1.479
GET_staff-slips-storage/staff-slips	6279	6279	0.014	0.123	0.264	0.498	0.668	1.734
GET_tags	6278	6278	0.016	0.046	0.071	0.213	0.477	1.115
POST_authn/login	1	1	0.339	0.339	0.339	0.339	0.339	0.339
Search request by open status transaction workflow	6267	6265	3.521	5.245	5.889	8.547	11.256	600.081

Service CPU usage:

OKAPI - 400%-500%
Users 110%-130%
inventory storage - 95%-125%
circulation 75% -100%
Circulation storage 40%-50%

Service memory usage

There is no visible memory impact on all relevant modules

Browser not supported