Master Script longevity test - NLA report
Overview
In the scope of PERF-527: NLA: Longevity Tests of average use caseClosedit's needed to run tests to answer questions:
Summary
The current system can accommodate an average load only without Data Import. With data import, we will have 'HTTP 500 Internal Server Error. If the issue persists, please report it to EBSCO Connect.' for several workflows in the source-storage/records/{id}/formatted requests
PERF-582: Response 500 for NLA normal load test with Data Import job (2 SP)Blocked, and the general response time will be longer up to 2 times for all other workflows.
Service CPU usage did not exceed 44 % for all modules. We can observe spikes in CPU usage of DI modules at the beginning of the Data Import jobs. CPU usage of mod-inventory-storage grows up to 31% but decreased automatically to 6% (a possible reason is garbage collection process work). Without DI spikes average CPU usage was up to 13% for all other modules.
Service memory utilization was stable, and no memory leaks were suspected during tests.
Average instance CPU utilization was up to 8.5% and spiked at the beginning of Data Import up to 15%
Average DB CPU usage is up to 55%. For part of the test with DI job- 620 connections count. 395 connections - average usage during the test.
Recommendations & Jiras
Jiras
PERF-582: Response 500 for NLA normal load test with Data Import job (2 SP)Blocked
Test Runs & Results
Test # | # configuration | Test duration | comments |
|---|---|---|---|
1 | FYR workflow started at the beginning of the test | 10 hours | Res 500 ERROR during Data Import (DI) process |
2 | FYR workflow started at the end of the test | 24 hours (planned - not performed yet) | Will be performed after DI issues fix |
Test results from 1st test run:
Test # 1 | Workflow name | Total time it takes to complete workflow | Time-consuming requests for each workflow during DI, finished with Response body: HTTP 500 Internal Server Error. If the issue persists, please report it to EBSCO Connect. | |||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|---|
|
| 1st hour of the test Avg + FYR | 1st hour of the test 95th pct + FYR | 2nd hour of the test Avg | 2nd hour of the test 95th pct | 10th hour of the test Avg | 10th hour of the test 95th pct | Avg With DI (sec) normal load | 95th pct with DI (sec) normal load | Avg no DI (sec) normal load | 95th pct no DI (sec) normal load | |
1 | CICO_Checkin | 1.227 | 1.716 | 1.196 | 1.661 | 1.074 | 1.355 | 1.238 | 1.506 | 1.054 | 1.591 |
|
2 | CICO_Checkout | 1.978 | 2.694 | 1.952 | 2.635 | 1.767 | 2.168 | 2.156 | 2.829 | 1.650 | 1.948 |
|
3 | IO_View invoices | 0.945 | 1.221 | 0.932 | 1.142 | 0.904 | 1.012 | 0.907 | 1.305 | 0.763 | 0.913 |
|
4 | IO_Create invoices | 1.718 | 2.126 | 1.769 | 2.187 | 1.804 | 2.061 | 1.433 | 1.815 | 1.174 | 1.370 |
|
5 | IO_Edit invoices | 2.170 | 2.497 | 2.195 | 2.442 | 2.149 | 2.432 | 1.983 | 2.422 | 1.581 | 1.897 |
|
6 | IO_Delete invoices | 1.119 | 1.348 | 1.090 | 1.296 | 1.141 | 1.307 | 1.070 | 1.196 | 0.804 | 0.927 |
|
7 | AIE_Approving Invoices | 1.174 | 1.940 | 1.232 | 1.946 | 1.662 | 2.276 | 1.752 | 2.211 | 1.453 | 1.940 |
|
8 | VAR_View Authority records | 0.310 | 0.496 | 0.305 | 0.444 | 0.278 | 0.418 | 22.037 | 30.604 | 0.289 | 0.381 | VAR_GET /source-storage/records/marc_id/formatted |
9 | VTT_View MARC tag table | 1.422 | 2.392 | 1.473 | 2.370 | 1.039 | 1.369 | 41.272 | 61.935 | 0.987 | 1.284 | VTT_GET source-storage/records/{id}/formatted *2 |
10 | VH_View holdings records | 1.733 | 2.507 | 1.626 | 2.296 | 1.361 | 1.616 | 27.328 | 33.579 | 1.526 | 1.922 | VH_GET source-storage/records/{id}/formatted |
11 | VB_View Bib | 1.179 | 2.011 | 1.177 | 1.881 | 0.850 | 1.129 | 22.851 | 31.634 | 0.841 | 1.168 | VB_GET source-storage/records/{id}/formatted |
12 | PRO_View patron records | 0.636 | 1.032 | 0.628 | 1.069 | 0.559 | 0.869 | 0.672 | 1.118 | 0.566 | 0.883 |
|
13 | PRO_Delete patron records | 0.703 | 1.181 | 0.844 | 1.113 | 0.716 | 1.049 | 0.892 | 1.336 | 0.638 | 1.070 |
|
14 | PRO_Update patron records | 1.163 | 1.915 | 1.011 | 1.703 | 1.091 | 1.692 | 1.386 | 2.097 | 1.043 | 1.625 |
|
15 | PRO_Create patron records | 1.210 | 1.482 | 1.192 | 1.482 | 1.189 | 1.399 | 1.547 | 1.979 | 1.098 | 1.261 |
|
16 | LO_View Ledger | 0.048 | 0.086 | 0.048 | 0.070 | 0.077 | 0.120 | 0.122 | 0.458 | 0.050 | 0.088 |
|
17 | LO_Create ledger | 0.635 | 0.799 | 0.631 | 0.744 | 0.710 | 0.848 | 0.684 | 0.840 | 0.616 | 0.761 |
|
18 | LO_Edit ledger | 0.050 | 0.087 | 0.050 | 0.081 | 0.079 | 0.128 | 0.076 | 0.094 | 0.054 | 0.085 |
|
19 | LO_Delete a ledger | 0.046 | 0.077 | 0.047 | 0.073 | 0.080 | 0.115 | 0.080 | 0.129 | 0.046 | 0.080 |
|
20 | DE_Export bib "Default instances export job profile" | 5 sec | - | 4 sec | - | 5 sec | - | 11 sec (5000 records) | - | 5 sec (5000 records) | - |
|
21 | DE_Export holdings "Default holdings export job profile" | 44 sec | - | 28 sec | - | 27 sec | - | 3 min 16 sec (5000 records) | - | 26 sec (5000 records) | - |
|
22 | DE_Export authority records "Default authority export job profile" | 4 sec | - | 3 sec | - | 3 sec | - | 8 sec (5000 records) | - | 3 sec (5000 records) | - | |