[ECS] Check-in-check-out Test Report (Quesnelia)
Overview
The test's target is to investigate the behavior of the Check-In/Check-Out activities during fixed load tests.
https://folio-org.atlassian.net/browse/PERF-857
Summary
Check-In/Check-Out tests on Quesnelia were carried out with 8, 20, 30, and 75 virtual users for 45 minutes and 30 virtual users for 24 hours to detect any trends in resource utilization (only 16 hours of results available).
45-minute tests
Response times in tests with 8, 20, and 30 virtual users didn't differ a lot but the lowest response times were with 20 virtual users. Average Check-In - 441 ms, average Check-Out - 893 ms
The highest response time results were with 75 virtual users. Average Check-In - 546 ms, Check-Out - 1176 ms
Reports: , , ,
Longevity test
Average Check-In - 458 ms, Check-Out - 1258 ms (the results from the test 16 hours)
Longevity tests show the growth of memory usage of mod-agreements from 73 to 80%. No memory leaks were suspected during the longevity test for all other modules.
RDS CPU grew from 30% to 55% on average. So it has a growing trend during longevity test and spikes up to 96% every 30 min corresponding to the circulation job.
Resources
CPU utilization
45-minute and longevity tests consumed CPU in correspondence to the number of vUsers but some modules spiked during the 75 vUsers test - nginx-okapi - 423%, mod-users-b - 105%.
Memory consumption
Longevity tests show the growth of memory usage of mod-agreements from 73 to 80%. No memory leaks were suspected during the longevity test for all other modules.
Average RDS CPU utilization
8 vUsers - 13%, 20 vUsers - 18%, 30 vUsers - 23%, 75 vUsers - 42% During longevity test CPU grew from 30% to 55% on average. So it has a growing trend during longevity test and spikes up to 96% every 30 min corresponding to the circulation job.
CPU (User) usage by broker
Common CPU utilization by the broker during all tests was up to 12% with equal distribution between brokers
Common notes
Longevity test revealed Database CPU Utilization growing trend and CI and CO response time increasing.
Recommendations
Add more CPU to nginx-okapi to exclude module fluctuations
Test Runs
The following table contains test configuration information
Test # | Test artefact | vUsers | Ramp-up, sec | Duration, sec |
1 | circulation_checkInCheckOut_qcon | 8 | 80 | 2700 |
2 |
| 20 | 200 | 2700 |
3 |
| 30 | 300 | 2700 |
4 |
| 75 | 750 | 2700 |
5 |
| 30 | 300 | 86400 |
Results
Errors:
Error messages: Response 401 due to changed login process and token expiration time 10 min (0.04%).
Response time
The table contains results of Check-in, Check-out tests in the Quesnelia release.
45-minute tests
| 8 vUsers | 20 vUsers | 30 vUsers | 75 vUsers | ||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|---|
Requests | Response Times (ms) | Response Times (ms) | Response Times (ms) | Response Times (ms) | ||||||||
Label | #Samples | Average | 95th pct | #Samples | Average | 95th pct | #Samples | Average | 95th pct | #Samples | Average | 95th pct |
Check-Out Controller | 1815 | 910.51 | 1102.40 | 4324 | 893.26 | 1054.00 | 6626 | 899.13 | 1058.00 | 10986 | 1176.40 | 1569.00 |
Check-In Controller | 1391 | 476.67 | 598.80 | 3527 | 441.99 | 551.60 | 4911 | 435.40 | 544.00 | 14564 | 546.74 | 760.00 |
POST_circulation/check-out-by-barcode (Submit_barcode_checkout) | 1798 | 275.86 | 340.00 | 4275 | 268.25 | 315.00 | 6556 | 268.31 | 316.00 | 14434 | 331.68 | 491.00 |
POST_circulation/check-in-by-barcode (Submit_barcode_checkin) | 1388 | 243.06 | 313.55 | 3518 | 214.64 | 275.00 | 4906 | 205.53 | 267.00 | 11159 | 233.01 | 348.00 |
GET_circulation/loans (Submit_patron_barcode) | 1805 | 143.41 | 173.00 | 4292 | 141.66 | 160.00 | 6586 | 140.52 | 161.00 | 14493 | 177.03 | 273.00 |
GET_circulation/loans (Submit_barcode_checkout) | 1795 | 140.59 | 163.20 | 4275 | 139.57 | 158.00 | 6550 | 140.00 | 159.00 | 14418 | 174.65 | 265.00 |
Longevity test
30 vUsers Longevity |
|
|
|
|---|---|---|---|
Requests | Samples, Response Times |
|
|
Label | #Samples | 95th pct | Average |
Check-Out Controller | 144129 | 1.998 | 1.258 |
Check-In Controller | 108447 | 0.645 | 0.458 |
POST_circulation/check-out-by-barcode (Submit_barcode_checkout) | 142488 | 0.613 | 0.383 |
POST_circulation/check-in-by-barcode (Submit_barcode_checkin) | 107907 | 0.301 | 0.200 |
Resources Utilization
CPU Utilization
45-minute and longevity tests consumed CPU in correspondence to the number of vUsers but some modules spiked during the 75 vUsers test - nginx-okapi - 423%, mod-users-b - 105%.
45 minute tests
Longevity test
Memory Consumption
45-minute tests didn't reveal some problems with memory usage by modules. Longevity tests show the growth of memory usage of mod-agreements from 73 to 80%. No memory leaks were suspected during the longevity test for all other modules.
45-minute tests
Longevity test
RDS CPU Utilization
RDS CPU utilized:
8 vUsers - 13%, 20 vUsers - 18%, 30 vUsers - 23%, 75 vUsers - 42% During longevity test CPU grew from 30% to 55% on average. So it has a growing trend during longevity test and spikes up to 96% every 30 min corresponding to the circulation job.
45-minute tests
Longevity test
RDS Database Connections
For both 45-minute and longevity tests RDS used max 615-630 connections. Without test it was 550 connections.
45-minute tests
Longevity test
CPU (User) usage by broker
Common CPU utilization by broker during all tests was 12% with equal distribution between brokers
45 minute tests
Longevity test
Appendix
Infrastructure
PTF -environment qcon |
|---|
|
DB table records size:
|
|---|
Modules