[ECS] Check-in-check-out Test Report (Quesnelia)
- Olga Kondratenko
Overview
The test's target is to investigate the behavior of the Check-In/Check-Out activities during fixed load tests.
PERF-857 - Getting issue details... STATUS
Summary
Check-In/Check-Out tests on Quesnelia were carried out with 8, 20, 30, and 75 virtual users for 45 minutes and 30 virtual users for 24 hours to detect any trends in resource utilization (only 16 hours of results available).
45-minute tests
- Response times in tests with 8, 20, and 30 virtual users didn't differ a lot but the lowest response times were with 20 virtual users. Average Check-In - 441 ms, average Check-Out - 893 ms
- The highest response time results were with 75 virtual users. Average Check-In - 546 ms, Check-Out - 1176 ms
- Reports: reports_test_results_8.zip, reports_test_results_20.zip, reports_test_results_30.zip, reports_test_results_75.zip
Longevity test
- Average Check-In - 458 ms, Check-Out - 1258 ms (the results from the test 16 hours)
- Longevity tests show the growth of memory usage of mod-agreements from 73 to 80%. No memory leaks were suspected during the longevity test for all other modules.
- RDS CPU grew from 30% to 55% on average. So it has a growing trend during longevity test and spikes up to 96% every 30 min corresponding to the circulation job.
- Grafana results
Resources
- CPU utilization
- 45-minute and longevity tests consumed CPU in correspondence to the number of vUsers but some modules spiked during the 75 vUsers test - nginx-okapi - 423%, mod-users-b - 105%.
- Memory consumption
- Longevity tests show the growth of memory usage of mod-agreements from 73 to 80%. No memory leaks were suspected during the longevity test for all other modules.
- Average RDS CPU utilization
- 8 vUsers - 13%, 20 vUsers - 18%, 30 vUsers - 23%, 75 vUsers - 42% During longevity test CPU grew from 30% to 55% on average. So it has a growing trend during longevity test and spikes up to 96% every 30 min corresponding to the circulation job.
- CPU (User) usage by broker
- Common CPU utilization by the broker during all tests was up to 12% with equal distribution between brokers
Common notes
- Longevity test revealed Database CPU Utilization growing trend and CI and CO response time increasing.
Recommendations
- Add more CPU to nginx-okapi to exclude module fluctuations
Test Runs
The following table contains test configuration information
Test # | Test artefact | vUsers | Ramp-up, sec | Duration, sec |
1 | circulation_checkInCheckOut_qcon | 8 | 80 | 2700 |
| 2 | 20 | 200 | 2700 | |
| 3 | 30 | 300 | 2700 | |
| 4 | 75 | 750 | 2700 | |
| 5 | 30 | 300 | 86400 |
Results
Errors:
- Error messages: Response 401 due to changed login process and token expiration time 10 min (0.04%).
Response time
The table contains results of Check-in, Check-out tests in the Quesnelia release.
45-minute tests
| 8 vUsers | 20 vUsers | 30 vUsers | 75 vUsers | |||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|---|
| Requests | Response Times (ms) | Response Times (ms) | Response Times (ms) | Response Times (ms) | ||||||||
| Label | #Samples | Average | 95th pct | #Samples | Average | 95th pct | #Samples | Average | 95th pct | #Samples | Average | 95th pct |
| Check-Out Controller | 1815 | 910.51 | 1102.40 | 4324 | 893.26 | 1054.00 | 6626 | 899.13 | 1058.00 | 10986 | 1176.40 | 1569.00 |
| Check-In Controller | 1391 | 476.67 | 598.80 | 3527 | 441.99 | 551.60 | 4911 | 435.40 | 544.00 | 14564 | 546.74 | 760.00 |
| POST_circulation/check-out-by-barcode (Submit_barcode_checkout) | 1798 | 275.86 | 340.00 | 4275 | 268.25 | 315.00 | 6556 | 268.31 | 316.00 | 14434 | 331.68 | 491.00 |
| POST_circulation/check-in-by-barcode (Submit_barcode_checkin) | 1388 | 243.06 | 313.55 | 3518 | 214.64 | 275.00 | 4906 | 205.53 | 267.00 | 11159 | 233.01 | 348.00 |
| GET_circulation/loans (Submit_patron_barcode) | 1805 | 143.41 | 173.00 | 4292 | 141.66 | 160.00 | 6586 | 140.52 | 161.00 | 14493 | 177.03 | 273.00 |
| GET_circulation/loans (Submit_barcode_checkout) | 1795 | 140.59 | 163.20 | 4275 | 139.57 | 158.00 | 6550 | 140.00 | 159.00 | 14418 | 174.65 | 265.00 |
Longevity test
| 30 vUsers Longevity | |||
|---|---|---|---|
| Requests | Samples, Response Times | ||
| Label | #Samples | 95th pct | Average |
| Check-Out Controller | 144129 | 1.998 | 1.258 |
| Check-In Controller | 108447 | 0.645 | 0.458 |
| POST_circulation/check-out-by-barcode (Submit_barcode_checkout) | 142488 | 0.613 | 0.383 |
| POST_circulation/check-in-by-barcode (Submit_barcode_checkin) | 107907 | 0.301 | 0.200 |
Resources Utilization
CPU Utilization
45-minute and longevity tests consumed CPU in correspondence to the number of vUsers but some modules spiked during the 75 vUsers test - nginx-okapi - 423%, mod-users-b - 105%.
45 minute tests
Longevity test
Memory Consumption
45-minute tests didn't reveal some problems with memory usage by modules. Longevity tests show the growth of memory usage of mod-agreements from 73 to 80%. No memory leaks were suspected during the longevity test for all other modules.
45-minute tests
Longevity test
RDS CPU Utilization
RDS CPU utilized:
8 vUsers - 13%, 20 vUsers - 18%, 30 vUsers - 23%, 75 vUsers - 42% During longevity test CPU grew from 30% to 55% on average. So it has a growing trend during longevity test and spikes up to 96% every 30 min corresponding to the circulation job.
45-minute tests
Longevity test
RDS Database Connections
For both 45-minute and longevity tests RDS used max 615-630 connections. Without test it was 550 connections.
45-minute tests
Longevity test
CPU (User) usage by broker
Common CPU utilization by broker during all tests was 12% with equal distribution between brokers
45 minute tests
Longevity test
Appendix
Infrastructure
| PTF -environment qcon |
|---|
|
DB table records size:
|
|---|
Modules