CICO - Eureka cluster's without NGINX Test Report (Sunflower)
Overview
Testing of Check-In/Check-Out (CI/CO) fixed load tests on Eureka based environment in Sunflower ECS release without NGINX.
The purpose of testing is to evaluate the impact of removing NGINX.
Testing was carried out on non-HA (High Availability) mode.
Testing was carried out on member tenant.
Comparison of results with and without NGINX.
The current ticket:
PERF-1158: Test Eureka cluster's performance without NGINXClosed
Summary
Tests without NGINX completed successfully with no errors.
Tests with NGINX showed an Error Rate starting from 30 users, increasing at 75 users.
Response times without NGINX are the same or 4% faster for Check-Out and 2%-14% faster for Check-in.
Performance without NGINX showed more stable behaviour, with faster response times and no failures.
Test Runs
The following table contains tests configuration information
Test # | vUsers | Ramp-up, sec | Duration, sec |
|
1 | 8 | 80 | 2700 | without NGINX |
2 | 20 | 200 | 2700 | |
3 | 30 | 300 | 2700 | |
4 | 75 | 750 | 2700 | |
5 (Test#1 rerun) | 8 | 80 | 2700 | |
6 | 30 | 300 | 86400 | |
7 | 8 | 80 | 2700 | with NGINX |
8 | 20 | 200 | 2700 | |
9 | 30 | 300 | 2700 | |
10 | 75 | 750 | 2700 | |
11 | 30 | 300 | 36000 |
Results
Response time
The table contains results of Check-in, Check-out (Eureka cluster's without NGINX) tests in Sunflower release.
Test #5, #2, #3, #4
A short period of response time instability was observed. An additional run (Test #5) with 8 vUsers did not reproduce the issue.
| 8 vUsers (test #5) | 20 vUsers (test #2) | 30 vUsers (test #3) | 75 vUsers (test #4) | ||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|---|
Requests | Response Times (ms) | Response Times (ms) | Response Times (ms) | Response Times (ms) | ||||||||
Label | #Samples | 95th pct | Average | #Samples | 95th pct | Average | #Samples | 95th pct | Average | #Samples | 95th pct | Average |
Check-Out Controller | 2559 | 1039 | 916 | 6274 | 1355 | 964 | 9278 | 1437 | 998 | 19259 | 2677 | 1520 |
Check-In Controller | 1991 | 493 | 395 | 4794 | 548 | 379 | 6801 | 583 | 385 | 14591 | 946 | 557 |
POST_circulation/check-out-by-barcode (Submit_barcode_checkout) | 2559 | 325 | 291 | 6277 | 431 | 313 | 9279 | 482 | 325 | 19265 | 1141 | 528 |
POST_circulation/check-in-by-barcode (Submit_barcode_checkin) | 1992 | 269 | 219 | 4794 | 335 | 203 | 6801 | 361 | 203 | 14593 | 569 | 290 |
GET_circulation/loans (Submit_patron_barcode) | 2561
| 128
| 110 | 6278 | 179 | 118 | 9284 | 219 | 124 | 19279
| 389 | 199 |
GET_circulation/loans (Submit_barcode_checkout) | 2559
| 127
| 109
| 6274 | 173 | 118 | 9278 | 204 | 123 | 19260 | 383 | 197 |
GET_inventory/items (Submit_barcode_checkout) | 2559 | 41 | 32 | 6277 | 63 | 37 | 9280 | 76 | 39 | 19266 | 171 | 84 |
Test #6
30 vUsers Longevity test (test #6) | |||
Requests |
| Response Times (ms) | |
Label | NumberOfSamples | pct95 | Average |
Check-Out Controller | 302166 | 2325 | 1280 |
Check-In Controller | 228352 | 468 | 336 |
POST_circulation/check-out-by-barcode (Submit_barcode_checkout) | 302166 | 873 | 406 |
POST_circulation/check-in-by-barcode (Submit_barcode_checkin) | 228352 | 253 | 160 |
GET_circulation/loans (Submit_patron_barcode) | 302171 | 206 | 122 |
GET_circulation/loans (Submit_barcode_checkout) | 302166 | 207 | 123 |
GET_inventory/items (Submit_barcode_checkout) | 302166 | 73 | 36 |
The table contains results of Check-in, Check-out (with NGINX) tests in Sunflower release (Response times, ms).
Test #7, #8, #9, #10
There was an error rate observed during tests with 30 and 75 users (Response Code 500 for POST_circulation/check-out-by-barcode (Submit_barcode_checkout)).
Requests | 8 vUsers (test #7) | 20 vUsers (test #8) | 30 vUsers (test #9) | 75 vUsers (test #10) | ||||||||||||
Label | NumberOfSamples | pct95 | Average | NumberOfSamples | pct95 | Average | NumberOfSamples | pct95 | Average | Error Count | Error Rate, % | NumberOfSamples | pct95 | Average | Error Count | Error Rate, % |
Check-Out Controller | 2572 | 1313 | 951 | 6237 | 1266 | 994 | 9071 | 1394 | 1035 | 710 | 8 | 19747 | 1873 | 1278 | 9175 | 46 |
Check-In Controller | 1895 | 483 | 404 | 4718 | 527 | 398 | 6887 | 582 | 446 | 326 | 5 | 15195 | 721 | 569 | 4059 | 27 |
POST_circulation/check-out-by-barcode (Submit_barcode_checkout) | 2573 | 335 | 305 | 6241 | 388 | 316 | 9075 | 445 | 327 | 352 | 4 | 19754 | 781 | 364 | 5010 | 25 |
POST_circulation/check-in-by-barcode (Submit_barcode_checkin) | 1895 | |||||||||||||||