Effect of horizontal scaling on Eureka Sunflower [non-ECS]
Overview
The main idea of the test to understand the effect of horizontal scaling on Eureka by perform a CICO test of 20 users and 40 users on a non-HA and HA environment. Additional task is to check how availability zone affect the performance switching between us-east-1a and us-east-1b.PERF-1225: Effect of horizontal scaling on Eureka
Summary
All tests completed successfully.
Testing results in non-HA and HA environments show that non-HA mode perform better than HA
20 vUsers tests response time slightly faster in non-HA
CI 609ms - 17%(105ms), CO 1346ms - 9%(125ms)
40 vUsers tests response time faster in non-HA
CI 870ms - 54%(471ms), CO 2039ms - 23%(486ms)
Testing results in different AZ (availability zone) reveal light performance improvement (3%) in 20 vUser tests in us-east-1a zone (the same database AZ) but for 40vUsers test improvement in us-east-1a was only in HA mode (5%). In non-HA response times better in us-east-1b 10% in average for both CI and CO.
Additional test runs in ECS environment show that performance of CI/CO flows was not affected by horizontal scaling (non-HA, HA, 4 tasks). It perform the same or slightly better - 30-60 ms.
Non-HA |
|
|
|
|
|
vUsers | Label | us-east-1a | us-east-1b | Difference | Faster AZ |
20 | CI | 547 ms | 555 ms | 1% (8 ms) | 1a |
20 | CO | 1306 ms | 1326 ms | 2% (20 ms) | 1a |
40 | CI | 1072 ms | 951 ms | 11% (121 ms) | 1b |
40 | CO | 2435 ms | 2237 ms | 8% (198 ms) | 1b |
HA |
|
|
|
|
|
vUsers | Label | us-east-1a | us-east-1b | Difference | Faster AZ |
20 | CI | 588 ms | 657 ms | 12% (69 ms) | 1a |
20 | CO | 1374 ms | 1413 ms | 3% (39 ms) | 1a |
40 | CI | 983 ms | 1032 ms | 5% (49 ms) | 1a |
40 | CO | 2322 ms | 2432 ms | 5% (110 ms) | 1a |
Additional test results on ECS environment in HA and non-HA mode with 40 vUsers. This effort also include 2 tests on HA mode (2 tasks for CICO involved modules) and 2 tests with 4 tasks for CICO involved modules:
In a new environment comparing HA and non-HA we see that Check-In/Check-Out perform 5% better (CI - 28ms, CO - 58ms) in HA mode
| HA | non-HA |
|
|
|
|---|---|---|---|---|---|
Label | Average | Average | Delta, ms | Delta, % | Faster mode |
CI | 447 | 475 | 28 | 6% | HA |
CO | 1255 | 1313 | 58 | 5% | HA |
Raising number of tasks to 4 do not affect performance
Task count in CICO involved modules - 4 | Run #1 HA | Run #2 HA | Previous HA |
Label | Average | Average | Average |
CI | 478 | 436 | 447 |
CO | 1326 | 1234 | 1255 |
Test Runs
Test # | vUsers | Availability mode | Availability zone | Ramp-up, sec | Duration, sec |
|---|---|---|---|---|---|
1 | 20 | HA | us-east-1a / us-east-1b | 200 | 2700 |
2 | 40 | HA | us-east-1a / us-east-1b | 400 | 2700 |
3 | 20 | non-HA | us-east-1a / us-east-1b | 200 | 2700 |
4 | 40 | non-HA | us-east-1a / us-east-1b | 400 | 2700 |
5 | 20 | HA | us-east-1a | 200 | 2700 |
6 | 40 | HA | us-east-1a | 400 | 2700 |
7 | 20 | non-HA | us-east-1a | 200 | 2700 |
8 | 40 | non-HA | us-east-1a | 400 | 2700 |
9 | 20 | HA | us-east-1b | 200 | 2700 |
10 | 40 | HA | us-east-1b | 400 | 2700 |
11 | 20 | non-HA | us-east-1b | 200 | 2700 |
12 | 40 | non-HA | us-east-1b | 400 | 2700 |
Results
Test results
Tests # 1, 2, 3, 4 | Response times (ms) | |||||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|---|
| 20vUsers non-HA (test #3) | 20vUsers HA (test #1) | 40vUsers non-HA (test #4) | 40vUsers HA (test #2) | ||||||||
Label | #Samples | Average | 90th pct | #Samples | Average | 90th pct | #Samples | Average | 90th pct | #Samples | Average | 90th pct |
Check-In Controller | 4452 | 609 | 777 | 4340 | 714 | 1031 | 7594 | 870 | 1304 | 7115 | 1341 | 2229 |
Check-Out Controller | 5872 | 1346 | 1679 | 5768 | 1471 | 1966 | 10280 | 2039 | 3064 | 9356 | 2525 | 3911 |
POST_circulation/check-in-by-barcode (Submit_barcode_checkin) | 4453 | 275 | 372 | 4340 | 314 | 489 | 7595 | 432 | 706 | 7119 | 523 | 880 |
POST_circulation/check-out-by-barcode (Submit_barcode_checkout) | 5872 | 608 | 790 | 5771 | 667 | 971 | 10281 | 1000 | 1738 | 9364 | 1290 | 2295 |
Tests #7, 8, 11, 12 | Response times (ms) | |||||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|---|
| 20vUsers non-HA us-east-1a | 20vUsers non-HA us-east-1b | 40vUsers non-HA us-east-1a | 40vUsers non-HA us-east-1b | ||||||||
Label | #Samples | Average | 90th pct | #Samples | Average | 90th pct | #Samples | Average | 90th pct | #Samples | Average | 90th pct |
Check-In Controller | 4386 | 547 | 695 | 4587 | 555 | 691 | 7144 | 1072 | 1716 | 7393 | 951 | 1463 |
Check-Out Controller | 5971 | 1306 | 1665 | 5891 | 1326 | 1705 | ||||||