Effect of horizontal scaling on Eureka Sunflower [non-ECS]

Effect of horizontal scaling on Eureka Sunflower [non-ECS]

Overview

  • The main idea of the test to understand the effect of horizontal scaling on Eureka by perform a CICO test of 20 users and 40 users on a non-HA and HA environment. Additional task is to check how availability zone affect the performance switching between us-east-1a and us-east-1b.PERF-1225: Effect of horizontal scaling on Eureka

Summary

  • All tests completed successfully.

  • Testing results in non-HA and HA environments show that non-HA mode perform better than HA

    • 20 vUsers tests response time slightly faster in non-HA 

      • CI 609ms - 17%(105ms), CO 1346ms - 9%(125ms)

    • 40 vUsers tests response time faster in non-HA 

      • CI 870ms - 54%(471ms), CO 2039ms - 23%(486ms)

  • Testing results in different AZ (availability zone) reveal light performance improvement (3%) in 20 vUser tests in us-east-1a zone (the same database AZ) but for 40vUsers test improvement in us-east-1a was only in HA mode (5%). In non-HA response times better in us-east-1b 10% in average for both CI and CO.

  • Additional test runs in ECS environment show that performance of CI/CO flows was not affected by horizontal scaling (non-HA, HA, 4 tasks). It perform the same or slightly better - 30-60 ms.

Non-HA

 

 

 

 

 

vUsers

Label

us-east-1a

us-east-1b

Difference

Faster AZ

20

CI

547 ms

555 ms

1% (8 ms)

1a

20

CO

1306 ms

1326 ms

2% (20 ms)

1a

40

CI

1072 ms

951 ms

11% (121 ms)

1b

40

CO

2435 ms

2237 ms

8% (198 ms)

1b

HA

 

 

 

 

 

vUsers

Label

us-east-1a

us-east-1b

Difference

Faster AZ

20

CI

588 ms

657 ms

12% (69 ms)

1a

20

CO

1374 ms

1413 ms

3% (39 ms)

1a

40

CI

983 ms

1032 ms

5% (49 ms)

1a

40

CO

2322 ms

2432 ms

5% (110 ms)

1a

  • Additional test results on ECS environment in HA and non-HA mode with 40 vUsers. This effort also include 2 tests on HA mode (2 tasks for CICO involved modules) and 2 tests with 4 tasks for CICO involved modules:

    • In a new environment comparing HA and non-HA we see that Check-In/Check-Out perform 5% better (CI - 28ms, CO - 58ms) in HA mode

 

HA

non-HA

 

 

 

 

HA

non-HA

 

 

 

Label

Average

Average

Delta, ms

Delta, %

Faster mode

CI

447

475

28

6%

HA 

CO

1255

1313

58

5%

HA 

  • Raising number of tasks to 4 do not affect performance

Task count in CICO involved modules - 4

Run #1 HA

Run #2 HA

Previous HA

Label

Average

Average

Average

CI

478

436

447

CO

1326

1234

1255

Test Runs 

Test #

vUsers

Availability mode

Availability zone

Ramp-up, sec

Duration, sec 

Test #

vUsers

Availability mode

Availability zone

Ramp-up, sec

Duration, sec 

1

20

HA

us-east-1a / us-east-1b

200

2700

2

40

HA

us-east-1a / us-east-1b

400

2700

3

20

non-HA

us-east-1a / us-east-1b

200

2700

4

40

non-HA

us-east-1a / us-east-1b

400

2700

5

20

HA

us-east-1a

200

2700

6

40

HA

us-east-1a

400

2700

7

20

non-HA

us-east-1a

200

2700

8

40

non-HA

us-east-1a

400

2700

9

20

HA

us-east-1b

200

2700

10

40

HA

us-east-1b

400

2700

11

20

non-HA

us-east-1b

200

2700

12

40

non-HA

us-east-1b

400

2700

 

Results

Test results

Tests # 1, 2, 3, 4

Response times (ms)

Tests # 1, 2, 3, 4

Response times (ms)

 

20vUsers non-HA (test #3)

20vUsers HA (test #1)

40vUsers non-HA (test #4)

40vUsers HA (test #2)

Label

#Samples

Average

90th pct

#Samples

Average

90th pct

#Samples

Average

90th pct

#Samples

Average

90th pct

Check-In Controller

4452

609

777

4340

714

1031

7594

870

1304

7115

1341

2229

Check-Out Controller

5872

1346

1679

5768

1471

1966

10280

2039

3064

9356

2525

3911

POST_circulation/check-in-by-barcode (Submit_barcode_checkin)

4453

275

372

4340

314

489

7595

432

706

7119

523

880

POST_circulation/check-out-by-barcode (Submit_barcode_checkout)

5872

608

790

5771

667

971

10281

1000

1738

9364

1290

2295

Tests #7, 8, 11, 12

Response times (ms)

Tests #7, 8, 11, 12

Response times (ms)

 

20vUsers non-HA us-east-1a

20vUsers non-HA us-east-1b

40vUsers non-HA us-east-1a

40vUsers non-HA us-east-1b

Label

#Samples

Average

90th pct

#Samples

Average

90th pct

#Samples

Average

90th pct

#Samples

Average

90th pct

Check-In Controller

4386

547

695

4587

555

691

7144

1072

1716

7393

951

1463

Check-Out Controller

5971

1306

1665

5891

1326

1705