PTF-RTAC Test Report QCP1 vs QECP1

Owned by Mykhailo Petryshyn
Last updated: Oct 30, 2024
5 min read

 

Overview

This report outlines the findings from performance tests conducted on the edge-rtac endpoint across differing user loads of 1, 10, and 20, over 30-minute intervals. Aimed at assessing the endpoint's ability to fetch RTAC information for 50 instance UUIDs with 5 holdings under varied conditions, the study contrasts the performance benchmarks obtained on the QCP1 environment against data from QECP1. The goal was to glean insights into the service's responsiveness, throughput, and reliability, focusing on edge-rtac's performance.

 

Summary

 

Recommendations & Jiras (Optional)

Test Runs 

Test #

Test #

Env.

Number of Instances

Test Conditions

Additional Information

Status

Test #

Test #

Env.

Number of Instances

Test Conditions

Additional Information

Status

1

T1_QCP1_1VU_25Ins

QCP1

25

1VU

Duration=30 minutes. Each HTTP request contains 25 instances, that have 5 holding each.

PASSED

2

T2_QCP1_10VU_25Ins

QCP1

25

10VU

Duration=30 minutes + 10 seconds(Rump-Up)

PASSED

3

T3_QCP1_20VU_25Ins

QCP1

25

20VU

Duration=30 minutes + 20 seconds(Rump-Up)

PASSED

4

T4_QECP1_1VU_25Ins

QECP1

25

1VU

Duration=30 minutes

PASSED

5

T5_QECP1_10VU_25Ins

QECP1

25

10VU

Duration=30 minutes + 10 seconds(Rump-Up)

PASSED

6

T6_QECP1_20VU_25Ins

QECP1

25

20VU

Duration=30 minutes + 20 seconds(Rump-Up)

PASSED

7

T7_QCP1_1VU_50Ins

QCP1

50

1VU

Duration=30 minutes

PASSED

8

T8_QCP1_10VU_50Ins

QCP1

50

10VU

Duration=30 minutes + 10 seconds(Rump-Up)

PASSED

9

T9_QCP1_20VU_50Ins

QCP1

50

20VU

Duration=30 minutes + 20 seconds(Rump-Up)

PASSED

10

T10_QECP1_1VU_50Ins

QECP1

50

1VU

Duration=30 minutes

PASSED

11

T11_QECP1_10VU_50Ins

QECP1

50

10VU

Duration=30 minutes + 10 seconds(Rump-Up)

PASSED

12

T12_QECP1_20VU_50Ins

QECP1

50

20VU

Duration=30 minutes + 20 seconds(Rump-Up)

PASSED

Results

Num of Instances

Tests on
QCP1 Environment

Average, ms

pct90,
ms

Tests on
QECP1 Environment

Average, ms

pct90,

ms

DIFF QCP1/QECP1
Average, %

DIFF QCP1/QECP1
pct90, %

Num of Instances

Tests on
QCP1 Environment

Average, ms

pct90,
ms

Tests on
QECP1 Environment

Average, ms

pct90,

ms

DIFF QCP1/QECP1
Average, %

DIFF QCP1/QECP1
pct90, %

 

25 Inst

T1_QCP1_1VU_25Ins

670

963

T4_QECP1_1VU_25Ins

1272

2143

90

123

T2_QCP1_10VU_25Ins

3021

3937

T5_QECP1_10VU_25Ins

12426

17123

311

334

T3_QCP1_20VU_25Ins

6097

7432

T6_QECP1_20VU_25Ins

24712

30264

305

307

 

50 Inst

T7_QCP1_1VU_50Ins

1245

1890

T10_QECP1_1VU_50Ins

2686

4040

115

113

T8_QCP1_10VU_50Ins

5555

7318

T11_QECP1_10VU_50Ins

25780

31448

364

330

T9_QCP1_20VU_50Ins

10930

13018

T12_QECP1_20VU_50Ins

47917

59241

338

335

All data

Transaction

Number Of Samples

Average

Median

pct90

pct95

pct99

Min

Max

T1_QCP1_1VU_25Ins RTAC_GET_/rtac?instanceIds=id1,id2,id3,idn&apikey={API_KEY}&fullPeriodicals=true all holdings for instance by ID

2715

670

642

963

1386

3494

352

3494

T2_QCP1_10VU_25Ins RTAC_GET_/rtac?instanceIds=id1,id2,id3,idn&apikey={API_KEY}&fullPeriodicals=true all holdings for instance by ID

5855

3021

3027

3937

4327

5647

612

56595

T3_QCP1_20VU_25Ins RTAC_GET_/rtac?instanceIds=id1,id2,id3,idn&apikey={API_KEY}&fullPeriodicals=true all holdings for instance by ID

5694

6097

6120

7432

8121

22183

832

22202

T4_QECP1_1VU_25Ins RTAC_GET_/rtac?instanceIds=id1,id2,id3,idn&apikey={API_KEY}&fullPeriodicals=true all holdings for instance by ID

1420

1272

1259

2143

2582

23814

601

24022

T5_QECP1_10VU_25Ins RTAC_GET_/rtac?instanceIds=id1,id2,id3,idn&apikey={API_KEY}&fullPeriodicals=true all holdings for instance by ID

1415

12426

12562

17123

18968

30461

886

30465

 

T6_QECP1_20VU_25Ins RTAC_GET_/rtac?instanceIds=id1,id2,id3,idn&apikey={API_KEY}&fullPeriodicals=true all holdings for instance by ID

1407

24712

25187

30264

32470

39332

1330

39373

T7_QCP1_1VU_50Ins RTAC_GET_/rtac?instanceIds=id1,id2,id3,idn&apikey={API_KEY}&fullPeriodicals=true all holdings for instance by ID

1447

1245

1215

1890

2377

3221

660

3226

T8_QCP1_10VU_50Ins RTAC_GET_/rtac?instanceIds=id1,id2,id3,idn&apikey={API_KEY}&fullPeriodicals=true all holdings for instance by ID

3189

5555

5560

7318

8051

10686

1200

10719

T9_QCP1_20VU_50Ins RTAC_GET_/rtac?instanceIds=id1,id2,id3,idn&apikey={API_KEY}&fullPeriodicals=true all holdings for instance by ID

3223

10930

10989

13018

13742

16470

1107

16489

T10_QECP1_1VU_50Ins RTAC_GET_/rtac?instanceIds=id1,id2,id3,idn&apikey={API_KEY}&fullPeriodicals=true all holdings for instance by ID

661

2686

2635

4040

4839

26655

1218

26678

T11_QECP1_10VU_50Ins RTAC_GET_/rtac?instanceIds=id1,id2,id3,idn&apikey={API_KEY}&fullPeriodicals=true all holdings for instance by ID

675

25780

26222

31448

34392

37909

2206

37909

T12_QECP1_20VU_50Ins RTAC_GET_/rtac?instanceIds=id1,id2,id3,idn&apikey={API_KEY}&fullPeriodicals=true all holdings for instance by ID

708

47917

49289

59241

60006

83957

2046

84196

Response time graph

 

 

QCP1

QECP1

 

 

QCP1

QECP1

25 Ins.

1

image-20241029-102345.png
image-20241029-103024.png

10

20

50 Ins.

1

10

20

 

 

Throughput for Tests T1-T12

 

 

 

Resource utilization

CPU Utilization

 

 

QCP1

QECP1

 

 

QCP1

QECP1

25 Ins.

1

10

Eurika service during

20

50 Ins.

1

10

 

20

 

Memory Utilization

 

 

QCP1

QECP1

 

 

QCP1

QECP1

25 Ins.

1VU

10VU

20VU

50 Ins.

1VU

10VU

20VU

 



Errors

 

 

 

Appendix

 

Infrastructure

 

Methodology/Approach