/
PTF-Performance testing of Light weight token.

PTF-Performance testing of Light weight token.

 

Overview

PTF plan to test performance for the upcoming feaute- Light weight token(LWT), which will be included in CSP1.

The goal is to compare the performance of the Sunflower Eureka CSP1 with Light weight token enabled and disabled.

For the evaluation we will run:

  • CICO (Check-In / Check-Out) tests: 8, 30, and 75 users;

  • Data Import tests (Create and Update) with 5K, 10K, 25K, 50K, and 100K records to validate sidecar performance under different data volumes.

 

 

Summary

  • All tests finished successfully without significant errors and module/sidecars crashes.

  • Data-import

    • For Create operations Sidecars 3.0.10-nb demonstrates consistent ~20–30 % improvement for 5k-100k records set

    • For Update operations, 3.0.10-nb has higher update durations (≈ 20–40 % slower) for 10k-100k records set.

  • Check-In and Check-Out. Average response times for both Check-In and Check-Out increased sharply.
    As the number of virtual users grew from 8 to 75, average response times increased from 35% to 17 times

  • Sidecars resource utiization

    a) CPU utilization graph demonstrates a correlation between the increase in workload (load) and the CPU resource consumption across the containers. No anomalies detected;
    b) Memory utilization graph illustrates stable memory usage across the containers throughout the entire testing period, no memory leaks.

  • The stability of Sidecar version 3.0.10-nb has been verified through CICO (Check-In / Check-Out) Longevity Test.

    The test, which simulated a high-load environment by running a 30-user longevity scenario for 10 hours, was completed without any issues. The system showed stability throughout the entire test , with no crashes, performance degradation.

Test Runs 

CICO (Check-In / Check-Out)

Test #

vUsers

Ramp-up, sec

Duration, sec 

Test #

vUsers

Ramp-up, sec

Duration, sec 

1

8

80

2700

2

20

200

2700

3

30

300

2700

4

75

750

2700

 

Data Import

Test #

Size

Profile

Test #

Size

Profile

1

5K

PTF-Create-2

2

10K

PTF-Create-2

3

25K

PTF-Create-2

4

50K

PTF-Create-2

5

100K

PTF-Create-2

6

5K

PTF-UpdateSuccess-6

7

10K

PTF-UpdateSuccess-6

8

25K

PTF-UpdateSuccess-6

9

50K

PTF-UpdateSuccess-6

10

100K

PTF-UpdateSuccess-6

Results/Comparisons

CICO comparison

-LWT - light weight token disabled

+LWT - light weight token enabled

 Test parameters

Response Times, milliseconds

8 vUsers 

30 vUsers 

75 vUsers 

Additional condition

-LWT

+LWT

-LWT

+LWT

-LWT

+LWT

-LWT

+LWT

-LWT

+LWT

-LWT

+LWT

Metrics

Average

90th pct.

Average

90th pct.

Average

90th pct.

Check-Out Controller

1125

1161

1334

1494

1066

1090

1325

1397

1668

1932

2540

2801

Check-In Controller

447

450

692

708

381

398

502

515

572

676

839

980

 

Data Import Comparison

This set of test were performed to check sidecars durability during high load and compare durations of data import.

Test #

Data-import test

Profile

Duration

Sunflower CSP1

-LWT

 

Duration

Sunflower CSP1 +LWT

Duration

Sunflower CSP1 +LWT
Additional testing results

Status

Test #

Data-import test

Profile

Duration

Sunflower CSP1

-LWT

 

Duration

Sunflower CSP1 +LWT

Duration

Sunflower CSP1 +LWT
Additional testing results

Status

1.1

5k MARC BIB Create

PTF - Create 2

3 min 42 s

4 min 07 s

3 min 3 s

Completed

1.2 

10k MARC BIB Create

PTF - Create 2

7 min 08 s

7 min 16 s

5 min 32 s

Completed

1.3

25k MARC BIB Create

PTF - Create 2

16 min 29 s

17 min 30 s

12 min 25 s

Completed 

1.4

50k MARC BIB Create

PTF - Create 2

36 min 4 s

35 min 54 s

26 min 20 s

Completed 

1.5

100k MARC BIB Create

PTF - Create 2

1 hr 11 min

1 hr 14 min

54 min 8 s

Completed

2.1 

5k MARC BIB Update

PTF - Updates Success - 6

2 min 31 s

3 min 41 s

2 min 44 s

Completed

2.2

10k MARC BIB Update

PTF - Updates Success - 6

4 min 11

4 min 58

5 min 33 s

Completed

2.3

25k MARC BIB Update

PTF - Updates Success - 6

9 min 49 s

12 min 33 s

12 min 51 s

Completed

2.4

50k MARC BIB Update

PTF - Updates Success - 6

19 min 27 s

24 min 16

25 min 32 s

Completed

2.5

100k MARC BIB Update

PTF - Updates Success - 6

40 min 30 s

53 min 11 s

54 min 41 s

Completed

Resource Utilization

CPU Utilization graph

Test name

-LWT

+LWT

Test name

-LWT

+LWT

CICO
8VU-30VU-75VU

image-20251021-103516.png

 

image-20251021-103443.png

Data import Create

5k - 10k - 25k- 50k - 100k

image-20251021-104119.png

 

image-20251021-104153.png

 

Data import Update

5k - 10k - 25k- 50k - 100k

image-20251021-104334.png

 

image-20251021-104249.png

 

Memory utilization

Test name

-LWT

+LWT

Test name

-LWT

+LWT

CICO
8VU-30VU-75VU

image-20251021-104827.png

 

image-20251021-104746.png

 

Data import Create

5k - 10k - 25k- 50k - 100k

image-20251021-105307.png

 

image-20251021-105348.png

 

Data import Update

5k - 10k - 25k- 50k - 100k

image-20251021-105548.png

 

image-20251021-105509.png

 

Database CPU utilization

Test name

-LWT

+LWT

Test name

-LWT

+LWT

CICO

image-20251021-105729.png

 

image-20251021-105853.png

 

Data import Create

5k - 10k - 25k- 50k - 100k

image-20251021-105953.png
image-20251021-110030.png

Data import Update

5k - 10k - 25k- 50k - 100k

image-20251021-110134.png
image-20251021-110106.png

Data-import Update with LWT and without

image-20251021-110237.png

 

Database Connections

Test name

-LWT

+LWT

Test name

-LWT

+LWT

CICO

image-20251021-120226.png

 

image-20251021-120147.png

 

Data import Create

5k - 10k - 25k- 50k - 100k

image-20251021-120316.png

 

image-20251021-120342.png

 

Data import Update

5k - 10k - 25k- 50k - 100k

image-20251021-120439.png

 

image-20251021-120413.png

 

RDS Performace metrics

Test name

-LWT

+LWT

Test name

-LWT

+LWT

CICO

image-20251021-120955.png
image-20251021-121005.png

 

image-20251021-120908.png
image-20251021-120922.png

 

image-20251021-120811.png

 

Data import Create

5k - 10k - 25k- 50k - 100k

image-20251021-121554.png
image-20251021-121604.png

 

image-20251021-121720.png
image-20251021-121728.png

 

Data import Update

5k - 10k - 25k- 50k - 100k

image-20251021-122117.png
image-20251021-122131.png

 

image-20251021-122241.png
image-20251021-122257.png

 



Appendix

Infrastructure

PTF -environment secp1

PTF -environment secp1