Table of Contents

outline	true

O

verview

This document contains the results of Central Ordering testing scenarios to create order with PO lines plus Change instance connection and Receive all pieces for Ramsons [ECS] environments.

Jira Legacy

server	System Jira
columnIds	issuekey,summary,issuetype,created,updated,duedate,assignee,reporter,priority,status,resolution
columns	key,summary,type,created,updated,due,assignee,reporter,priority,status,resolution
serverId	01505d01-b853-3c2e-90f1-ee9b165564fc
key	PERF-1027

Summary

...

In Test 3, the Create Order operation involved creating orders with 50 Order Lines, compared to 1 Order Lines in Test 1 and 10 Order Lines in Test 2. This explains the significant increase in response times for the Open Order and Unopen Order operations:

Open Order: Response times increased nearly 10x in Test 3 compared to Test 1 due to the higher load (50 items per order).
Unopen Order: Similarly, response times grew approximately 4-5x in Test 3.

For other operations (Create Order, Create Order Line, Receive Pieces, and Change Instance Connection), response times increased moderately but remained within acceptable ranges as the load increased.

Conclusion: The increase in item count per order directly impacted the performance of Open Order and Unopen Order, highlighting their sensitivity to load.

...

Table of Contents

outline	true

Overview

This document contains the results of Acquisition Audit Log Performance testing for Ramsons [ECS] environments.

Jira Legacy

server	System Jira
columnIds	issuekey,summary,issuetype,created,updated,duedate,assignee,reporter,priority,status,resolution
columns	key,summary,type,created,updated,due,assignee,reporter,priority,status,resolution
serverId	01505d01-b853-3c2e-90f1-ee9b165564fc
key	PERF-1027

Summary

All tests were successfully completed, and all API calls executed without errors.
The response time variation for different data sizes or changes in each Order, Order Line, or Piece history is around 5%.
The duration for inserting one row remains consistent across different data sizes.
All test were executed without any load on background for our system.

Test Runs

Test №	Date	Test Conditions	Results
1	12/1220/24, 913:48 15 AM UTC	PO with PO Lines (1) + locations (5) + Change instance connection (Move holdings)View Order history, View Order Line history and View Pieces history with 10, 50, 100 and 1000 changes on Data Set with 500k records	Completed
2	12/1220/24, 11:32 AM UTCPO with PO Lines (10) + locations (3) + Change instance connection (Create new holdings)13:11 AM UTC	View Order history, View Order Line history and View Pieces history with 10, 50, 100 and 1000 changes on Data Set with 1 million records	Completed
3	12/1220/24, 115:21 45 PM UTC	PO with PO Lines (50) + locations (3) + Change instance connection (Move holdings, Find or Create new holdings)View Order history, View Order Line history and View Pieces history with 10, 50, 100 and 1000 changes on Data Set with 5 million records	Completed

Test Results

Test №1

PO with PO Lines (1) + locations (5) + Change instance connection (Move holdings)The following table collect transactions results for tests №1, №2 and №3.

Transactions

Response Time (milliseconds)

Data size 500k

Data size 1 million

Data size 5 million

Average

pct90

pct95

pct99

Average	pct90

pct95pct99

	Average	pct90

pct95

pct99

TC_open order 10

813

2513

25132513

872

2526

25262526

894

2650

TC_open order 50

446

687

731735

453

687

796

800

469

647

775775

TC_open order 100

737

1343

13431343

739

13461346

1346

814

17221722

	1722
TC_open order 1000	670	884

11491149

657

858

10091009

680

838

877880

TC_open order line 10

1363

31593159

3159

1468

34643464

3464

1560

40834083

	4083
TC_open order line 50	913	1343

17121730

925

1591

19101910

936

1458

16261632


TC_open order line 100	1181	1784

1981

1170

1978

19961996

1235

2242

2566

TC_open order line 1000

1197

1534

18191937

1192

1485

16351699

1213

1529

1976

2092

TC_open pieces 10

1268

26432643

2643

1249

2621

1269

2645

26452645

TC_open pieces 50

880

1065

18711913

881

1254

14751499

915

1368

17291739


TC_open pieces 100	1111	1844

18691869

1126

1531

15791668

1144

1811

18451845

TC_open pieces 1000

1062

1286

14261462

1110

1305

16311803

1092

1489

16341639


TC_view order history 10	541	1652

16521652558161316131613544138813881388

	558	1613	544	1388
TC_view order history 50	254	383	249	314	265	380
TC_view order history 100	477	805	486	886	505	849
TC_view order history 1000	452	647	445	528	464	681
TC_view order line history

50

10

754

254

1610

383

738

474

1564

477

823

249314384387265380537537

1974
TC_view order line history 50	452	619	449	587	489	788
TC_view order line history 100

477

695

805

1068

829

677

858

1037

486886886886505849856856968

718	968
TC_view order line history 1000

452647787787445528591591464681775

733	900	760	1207	746	926
TC_view pieces history 10	753	1433	745	1422	782	1801
TC_view

order line history 10754161016101610738156415641564823197419741974

pieces history 50	452	701	449	632	454	626
TC_view pieces history 100	670	1008	656	1086	701	1081
TC_view

order line

pieces history

50

1000

452

710

619

945

816

701

816

1095

449

694

587629634489788788788TC_view order line history 100695106812741307677103711321132718968968968TC_view order line history 100073390097398876012071207120774692611511151TC_view pieces history 10753143314331433745142214221422782180118011801TC_view pieces history 50452701774783449632797869454626692695TC_view pieces history 100670100811131133656108611401140701108111601160TC_view pieces history 1000710945945945701109510951095694862906910

Transactions

Response Time (milliseconds)Data size 500kData size 1 millionData size 5 millionData size 500kData size 1 millionData size 5 millionAverageAverageAveragepct90pct90pct90

862

The following table collect API results for tests №1, №2 and №3:

GET audit-data/acquisition/order/order_id
GET audit-data/acquisition/order-line/order_line_id
GET audit-data/acquisition/piece/piece_id/status-change-history

222

	Data size 500k	Data size 1 million	Data size 5 million	Data size 500k	Data size 1 million	Data size 5 million
Transactions	Response Time (milliseconds)
	Average	Average	Average	pct90	pct90	pct90
GET audit-data/acquisition/order/[order_10]	27	28	34	72	74	89
GET audit-data/acquisition/order-line/[po_line_id_10]	29	27	29	77	70	90
GET audit-data/acquisition/piece/]pieces_10]/status-change-history	28	28	27	64	68	69
GET audit-data/acquisition/order/[order_50]	18	16	18	58	47	48
GET audit-data/acquisition/order-line/[po_line_id_50]	20	23	40	53	60	344
GET audit-data/acquisition/piece/]pieces_50]/status-change-history	17	20	19	44	51	51
GET audit-data/acquisition/order/[order_100]	22	20	23	62	59	66
GET audit-data/acquisition/order-line/[po_line_id_100]	27	25	32	65	63	75
GET audit-data/acquisition/piece/]pieces_100]/status-change-history	22	21	23	61	62	60
GET audit-data/acquisition/order/[order_101000]	2747	2852	3448	72100	7498	8981
GET audit-data/acquisition/order-line/[po_line_id_101000]	29122	27126	29132	77212	70231	90222
GET audit-data/acquisition/piece/]pieces_101000]/status-change-history	28	28	27	64	68	69
GET audit-data/acquisition/order/[order_50]	18	16	18	58	47	48
GET audit-data/acquisition/order-line/[po_line_id_50]	20	23	40	53	60	344
GET audit-data/acquisition/piece/]pieces_50]/status-change-history	17	20	19	44	51	51
GET audit-data/acquisition/order/[order_100]	22	20	23	62	59	66
GET audit-data/acquisition/order-line/[po_line_id_100]	27	25	32	65	63	75
GET audit-data/acquisition/piece/]pieces_100]/status-change-history	22	21	23	61	62	60
GET audit-data/acquisition/order/[order_1000]	47	52	48	100	98	81
GET audit-data/acquisition/order-line/[po_line_id_1000]	122	126	132	212	231		60	50	53	117	105	96

The following table collect INSERT INTO duration results for different data size.

	Data size 500k	Data size 1 million	Data size 5 million	Data size 1 million	Data size 5 million
	INSERT INTO 1 row	INSERT INTO 1 row	INSERT INTO 1 row	INSERT INTO 100k	INSERT INTO 100k
Order acquisition_order_log	201 msec	211 msec	236 msec	4 secs 361 msec	5 secs 330 msec
Order Line acquisition_order_line_log	198 msec	222 msec	191 msec	6 secs 857 msec	7 secs 73 msec
Pieces acquisition_piece_log	183 msec	239 msec	303 msec	4 secs 399 msec	4 secs 393 msec

The following table collect size for tables and indexes.

table_name	table_size	indexes_size	row_count
cs00000int_mod_audit.acquisition_order_log	2811 MB	358 MB	5.300.543
cs00000int_mod_audit.acquisition_order_line_log	11 GB	366 MB	5.400.089
cs00000int_mod_audit.acquisition_piece_log	3957 MB	374 MB	5.220.071

Comparison

Test №1-№2-№3

The following table compares the average response times for all transactions between tests №1, №2 and №3.

Image Added

The following table compares the average response times for API calls:
- GET audit-data/acquisition/

...

Comparison

Test №1-№2-№3

The following table compares the results between tests №1, №2 and №3.

Open Order: Response times increased nearly 10x in Test 3 compared to Test 1 due to the higher load (50 items per order).
Unopen Order: Similarly, response times grew approximately 4-5x in Test 3.

...

- order/order_id
- GET audit-data/acquisition/order-line/order_line_id
- GET audit-data/acquisition/piece/piece_id/status-change-history

Image Added

Resource utilization for Test #1-2-3

Expand

title	Resource utilization table

CPU		RAM
mod-inventory-b	1.40%	mod-inventory-b	75%
mod-search-b	0.20%	mod-finance-b	68%
mod-inventory-storage-b	0.10%	mod-orders-b	55%
mod-orders-storage-b	0.10%	mod-orders-storage-b	44%
mod-orders-b	0.10%	mod-audit-b	39%
mod-audit-b	0.10%	mod-search-b	39%
mod-finance-b	0.10%	mod-finance-storage-b	32%
mod-finance-storage-b	0.10%	mod-inventory-storage-b	23%

Service CPU Utilization

Here we can see that mod-search initiated reindexing for Test #2all modules show a stable trend..

Service Memory Utilization

Here we can see that all modules show a stable trend.

Kafka metrics

Image RemovedImage Added

OpenSearch Data Nodes metrics

Image RemovedImage Added

DB CPU Utilization

DB CPU in the average were 20% for Scenario 1, 30% for Scenario 2 and 40% for Scenario 3was 12% for all Scenarios.

DB Connections

Max DB connections was 15601250.

DB load

...

PTF - environment Ramsons (rcon)

10 m6g.2xlarge EC2 instances located in US East (N. Virginia)us-east-1 [Number of ECS instances, instance type, location region]
1 instance of db.r6.xlarge database instance: Writer instance
OpenSearch
- domain: ptf-test
- Number of nodes: 7
- Version: OpenSearch_2_13_R20240520-P5
MSK - fse-tenant
- 4 kafka.m7g.xlarge brokers in 2 zones
- Apache Kafka version 3.7.x
- EBS storage volume per broker 300 GiB
- auto.create.topics.enable=true
- log.retention.minutes=480
- default.replication.factor=3

...

Open Order tub
Open Order (with 1000 changes in history)
View Order history
Open Order Line (with 1000 changes in history)
View Order Line history
Open Pieces (with 1000 changes in history)
View Pieces history

Attached files

Ramsons ECS (rcon) module configuration parameters:

JMeter script:JMeter script:

View file

name	RCON_history.jmx
height	250

Version	Old Version 13	New Version Current
Changes made by	Stanislav Nehrii	Stanislav Nehrii
Saved on	Dec 24, 2024	Dec 26, 2024

Versions Compared

Key

O

verview

Summary

Overview

Summary

Test Results

Test №1

Comparison

Test №1-№2-№3

Comparison

Test №1-№2-№3

Resource utilization for Test #1-2-3

Service CPU Utilization

Service Memory Utilization

Kafka metrics

OpenSearch Data Nodes metrics

DB CPU Utilization

DB Connections

DB load

Attached files

Page Comparison

Versions Compared

Key

O

verview

Summary

Overview

Summary

Test Results

Test №1

Comparison

Test №1-№2-№3

Comparison

Test №1-№2-№3

Resource utilization for Test #1-2-3

Service CPU Utilization

Service Memory Utilization

Kafka metrics

OpenSearch Data Nodes metrics

DB CPU Utilization

DB Connections

DB load

Attached files