Table of Contents outline true
O
verview
- This document contains the results of Central Ordering testing scenarios to create order with PO lines plus Change instance connection and Receive all pieces for Ramsons [ECS] environments.
Jira Legacy
| Table of Contents | ||
|---|---|---|
|
O
verview
- This document contains the results of Central Ordering testing scenarios to create order with PO lines plus Change instance connection and Receive all pieces for Ramsons [ECS] environments.
| Jira Legacy | ||||||||||
|---|---|---|---|---|---|---|---|---|---|---|
|
Summary
- All tests were successfully completed, and all API calls were executed without errors.
- Response time for Change Instance Connection for Test №2 increased by 45% compared to Test №1 and for Test №3 Response time increased by 125% compared to Test 1. It appears that the Find or Create New Holdings action performed multiple operations, which explains the extended response time for this action.
In Test 3, the Create Order operation involved creating orders with 50 Order Lines, compared to 1 Order Lines in Test 1 and 10 Order Lines in Test 2. This explains the significant increase in response times for the Open Order and Unopen Order operations:
- Open Order: Response times increased nearly 10x in Test 3 compared to Test 1 due to the higher load (50 items per order).
- Unopen Order: Similarly, response times grew approximately 4-5x in Test 3.
For other operations (Create Order, Create Order Line, Receive Pieces, and Change Instance Connection), response times increased moderately but remained within acceptable ranges as the load increased.
Conclusion: The increase in item count per order directly impacted the performance of Open Order and Unopen Order, highlighting their sensitivity to load.
- In Test №2, the Create New Holdings operation for Change Instance Connection triggered the reindex function. As a result, we observed that mod-search initiated reindexing, as evidenced by SQL queries N1 and N4 appearing in the TOP 5 SQL Statements table. This process placed a significant load on the database and continued for over 3 hours after the tests were completed.
Test Runs
...
Test №
...
Date
...
Test Conditions
...
1
...
View Order history, View Order Line history and View Pieces history with 10, 50, 100 and 1000 changes on Data Set with 500k records
...
View Order history, View Order Line history and View Pieces history with 10, 50, 100 and 1000 changes on Data Set with 1 million records
...
12/20/24, 15:45 PM UTC
...
View Order history, View Order Line history and View Pieces history with 10, 50, 100 and 1000 changes on Data Set with 5 million records
...
Test Results
Test №1
PO with PO Lines (1) + locations (5) + Change instance connection (Move holdings)
Transactions
...
Transactions
...
- .
- The response time variation for different data sizes or changes in each Order, Order Line, or Piece history is around 5%.
- The duration for inserting one row remains consistent across different data sizes.
Test Runs
Test № | Date | Test Conditions | Results |
1 | 12/20/24, 13:15 AM UTC | View Order history, View Order Line history and View Pieces history with 10, 50, 100 and 1000 changes on Data Set with 500k records | Completed |
| 2 | 12/20/24, 13:11 AM UTC | View Order history, View Order Line history and View Pieces history with 10, 50, 100 and 1000 changes on Data Set with 1 million records | Completed |
| 3 | 12/20/24, 15:45 PM UTC | View Order history, View Order Line history and View Pieces history with 10, 50, 100 and 1000 changes on Data Set with 5 million records | Completed |
Test Results
The following table collect transactions results for tests №1, №2 and №3.
Transactions | Response Time (milliseconds) | |||||
| Data size 500k | Data size 1 million | Data size 5 million | ||||
| Average | pct90 | Average | pct90 | Average | pct90 | |
|---|---|---|---|---|---|---|
| TC_open order 10 | 813 | 2513 | 872 | 2526 | 894 | 2650 |
| TC_open order 50 | 446 | 687 | 453 | 687 | 469 | 647 |
| TC_open order 100 | 737 | 1343 | 739 | 1346 | 814 | 1722 |
| TC_open order 1000 | 670 | 884 | 657 | 858 | 680 | 838 |
| TC_open order line 10 | 1363 | 3159 | 1468 | 3464 | 1560 | 4083 |
| TC_open order line 50 | 913 | 1343 | 925 | 1591 | 936 | 1458 |
| TC_open order line 100 | 1181 | 1784 | 1170 | 1978 | 1235 | 2242 |
| TC_open order line 1000 | 1197 | 1534 | 1192 | 1485 | 1213 | 1529 |
| TC_open pieces 10 | 1268 | 2643 | 1249 | 2621 | 1269 | 2645 |
| TC_open pieces 50 | 880 | 1065 | 881 | 1254 | 915 | 1368 |
| TC_open pieces 100 | 1111 | 1844 | 1126 | 1531 | 1144 | 1811 |
| TC_open pieces 1000 | 1062 | 1286 | 1110 | 1305 | 1092 | 1489 |
| TC_view order history 10 | 541 | 1652 | 558 | 1613 | 544 | 1388 |
| TC_view order history 50 | 254 | 383 | 249 | 314 | 265 | 380 |
| TC_view order history 100 | 477 | 805 | 486 | 886 | 505 | 849 |
| TC_view order history 1000 | 452 | 647 | 445 | 528 | 464 | 681 |
| TC_view order line history 10 | 754 | 1610 | 738 | 1564 | 823 | 1974 |
| TC_view order line history 50 | 452 | 619 | 449 | 587 | 489 | 788 |
| TC_view order line history 100 | 695 | 1068 | 677 | 1037 | 718 | 968 |
| TC_view order line history 1000 | 733 | 900 | 760 | 1207 | 746 | 926 |
| TC_view pieces history 10 | 753 | 1433 | 745 | 1422 | 782 | 1801 |
| TC_view pieces history 50 | 452 | 701 | 449 | 632 | 454 | 626 |
| TC_view pieces history 100 | 670 | 1008 | 656 | 1086 | 701 | 1081 |
| TC_view pieces history 1000 | 710 | 945 | 701 | 1095 | 694 | 862 |
The following table collect API results for tests №1, №2 and №3:
- GET audit-data/acquisition/order/order_id
- GET audit-data/acquisition/order-line/order_line_id
- GET audit-data/acquisition/piece/piece_id/status-change-history
Transactions | Response Time (milliseconds) | |||||
| Data size 500k | Data size 1 million | Data size 5 million | Data size 500k | Data size 1 million | Data size 5 million | |
|---|---|---|---|---|---|---|
| Average | Average | Average | pct90 | pct90 | pct90 | |
| GET audit-data/acquisition/order/[order_10] | 27 | 28 | 34 | 72 | 74 | 89 |
| GET audit-data/acquisition/order-line/[po_line_id_10] | 29 | 27 | 29 | 77 | 70 | 90 |
| GET audit-data/acquisition/piece/]pieces_10]/status-change-history | 28 | 28 | 27 | 64 | 68 | 69 |
| GET audit-data/acquisition/order/[order_50] | 18 | 16 | 18 | 58 | 47 | 48 |
| GET audit-data/acquisition/order-line/[po_line_id_50] | 20 | 23 | 40 | 53 | 60 | 344 |
| GET audit-data/acquisition/piece/]pieces_50]/status-change-history | 17 | 20 | 19 | 44 | 51 | 51 |
| GET audit-data/acquisition/order/[order_10100] | 2722 | 2820 | 3423 | 7262 | 7459 | 8966 |
| GET audit-data/acquisition/order-line/[po_line_id_10100] | 292727 | 25 | 2932 | 7765 | 7063 | 9075 |
| GET audit-data/acquisition/piece/]pieces_10100]/status-change-history | 282228 | 21 | 2723 | 6461 | 6862 | 6960 |
| GET audit-data/acquisition/order/[order_501000] | 1847 | 1652 | 1848 | 58100 | 4798 | 4881 |
| GET audit-data/acquisition/order-line/[po_line_id_501000] | 20122 | 23126 | 40132 | 53212 | 60231 | 344222 |
| GET audit-data/acquisition/piece/]pieces_501000]/status-change-history | 1760 | 2050 | 1953 | 44117 | 51105 | 51 |
| GET audit-data/acquisition/order/[order_100] | 22 | 20 | 23 | 62 | 59 | 66 |
| GET audit-data/acquisition/order-line/[po_line_id_100] | 27 | 25 | 32 | 65 | 63 | 75 |
| GET audit-data/acquisition/piece/]pieces_100]/status-change-history | 22 | 21 | 23 | 61 | 62 | 60 |
| GET audit-data/acquisition/order/[order_1000] | 47 | 52 | 48 | 100 | 98 | 81 |
| GET audit-data/acquisition/order-line/[po_line_id_1000] | 122 | 126 | 132 | 212 | 231 | 222 |
| GET audit-data/acquisition/piece/]pieces_1000]/status-change-history | 60 | 50 | 53 | 117 | 105 | 9696 |
The following table collect INSERT INTO duration results for different data size.
| Data size 500k | Data size 1 million | Data size 5 million | Data size 1 million | Data size 5 million | |
|---|---|---|---|---|---|
| INSERT INTO 1 row | INSERT INTO 1 row | INSERT INTO 1 row | INSERT INTO 100k | INSERT INTO 100k | |
| Order acquisition_order_log | 201 msec | 211 msec | 236 msec | 4 secs 361 msec | 5 secs 330 msec |
| Order Line acquisition_order_line_log | 198 msec | 222 msec | 191 msec | 6 secs 857 msec | 7 secs 73 msec |
| Pieces acquisition_piece_log | 183 msec | 239 msec | 303 msec | 4 secs 399 msec | 4 secs 393 msec |
The following table collect size for tables and indexes.
| table_name | table_size | indexes_size | row_count |
|---|---|---|---|
| cs00000int_mod_audit.acquisition_order_log | 2811 MB | 358 MB | 5.300.543 |
| cs00000int_mod_audit.acquisition_order_line_log | 11 GB | 366 MB | 5.400.089 |
| cs00000int_mod_audit.acquisition_piece_log | 3957 MB | 374 MB | 5.220.071 |
Comparison
Test №1-№2-№3
- The following table compares the
...
- average response times for all transactions between tests №1, №2 and №3.
- Open Order: Response times increased nearly 10x in Test 3 compared to Test 1 due to the higher load (50 items per order).
- Unopen Order: Similarly, response times grew approximately 4-5x in Test 3.
- The following table compares the average response times for API calls:
- GET audit-data/acquisition/order/order_id
- GET audit-data/acquisition/order-line/order_line_id
- GET audit-data/acquisition/piece/piece_id/status-change-history
Resource utilization for Test #1-2-3
| Expand | ||||||||||||||||||||||||||||||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| ||||||||||||||||||||||||||||||||||||
| ||||||||||||||||||||||||||||||||||||
Service CPU Utilization
Here we can see that mod-search initiated reindexing for Test #2all modules show a stable trend..
Service Memory Utilization
Here we can see that all modules show a stable trend.
Kafka metrics
OpenSearch Data Nodes metrics
...
DB CPU Utilization
DB CPU in the average were 20% for Scenario 1, 30% for Scenario 2 and 40% for Scenario 3was 12% for all Scenarios.
DB Connections
Max DB connections was 15601250.
DB load
...
PTF - environment Ramsons (rcon)
10 m6g.2xlarge EC2 instances located in US East (N. Virginia)us-east-1 [Number of ECS instances, instance type, location region]
1 instance of db.r6.xlarge database instance: Writer instance
OpenSearch
domain: ptf-test
Number of nodes: 7
Version: OpenSearch_2_13_R20240520-P5
MSK - fse-tenant
4 kafka.m7g.xlarge brokers in 2 zones
Apache Kafka version 3.7.x
EBS storage volume per broker 300 GiB
auto.create.topics.enable=true
log.retention.minutes=480
default.replication.factor=3
...