Table of Contents outline true
...
- This document contains the results of Central Ordering testing scenarios to create order with PO lines plus Change instance connection and Receive all pieces Acquisition Audit Log Performance testing for Ramsons [ECS] environments.
...
Test № | Date | Test Conditions | Results |
1 | 12/20/24, 13:15 AM UTC | View Order history, View Order Line history and View Pieces history with 10, 50, 100 and 1000 changes on Data Set with 500k records | Completed |
| 2 | 12/20/24, 13:11 AM UTC | View Order history, View Order Line history and View Pieces history with 10, 50, 100 and 1000 changes on Data Set with 1 million records | Completed |
| 3 | 12/20/24, 15:45 PM UTC | View Order history, View Order Line history and View Pieces history with 10, 50, 100 and 1000 changes on Data Set with 5 million records | Completed |
Test Results
The following table collect transactions results for tests №1, №2 and №3.
Transactions | Response Time (milliseconds) | |||||
| Data size 500k | Data size 1 million | Data size 5 million | ||||
| Average | pct90 | Average | pct90 | Average | pct90 | |
|---|---|---|---|---|---|---|
| TC_open order 10 | 813 | 2513 | 872 | 2526 | 894 | 2650 |
| TC_open order 50 | 446 | 687 | 453 | 687 | 469 | 647 |
| TC_open order 100 | 737 | 1343 | 739 | 1346 | 814 | 1722 |
| TC_open order 1000 | 670 | 884 | 657 | 858 | 680 | 838 |
| TC_open order line 10 | 1363 | 3159 | 1468 | 3464 | 1560 | 4083 |
| TC_open order line 50 | 913 | 1343 | 925 | 1591 | 936 | 1458 |
| TC_open order line 100 | 1181 | 1784 | 1170 | 1978 | 1235 | 2242 |
| TC_open order line 1000 | 1197 | 1534 | 1192 | 1485 | 1213 | 1529 |
| TC_open pieces 10 | 1268 | 2643 | 1249 | 2621 | 1269 | 2645 |
| TC_open pieces 50 | 880 | 1065 | 881 | 1254 | 915 | 1368 |
| TC_open pieces 100 | 1111 | 1844 | 1126 | 1531 | 1144 | 1811 |
| TC_open pieces 1000 | 1062 | 1286 | 1110 | 1305 | 1092 | 1489 |
| TC_view order history 10 | 541 | 1652 | 558 | 1613 | 544 | 1388 |
| TC_view order history 50 | 254 | 383 | 249 | 314 | 265 | 380 |
| TC_view order history 100 | 477 | 805 | 486 | 886 | 505 | 849 |
| TC_view order history 1000 | 452 | 647 | 445 | 528 | 464 | 681 |
| TC_view order line history 10 | 754 | 1610 | 738 | 1564 | 823 | 1974 |
| TC_view order line history 50 | 452 | 619 | 449 | 587 | 489 | 788 |
| TC_view order line history 100 | 695 | 1068 | 677 | 1037 | 718 | 968 |
| TC_view order line history 1000 | 733 | 900 | 760 | 1207 | 746 | 926 |
| TC_view pieces history 10 | 753 | 1433 | 745 | 1422 | 782 | 1801 |
| TC_view pieces history 50 | 452 | 701 | 449 | 632 | 454 | 626 |
| TC_view pieces history 100 | 670 | 1008 | 656 | 1086 | 701 | 1081 |
| TC_view pieces history 1000 | 710 | 945 | 701 | 1095 | 694 | 862 |
The following table collect API results for tests №1, №2 and №3:
- GET audit-data/acquisition/order/order_id
- GET audit-data/acquisition/order-line/order_line_id
- GET audit-data/acquisition/piece/piece_id/status-change-history
Transactions | Response Time (milliseconds) | |||||
| Data size 500k | Data size 1 million | Data size 5 million | Data size 500k | Data size 1 million | Data size 5 million | |
|---|---|---|---|---|---|---|
| Average | Average | Average | pct90 | pct90 | pct90 | |
| GET audit-data/acquisition/order/[order_10] | 27 | 28 | 34 | 72 | 74 | 89 |
| GET audit-data/acquisition/order-line/[po_line_id_10] | 29 | 27 | 29 | 77 | 70 | 90 |
| GET audit-data/acquisition/piece/]pieces_10]/status-change-history | 28 | 28 | 27 | 64 | 68 | 69 |
| GET audit-data/acquisition/order/[order_50] | 18 | 16 | 18 | 58 | 47 | 48 |
| GET audit-data/acquisition/order-line/[po_line_id_50] | 20 | 23 | 40 | 53 | 60 | 344 |
| GET audit-data/acquisition/piece/]pieces_50]/status-change-history | 17 | 20 | 19 | 44 | 51 | 51 |
| GET audit-data/acquisition/order/[order_100] | 22 | 20 | 23 | 62 | 59 | 66 |
| GET audit-data/acquisition/order-line/[po_line_id_100] | 27 | 25 | 32 | 65 | 63 | 75 |
| GET audit-data/acquisition/piece/]pieces_100]/status-change-history | 22 | 21 | 23 | 61 | 62 | 60 |
| GET audit-data/acquisition/order/[order_1000] | 47 | 52 | 48 | 100 | 98 | 81 |
| GET audit-data/acquisition/order-line/[po_line_id_1000] | 122 | 126 | 132 | 212 | 231 | 222 |
| GET audit-data/acquisition/piece/]pieces_1000]/status-change-history | 60 | 50 | 53 | 117 | 105 | 96 |
The following table collect INSERT INTO duration results for different data size.
| Data size 500k | Data size 1 million | Data size 5 million | Data size 1 million | Data size 5 million | |
|---|---|---|---|---|---|
| INSERT INTO 1 row | INSERT INTO 1 row | INSERT INTO 1 row | INSERT INTO 100k | INSERT INTO 100k | |
| Order acquisition_order_log | 201 msec | 211 msec | 236 msec | 4 secs 361 msec | 5 secs 330 msec |
| Order Line acquisition_order_line_log | 198 msec | 222 msec | 191 msec | 6 secs 857 msec | 7 secs 73 msec |
| Pieces acquisition_piece_log | 183 msec | 239 msec | 303 msec | 4 secs 399 msec | 4 secs 393 msec |
The following table collect size for tables and indexes.
| table_name | table_size | indexes_size | row_count |
|---|---|---|---|
| cs00000int_mod_audit.acquisition_order_log | 2811 MB | 358 MB | 5.300.543 |
| cs00000int_mod_audit.acquisition_order_line_log | 11 GB | 366 MB | 5.400.089 |
| cs00000int_mod_audit.acquisition_piece_log | 3957 MB | 374 MB | 5.220.071 |
Comparison
Test №1-№2-№3
- The following table compares the average response times for all transactions between tests №1, №2 and №3.
- The following table compares the average response times for API calls:
- GET audit-data/acquisition/order/order_id
- GET audit-data/acquisition/order-line/order_line_id
- GET audit-data/acquisition/piece/piece_id/status-change-history
Resource utilization for Test #1-2-3
...
Here we can see that all modules show a stable trend.
Kafka metrics
OpenSearch Data Nodes metrics
...
PTF - environment Ramsons (rcon)
10 m6g.2xlarge EC2 instances located in US East (N. Virginia)us-east-1 [Number of ECS instances, instance type, location region]
1 instance of db.r6.xlarge database instance: Writer instance
OpenSearch
domain: ptf-test
Number of nodes: 7
Version: OpenSearch_2_13_R20240520-P5
MSK - fse-tenant
4 kafka.m7g.xlarge brokers in 2 zones
Apache Kafka version 3.7.x
EBS storage volume per broker 300 GiB
auto.create.topics.enable=true
log.retention.minutes=480
default.replication.factor=3
...