[Trillium] Funds Batch Edit testing
Overview
This document contains the results of Funds Batch Edit testing on asptc [NON-ECS].
Summary
Funds Batch Edit tests with 100 - 1000 funds per ledger passed successfully.
Main transactions average response times at 1000 funds:
Open Ledger - 1.76s,
Open Batch Allocations - 1.07s,
Recalculate - 250ms,
Funds Batch Edit Save - 5.12s.
A noticeable RT increase is observed between 200 and 500 funds:
Compared 100 to 200 funds: ~2–44% increase (moderate growth),
Compared 200 to 500 funds: ~117–255% increase (significant change),
Compared 500 to 1000 funds: ~2–68% increase (more stable/linear growth pattern).
Service CPU utilization during 1000-fund execution:
mod-orders-storage - 90%,
mod-finance-storage - 55%,
mod-finance - 21%.
RDS CPU utilization during 1000 funds editing - 11%.
Test Runs
Test | Number of Funds | Status |
|---|---|---|
Test 1 | 100 | Completed |
Test 2 | 200 | Completed |
Test 3 | 500 | Completed |
Test 4 | 700 | Completed |
Test 5 | 1000 | Completed |
Results
This table presents Funds Batch Edit performance results across different dataset sizes:
Response times generally increase with the growth in the number of funds. However, a noticeable increase is observed between 200 and 500 funds, where the average response times increase more significantly compared to other ranges.
Comparing 100–200 funds, the increase is moderate, averaging about 2–44% across transactions.
Comparing 200-500 funds led to a noticeable increase, averaging around 117–255%, depending on the transaction.
Comparing 500-1000 funds, the growth rate stabilizes, showing a relatively linear increase of about ~2–68%.
Overall, a gradual RT growth pattern is observed, with one noticeable step-up between 200 and 500 funds, after which the growth returns to a more predictable pattern.
Transaction | 100 funds | 200 funds | 500 funds | 700 funds | 1000 funds | |||||
Response times (ms) | ||||||||||
Average | 90% Line | Average | 90% Line | Average | 90% Line | Average | 90% Line | Average | 90% Line | |
TC_OpenLedger | 389 | 361 | 465 | 424 | 1615 | 1507 | 1745 | 1919 | 1760 | 1724 |
TC_OpenBatchAllocations | 319 | 319 | 324 | 365 | 1045 | 1067 | 1046 | 1061 | 1070 | 1117 |
TC_Recalculate | 58 | 52 | 75 | 79 | 163 | 143 | 166 | 184 | 250 | 308 |
TC_FundsBatchEditSave | 875 | 930 | 1123 | 1136 | 3984 | 4296 | 4491 | 4557 | 5124 | 5200 |
PUT_finance/finance-data (Save) | 428 |
| 618 |
| 1433 | 1706 | 1817 | 1830 | 2410 | 2442 |
Comparison
This table presents a comparison of the average response times for Sunflower (before improvements) and Trillium (after improvements):
Fund/Budget Count | PUT_finance/finance-data(save) | |
before improvements | after improvements | |
200 | 1.9s | 618ms |
500 | 2.1s | 1.43s |
700 | 3.6s | 1.82s |
1000 | 6min 52sec (failed) | 2.41s |
Resource utilization
Instance CPU Utilization
Service CPU Utilization
CPU utilization during 1000 funds test:
mod-orders-storage - 90% (percentage corresponds to the 64 CPU units allocated to the service)
mod-finance-storage - 55%
mod-finance - 21%
Service Memory Utilization
Memory usage showed stable trends with no spikes or growths:
Database metrics
RDS CPU utilization during 1000 funds test - 11%.
Appendix
Infrastructure
PTF - environment Trillium NON-ECS (asptc)
8 r7i.2xlarge EC2 instances located in US East (N. Virginia)us-east-1
1 instance of db.r7g.xlarge database instance: Writer instance
MSK ptf-loc
6 m7g.large brokers in 2 zones
Apache Kafka version 3.9.x
EBS storage volume per broker 300 GiB
OpenSearch 2.19 ptf-loc cluster
r7g.xlarge.search 4 data nodes
m7g.large.search 3 dedicated master nodes