| Table of Contents |
|---|
This document contains the results of testing Fiscal year rollover in scope of ticket
| Jira Legacy | ||||||
|---|---|---|---|---|---|---|
|
Summary
- Service CPU utilization for mod-orders-storage increased up to 60% during orders phase of 10K rollover, while during 50K and 100K orders rollover mod-orders-storage CPU utilization reaches only 20% maximum. More details...
- DB CPU utilization shows different behavior for rollovers of different data quantity. During 10K and 50k rollover DB CPU utilization increased during orders phase (31% → 83%, 61% → 85%), while during 100K rollover DB CPU utilization decreased during orders phase (65% → 30%, 67% → 40%). More details...
- Lock wait type was observed during orders phase of rollover. Lock waiting time is higher during 10K orders rollover than during 50K and 100K orders rollover. More details...
- "Out of shared memory" error occurred during 100K orders rollover with 90K open orders. This issue was reported previously in
.Jira Legacy server System JiraFOLIO Issue Tracker serverId 01505d016ccf3fe4-b8533301-3c2e368a-90f1983e-ee9b165564fc20c466b11a49 key MODFISTO-428 - "504 Gateway Time-out" error occurred during 100K orders rollover with 50K open orders. This issue was reported and marked as fixed previously (
) but should be investigated further.Jira Legacy server System JiraFOLIO Issue Tracker serverId 01505d016ccf3fe4-b8533301-3c2e368a-90f1983e-ee9b165564fc20c466b11a49 key MODORDERS-865
Recommendations and Jiras
Investigate application behavior during rollover:
- A set of improvements was prepared by dev team:
Jira Legacy server
...
FOLIO Issue Tracker serverId
...
6ccf3fe4-
...
3301-
...
368a-
...
983e-
...
. When these fixes are implemented, tests will be conducted again to assess performance and compare results with current ones.20c466b11a49 key MODFISTO-439
Recommendations and Jiras
Investigate application behavior during fiscal year rollover:
| Jira Legacy | ||||||
|---|---|---|---|---|---|---|
|
Investigate "Out of shared memory" and "504 Gateway Time-out" errors in scope of previously created tickets.
Conduct another set of tests when all the fixes are implemented.
Test Runs and Results
Test # | Scenario | Load level | Data quantity | Fiscal year rollover duration | Comments |
|---|---|---|---|---|---|
| 1. | Fiscal year rollover | 1 rollover at a time | 10K orders: 5K - open 5K - closed | 3 hours (2 hours financial rollover, 1 hour orders rollover) | No errors |
| 2. | 50K orders: 40K - open 10K - closed | 2 days 6 hours (1 day 8 hours financial rollover, 22 hours orders rollover) | No errors | ||
| 3. | 100K orders: 90K - open 10K - closed | 8 days (6 days 6 hours financial rollover, 1 day 18 hours orders rollover) | Rollover was stopped manually, "Out of shared memory" error occurred. | ||
| 4. | 100K orders: 50K - open 50K - closed | 6 days 13 hours (3 days 9 hours financial rollover, 3 days 4 hours orders rollover) | Rollover finished by itself, "504 Gateway Time-out" error occurred. |
...
Modules | Version | Task Definition | Running Tasks | CPU | Memory | MemoryReservation | MaxMetaspaceSize | Xmx |
|---|---|---|---|---|---|---|---|---|
| mod-orders | 12.6.6 | 10 | 2 | 1024 | 2048 | 1440 | 512 | 1024 |
| mod-orders-storage | 13.5.0 | 5 | 2 | 128 | 1024 | 896 | 128 | 768 |
| mod-finance | 4.7.1 | 5 | 2 | 128 | 1024 | 896 | 128 | 768 |
| mod-finance-storage | 8.4.1 | 8 | 2 | 128 | 1024 | 896 | 128 | 700 |
| okapi | 5.0.1 | 6 | 3 | 1024 | 1684 | 1440 | 512 | 922 |
| nginx-okapi | 2022.03.02 | 6 | 2 | 128 | 1024 | 896 | - | - |
| pub-okapi | 2022.03.02 | 6 | 2 | 128 | 1024 | 896 | - | 768 |
...
For other orders quantity and data structure JMeter script should be used (perf-testing/workflows-scripts/fiscal_year/FY_rollover_preparation on the GitHub). But in this case data generation will take a lot of time.
...