| Table of Contents | ||
|---|---|---|
|
...
- These tests are run to investigate the performance difference of mod-data-export with logLevel=info vs logLevel=warn for Juniper release. This testing was part of
where we observed that mod-data-export is continuously writing a lot of data to log. For 100K DE job, mod-data-export is writing 42Million records. This could also result in a crash if there is not enough CPU, memory allocated to mod-data-export.Jira Legacy server System Jira columnIds issuekey,summary,issuetype,created,updated,duedate,assignee,reporter,priority,status,resolution columns key,summary,type,created,updated,due,assignee,reporter,priority,status,resolution serverId 01505d01-b853-3c2e-90f1-ee9b165564fc key MDEXP-394
...
- .
...
Jira Legacy
| server | System Jira |
|---|---|
| columnIds | issuekey,summary,issuetype,created,updated,duedate,assignee,reporter,priority,status,resolution |
| columns | key,summary,type,created,updated,due,assignee,reporter,priority,status,resolution |
| serverId | 01505d01-b853-3c2e-90f1-ee9b165564fc |
| key | PERF-142 |
- mod-data-export-4.1.1 (snapshot version)
- mod-source-record-storage-5.1.4
- mod-source-record-manager-3.1.3
- okapi-4.8.2
- mod-authtoken-2.8.0
...
- 8 million inventory records
- 74 FOLIO back-end modules deployed in 144 ECS services
- 3 okapi ECS services
- 12 m5.large EC2 instances
- 1 writer db.r6g.xlarge 1 reader db.r6g.xlarge AWS RDS instance
- INFO logging level / WARN looging level
...
- With WARN level logging, seeing 9% improvement in memory utilization
- data-export/job-executions API is polled every 3 seconds vs 5 seconds in Goldenrod. This means there are more requests overall but it also improves data-export execution. Data-export job completes faster(around 2 seconds) compared to the previous release. Other than that, data-export is stable and see performance improvements compared to the Goldenrod releaseWith WARN level logging, No bumps in the memory were observed. Memory utilization stays stable for multiple Data Export job runs.
- A new feature added for data-export is it will kill failed job every 6 hours
- How many data-export jobs can run in parallel? Multiple jobs can run in parallel and 2.75 Million instance records can be exported at any given point in time. If we try to export more than that for example 3 Million instance records, mod-data-export crashes with OOM(Out Of Memory)
...
1 user - INFO level logging vs WARN level logging
Test | Total instances | Duration | mod-data-export log level | Total time to complete exporting all instances, holdings, and items | Total records logged in CloudWatch |
| 1. | 100,000 |
Fast queries involved during data-export job:
| 1 hour 6 minutes | INFO | 2.97 minutes | 42.5 Million | |
| 2. | 100,000 | 30 min | WARN | 18 minutes |
Slow API taking more than 100 ms to return for 1 user:
...
| 120K |
Total records logged for 100K, INFO vs WARN
INFO - 42.5 Million records
WARN - ~120K records
CPU Utilization
CPU utilization increase as the number of instances increases from 100 to 500k. CPU reaches maximum for 100k, 500k instances but there is not much difference for 1 user and 2 users.
...