...
Overview
Bulk Edit Edits - Establish a performance baseline for Items bulk updates PERF-406 in the Orchid release that has architectural changes that were implemented in UXPROD-3842. The goal is to make sure the performance did not deteriorate in comparison to Nolana release. Some questions can help us to determine the performance and stability of the new Bulk Edits implemenation:
- How long does it take to export 100, 1000, 10k, and 100K records?
- Use Can it for be used with up to 5 concurrent users. ?
- Run consecutively four jobs editing 10k item records
- Run simultaneously four jobs editing 10k item records
- Look for a memory trend and CPU usage
...
Test report for Bulk Edits items-app functionality 2023-03-08.
- For 1 concurrent job100 job 100 records can be edited in 1 min 9 s which is 3 times slower than in Nolana, 1000 records editing could be performed in approximately the same time as Nolana(2 min 40 s), and 10k records bulk editing is about 30% faster.
- 10k records per user, 5 users simultaneously (50k records total) can be uploaded and edited in about 20 minutes which is about 8 min faster compared to Nolana (about 28 min).
- The memory of mod-inventory-storage was high at 109% but stable. No memory leaks were found.
- CPU for mod-users was up to 125% (5 concurrent jobs 10k records updating) increased compared to Nolana (was about 40% with the same configurations). For all other modules did not exceed 65% for all of the tests.
- For all records number (100, 1k,10k), and 5 concurrent jobs - RDS CPU utilization did not exceed 60%.
...
More than 50% of jobs FAILED in about 28-33 min.
| Jira Legacy |
|---|
| server | System JiraFOLIO Issue Tracker |
|---|
| serverId | 01505d016ccf3fe4-b8533301-3c2e368a-90f1983e-ee9b165564fc20c466b11a49 |
|---|
| key | PERF-334 |
|---|
|
From time to time Job FAILED with the error from the s3 bucket
| Jira Legacy |
|---|
| server | System JiraFOLIO Issue Tracker |
|---|
| serverId | 01505d016ccf3fe4-b8533301-3c2e368a-90f1983e-ee9b165564fc20c466b11a49 |
|---|
| key | MODBULKOPS-76 |
|---|
|
Results
Test Runs
1#
...
One (concurrent) Job
| Number of records | Duration | Comments |
|---|
| 100 | 1 min 9 s |
|
| 1000 | 2 min 36 s |
|
| 10k | 17 min 50 s |
|
| 50k | 1 hour 58 min | or Error in about 28-33 min Connection reset (SocketException) |
System Jira| FOLIO Issue Tracker | | serverId |
|---|
|
|
01505d01b8533c2e90f1ee9b165564fc |
| 100k | always FAILD | Error in about 28-33 min Connection reset (SocketException) |
System Jira| FOLIO Issue Tracker | | serverId |
|---|
|
|
01505d01b8533c2e90f1ee9b165564fc2#
2# Items App 10k records 3, 4, and 5 concurrent jobs
| 10k records for each job |
| Number of concurrent jobs | Duration |
|---|
| 1 | 17 min 50 s |
| 3 | 18 min 50 s |
| 4 | 19 min 10 s |
| 5 | 20 min 20 s |
3#
...
Editing four jobs
...
consecutively with 10k item records each
| Job # | Job duration (run 2) | Job duration (run 1) |
|---|
| 1 | 17 min 47 s | 18 min 49 s |
| 2 | 17 min 53 s | 18 min 26 s |
| 3 | 17 min 45 s | 20 min 44 s |
| 4 | 18 min 5 s | ERROR occurs: We encountered an internal error. Please try again. (Service: S3, Status Code: 500, Request ID: 5W7F75FMHHH3KDWT, Extended Request ID: 36K8tkhFQHS1Mjt7sZc4jYrBduBWO/psei+33ZIIOnhrytq7Eie3mjDALtBplhZxSJv4CfrZpnw8Z6nqmz03ZB7b3yiRdecyXfZ/ZtEmN4g=) (S3Excepti |
4#
...
Items App 5 concurrent jobs
...
5 Concurrent Item Apps jobs
| "BARCODE". Records number per 1 user | Orchid (Total |
time time| Time) | Morning Glory (Total |
|---|
time| Time) |
|---|
| 100 | 1 min 10 s | 18 sec | 25-27 sec |
|---|
| 1000 | 2 min 57 s | 3 min | 4 min |
|---|
| 10k | 20 min 20 s | 28 min | 30 min |
|---|
| 25k | Results are not representative because of
Error in about 28-33 min Connection reset (SocketException) |
System Jira| FOLIO Issue Tracker | | serverId |
|---|
|
|
01505d01b8533c2e90f1ee9b165564fc | 1 hour 3 min | 50 min |
|---|
| 50k | Results are not representative because of
Error in about 28-33 min Connection reset (SocketException) |
System Jira| FOLIO Issue Tracker | | serverId |
|---|
|
|
01505d01b8533c2e90f1ee9b165564fc | about 2 hours for successful jobs. | - |
|---|
* "-" test was not performed due to errors that occurred
...
Modules memory and CPU parameters:
Module | SoftLimit | XMX | Revision | Version | desiredCount | CPUUnits | RWSplitEnabled | HardLimit | Metaspace | MaxMetaspaceSize |
|---|
| mod-inventory-storage-b | 1952 | 1440 | 3 | mod-inventory-storage:26.1.0-SNAPSHOT.644 | 2 | 1024 | False | 2208 | 384 | 512 |
| mod-inventory-b | 2592 | 1814 | 7 | mod-inventory:20.0.0-SNAPSHOT.392 | 2 | 1024 | False | 2880 | 384 | 512 |
| okapi-b | 1440 | 922 | 1 | okapi:5.1.0-SNAPSHOT.1352 | 3 | 1024 | False | 1684 | 384 | 512 |
| mod-users-b | 896 | 768 | 4 | mod-users:19.2.0-SNAPSHOT.584 | 2 | 128 | False | 1024 | 88 | 128 |
| mod-data-export-worker | 2600 | 2048 | 3 | mod-data-export-worker:3.0.0-SNAPSHOT.104 | 2 | 1024 | False | 3072 | 384 | 512 |
| mod-data-export-spring | 1844 | 1292 | 3 | mod-data-export-spring:2.0.0-SNAPSHOT.67 | 1 | 256 | False | 2048 | 200 | 256 |
| mod-bulk-operations | 3864 | 0 | 8 | mod-bulk-operations:1.0.1 | 2 | 400 | False | 4096 | 384 | 512 |
| mod-notes | 896 | 322 | 3 | mod-notes:5.1.0-SNAPSHOT.245 | 2 | 128 | False | 1024 | 128 | 128 |
| mod-agreements | 2580 | 2048 | 3 | mod-agreements:5.6.0-SNAPSHOT.117 | 2 | 128 | False | 3096 | 384 | 512 |
| nginx-okapi | 896 | 0 | 3 | nginx-okapi:2022.03.02 | 2 | 128 | False | 1024 | 0 | 0 |