Bulk Edit Users App report [Morning Glory] 26/09/2022
IN PROGRESS
Overview
Bulk Edit - Establish a performance baseline for user status bulk updates.
- How long does it take to export 100, 1000, 2500, 10k, and 100K records?
- Use it for up to 5 concurrent users.
- Look for a memory trend and CPU usage
Infrastructure
PTF -environment
- 10 m6i.2xlarge EC2 instances
- 2 instances of db.r6.xlarge database instances, one reader and one writer
- MSK
- 4 m5.2xlarge brokers in 2 zones
- auto.create-topics.enable = true
- log.retention.minutes=120
- 2 partitions per DI topics
- okapi (running tasks -3)
- 1024 CPU units, 1360 MB mem
- mod-data-export-spring (running tasks -1)
- 256 CPU units, 1844 MB mem
- mod-data-export-worker (running tasks -1)
- 1024 CPU units, 2248 MB mem
- mod-notes (running tasks -2)
- 128 CPU units, 896 MB mem
- mod-agreements (running tasks -2)
- 128 CPU units, 1382 MB mem
- mod-users (running tasks -2)
- 258 CPU units, 896 MB mem
Differences in modules memory and CPU parameters
| Module | Morning Glory (14/09/2022) | Morning Glory (23/09/2022) | Morning Glory (26/09/2022) | |||
| CPU | RAM | CPU | RAM | CPU | RAM | |
|---|---|---|---|---|---|---|
| mod-data-export-worker | 256 | 1844MB | 1024 | 2248MB | 1024 | 2600MB |
Software Versions
- mod-data-export-worker v 1.4.10
- mod-data-export-spring v 1.4.4
- mod-agreements:5.2.2
- mod-notes: 3.1.2
- mod-users: 18.3.0
- mod-inventory-18.2.2
- mod-inventory-storage-24.1.0
Summary
Test report for Bulk Edits users-app functionality 2022-09-29.
- 10k records per user, 5 users simultaneously can be uploaded in about 3 min, edited files uploaded in up to 10 minutes (depending on the number of records that should not be updated: 2 min if all records will be updated), and edited in 4 min (about 10 min total).
- The files with identifiers should be strictly determined.
- The memory of mod-users increases during the tests for 2500 records 5 parallel jobs by 1%(from 42% to 43%) and during the tests for 5000 (5 parallel) and 10 000 records (5 parallel) jobs by 2% (from 43% to 45%). For mod-data-export-worker memory was about 97% all the time. All other modules' memory usage was stable.
- CPU - 5000 & 10k records per user 5 parallel jobs -CPU of modules did not exceed 149%.
- RDS CPU utilization did not exceed 52%
Results
Test Runs
Users App - updating user status
Record identifier files location - http://carrier-io.int.folio.ebsco.com/artifacts?q=export
| 1 virtual user | "BARCODE" | ||
Records number per 1 user | Time to upload | Time of POST /bulk-edit/${jobId}/items-content-update/upload | Time to edit |
| 100 | 2 sec | 2 sec | 2 sec |
| 1000 | 12 sec | 8 sec | 18 sec |
| 2500 | 31 sec | 4.7 min MODEXPW-255 - Getting issue details... STATUS | 21 sec |
| 10k | 40 min if 0 records matched related to MODEXPW-255 - Getting issue details... STATUS | - | - |
* "-'' test was not performed due to 0 records can be updated
| 5 virtual users | "BARCODE" | ||
Records number per 1 user | Time to upload | Time of POST /bulk-edit/${jobId}/upload | Time to edit - commit changes |
| 2500 | 34-39 sec | up to 10 minutes (depending on the number of records that should not be updated) | 1 min 5 sec |
| 5000 | 1 min 12 sec | 35 sec if all records will be updated MODEXPW-255 - Getting issue details... STATUS | 1min 50 sec |
| 10k | 2min 15 sec | 2 min if all records will be updated | 3 min 58 sec |
Memory usage
The memory of mod-users increases during the tests for 2500 records 5 parallel jobs by 1%(from 42% to 43%) and during the tests for 5000 (5 parallel) and 10 000 records (5 parallel) jobs by 2%(from 43% to 45%). For mod-data-export-worker memory was about 97% all the time.
CPU utilization
2500 records per user 5 parallel jobs:
| modules | max CPU utilization |
|---|---|
| mod-users | 116% |
| mod-data-export-worker | 19% |
| nginx-okapi | 13% |
| okapi | 11% |
| mod-data-export-spring | 7% |
| mod-inventory | 6% |
| mod-agreements | 4% |
| mod-notes | 3% |
| mod-inventory-storage | 0.5% |
5000 & 10k records 5 parallel jobs -CPU of modules did not exceed 149%
RDS CPU utilization
2500 records - RDS CPU utilization did not exceed 42%
5000 & 10k records - RDS CPU utilization did not exceed 52%
Jiras issues related to the test:
MODEXPW-244
-
Getting issue details...
STATUS
- fixed
MODEXPW-150
-
Getting issue details...
STATUS
- fixed
MODEXPW-255 - Getting issue details... STATUS
MODBULKED-32 - Getting issue details... STATUS
PERF-299 - Getting issue details... STATUS