Overview
Bulk Edit - PERF-325 - Getting issue details... STATUS Establish a performance baseline for holdings bulk updates.
- How long does it take to export 100, 1000, and 10k records?
- Use it for up to 5 concurrent users.
- Look for a memory trend and CPU usage
Infrastructure
PTF -environment
- 9 m6i.2xlarge EC2 instances located in US West (Oregon)us-west-2 AWS region (comparing to 10 m6i.2xlarge EC2 instances located in US East (N. Virginia)us-east-1 for Morning Glory release)
- 2 instances of db.r6.xlarge database instances, one reader, and one writer
- MSK ptf-kakfka-1
- 4 m5.2xlarge brokers in 2 zones
Apache Kafka version 2.8.0
EBS storage volume per broker 300 GiB
- auto.create.topics.enable=truec
- log.retention.minutes=480
- default.replication.factor=3
Modules memory and CPU parameters
| # | modules | task definition | running tasks | CPU | memory | memoryReservation | maxMetaspaceSize | Xmx |
|---|---|---|---|---|---|---|---|---|
| mod-data-export-spring | 2 | 1 | 256 | 2048 | 1844 | 512 | 1536 (1236 in MG) | |
| mod-data-export-worker | 3 | 2 | 1024 | 3072 | 2600 | 612 | 2128 (1536 in MG) | |
| mod-users | 4 | 2 | 128 (258 in MG) | 1024 | 896 | 128 | 768 | |
| mod-notes | 2 | 2 | 128 | 1024 | 896 | 128m | 322 (768m in MG) | |
| mod-inventory | 2 | 2 | 1024 | 2880 | 2592 | 512m | 1814m | |
mod-inventory-storage | 5 | 2 | 1024 | 2208 | 1952 | 512m | 1440m | |
| okapi | 2 | 3 | 1024 | 1684 | 1440 | 512m | 922m | |
| mod-orders | 3 | 2 | 1024 | 2048 | 1440 | 512m | 896m | |
| mod-orders-storage | 2 | 2 | 128 | 1024 | 896 | 128m | 768m | |
| mod-circulation | 3 | 2 | 1024 | 1024 | 896 | 128m | 768m | |
| mod-circulation-storage | 6 | 2 | 1024 | 1536 | 1440 | 512m | 896m | |
| mod-agreements | 2 | 2 | 128 | 1592 | 1488 | 512m | 968m |
MG- Morning Glory release
Software Versions
- mod-data-export-worker-2.1.0-SNAPSHOT.85
- mod-data-export-spring-1.5.0-SNAPSHOT.58
- mod-agreements-5.4.0-SNAPSHOT.104
- mod-notes-4.0.0-SNAPSHOT.237
- mod-users-19.0.0-SNAPSHOT.573
- mod-inventory-19.0.0-SNAPSHOT.383
- mod-inventory-storage-25.0.0-SNAPSHOT.631
- okapi-4.14.4
Summary
Test report for Bulk Edits holdings-app functionality 2022-11-10.
- 10k records per user, 5 users simultaneously (50k records total) can be uploaded and updated in about 12 min 30 seconds total. 100k records of 1 user can be edited in about 2 hours 17 min
- The files with identifiers should be strictly determined.
- The memory usage of mod-data-export-worker increased from 40% to 76% at the beginning of the tests and slightly grows further PERF-328 - Getting issue details... STATUS . All other modules' memory usage was stable.
- CPU - for 5 parallel jobs - for mod-data-export-worker are spiking up to 155% at the start of tests( in general usage was about 8%). For all other modules did not exceed 40%. For 100k records jobs, CPU utilization increases only when additional jobs start.
- For all tests (100, 1k, and 10k -5 concurrent jobs, 100k records - 1job) - RDS CPU utilization did not exceed 50%.
Results
Test Runs
Items App - updating item status
Record identifier files location - http://carrier-io.int.folio.ebsco.com/artifacts?q=export
| 1 concurrent job | Nolana | ||
"BARCODE". Records number per 1 user | Time to upload | Time to edit - commit changes | Total time |
| 100 | 45 sec | up to 5 sec | 50 sec |
| 1000 | 1 min 40 sec | 30 sec | 2 min 10 sec |
| 10k | 6 min 10 sec | 6min 15 sec | 12 min 25 sec |
| 100k | 1 hour 2 min | 1 hour 15 min | 2 hours 17 min |
Tests for 5 concurrent jobs
| 1 concurrent job | Nolana | ||
"BARCODE". Records number per 1 user | Time to upload | Time to edit - commit changes | Total time |
| 100 | 46 sec | 3 sec | 49 sec |
| 1000 | 1 min 45 sec | 40 sec | 2 min 25 sec |
| 10k | 6 min 10 sec | 6min 30 sec | 12 min 25 sec |
Memory usage 5 concurrent jobs
mod-data-export-worker memory usage increased from 40% to 76% at the beginning of the tests and slightly grows further PERF-328 - Getting issue details... STATUS . All other modules' memory usage was stable.
Memory usage 100k records job
CPU utilization 5 concurrent jobs
CPU - Spike in CPU usage of mod-data-export-worker to 155% at the beginning of the tests. For all other modules did not exceed 40%.
CPU utilization 100k records job
RDS CPU utilization
For all jobs with records number 100, 1k, and 10k, of 5 concurrent jobs, 100k-1 concurrent job - RDS CPU utilization did not exceed 50%.
