Overview

Bulk Edit - PERF-325 - Getting issue details... STATUS Establish a performance baseline for holdings bulk updates.

How long does it take to export 100, 1000, and 10k records?
Use it for up to 5 concurrent users.
Look for a memory trend and CPU usage

Infrastructure

PTF -environment

9 m6i.2xlarge EC2 instances located in US West (Oregon)us-west-2 AWS region (comparing to 10 m6i.2xlarge EC2 instances located in US East (N. Virginia)us-east-1 for Morning Glory release)
2 instances of db.r6.xlarge database instances, one reader, and one writer
MSK ptf-kakfka-1
- 4 m5.2xlarge brokers in 2 zones
- Apache Kafka version 2.8.0
- EBS storage volume per broker 300 GiB
- auto.create.topics.enable=truec
- log.retention.minutes=480
- default.replication.factor=3

Modules memory and CPU parameters

modules	task definition	running tasks	CPU	memory	memoryReservation	maxMetaspaceSize	Xmx
mod-data-export-spring	2	1	256	2048	1844	512	1536 (1236 in MG)
mod-data-export-worker	3	2	1024	3072	2600	612	2128 (1536 in MG)
mod-users	4	2	128 (258 in MG)	1024	896	128	768
mod-notes	2	2	128	1024	896	128m	322 (768m in MG)
mod-inventory	2	2	1024	2880	2592	512m	1814m
mod-inventory-storage	5	2	1024	2208	1952	512m	1440m
okapi	2	3	1024	1684	1440	512m	922m
mod-orders	3	2	1024	2048	1440	512m	896m
mod-orders-storage	2	2	128	1024	896	128m	768m
mod-circulation	3	2	1024	1024	896	128m	768m
mod-circulation-storage	6	2	1024	1536	1440	512m	896m
mod-agreements	2	2	128	1592	1488	512m	968m

MG- Morning Glory release

Software Versions

mod-data-export-worker-2.0.1
mod-data-export-spring-1.5.0-SNAPSHOT.58
mod-agreements-5.4.0-SNAPSHOT.104
mod-notes-4.0.0-SNAPSHOT.237
mod-users-19.0.0-SNAPSHOT.573
mod-inventory-19.0.0-SNAPSHOT.383
mod-inventory-storage-25.0.0-SNAPSHOT.631
okapi-4.14.4

Summary

Test report for Bulk Edits holdings-app functionality 2022-11-10.

10k records per user, 5 users simultaneously (50k records total) can be uploaded and updated in about 12 min 30 seconds total. 100k records of 1 user can be edited in about 2 hours 17 min
The files with identifiers should be correctly specified.
The memory usage of mod-data-export-worker increased from 40% to 76% at the beginning of the tests and slightly grows further PERF-359. All other modules' memory usage was stable.
CPU - for 5 parallel jobs - for mod-data-export-worker are spiking up to 155% at the start of tests( in general usage was about 8%). For all other modules did not exceed 40%. For 100k records jobs, CPU utilization increases only when additional jobs start.
For all tests (100, 1k, and 10k -5 concurrent jobs, 100k records - 1job) - RDS CPU utilization did not exceed 50%.

Results

Test Runs

Items App - updating item status

Record identifier files location - http://carrier-io.int.folio.ebsco.com/artifacts?q=export

1 concurrent job	Nolana
"BARCODE". Records number per 1 user	Time to upload	Time to edit - commit changes	Total time
100	45 sec	up to 5 sec	50 sec
1000	1 min 40 sec	30 sec	2 min 10 sec
10k	6 min 10 sec	6min 15 sec	12 min 25 sec
100k	1 hour 2 min	1 hour 15 min	2 hours 17 min

Tests for 5 concurrent jobs

1 concurrent job	Nolana
"BARCODE". Records number per 1 user	Time to upload	Time to edit - commit changes	Total time
100	46 sec	3 sec	49 sec
1000	1 min 45 sec	40 sec	2 min 25 sec
10k	6 min 10 sec	6min 30 sec	12 min 25 sec

Memory usage 5 concurrent jobs

mod-data-export-worker memory usage increased from 40% to 76% at the beginning of the tests and slightly grows further PERF-328 - Getting issue details... STATUS . All other modules' memory usage was stable.