Overview

Bulk Edit - Establish a performance baseline for user status bulk updates.

How long does it take to export 100, 1000, 2500, 10k, and 100K records?
Use it for up to 5 concurrent users.
Look for a memory trend and CPU usage

Infrastructure

PTF -environment

10 m6i.2xlarge EC2 instances
2 instances of db.r6.xlarge database instances, one reader and one writer
MSK
- 4 m5.2xlarge brokers in 2 zones
- auto.create-topics.enable = true
- log.retention.minutes=120
- 2 partitions per DI topics
okapi (running tasks -3)
- 1024 CPU units, 1360 MB mem
mod-data-export-spring (running tasks -1)
- 256 CPU units, 1844 MB mem
mod-data-export-worker (running tasks -1)
- 1024 CPU units, 2248 MB mem
mod-notes (running tasks -2)
- 128 CPU units, 896 MB mem
mod-agreements (running tasks -2)
- 128 CPU units, 1382 MB mem
mod-users (running tasks -2)
- 258 CPU units, 896 MB mem

Differences in modules memory and CPU parameters

Module	CPU	RAM	CPU	RAM	CPU	RAM
Module	Morning Glory (14/09/2022)		Morning Glory (23/09/2022)		Morning Glory (26/09/2022)
mod-data-export-worker	256	1844MB	1024	2248MB	1024	2600MB

Software Versions

mod-data-export-worker v 1.4.10
mod-data-export-spring v 1.4.4
mod-agreements:5.2.2
mod-notes: 3.1.2
mod-users: 18.3.0
mod-inventory-18.2.2
mod-inventory-storage-24.1.0

Summary

Test report for Bulk Edits users-app functionality 2022-09-29.

10k records per user, 5 users simultaneously can be uploaded in about 3 min, edited files uploaded in up to 10 minutes (depending on the number of records that should not be updated: 2 min if all records will be updated), and edited in 4 min (about 10 min total).
The files with identifiers should be strictly determined.
The memory of mod-users increases during the tests for 2500 records 5 parallel jobs by 1%(from 42% to 43%) and during the tests for 5000 (5 parallel) and 10 000 records (5 parallel) jobs by 2% (from 43% to 45%). For mod-data-export-worker memory was about 97% all the time. All other modules' memory usage was stable.
CPU - 5000 & 10k records per user 5 parallel jobs -CPU of modules did not exceed 149%.
RDS CPU utilization did not exceed 52%

Results

Test Runs

Users App - updating user status

Record identifier files location - http://carrier-io.int.folio.ebsco.com/artifacts?q=export

1 virtual user	"BARCODE"
Records number per 1 user	Time to upload	Time of POST /bulk-edit/${jobId}/items-content-update/upload	Time to edit
100	2 sec	2 sec	2 sec
1000	12 sec	8 sec	18 sec
2500	31 sec	4.7 min https://folio-org.atlassian.net/browse/MODEXPW-255	21 sec
10k	40 min if 0 records matched related to https://folio-org.atlassian.net/browse/MODEXPW-255	-	-

* "-'' test was not performed due to 0 records can be updated

5 virtual users	"BARCODE"
Records number per 1 user	Time to upload	Time of POST /bulk-edit/${jobId}/upload	Time to edit - commit changes
2500	34-39 sec	up to 10 minutes (depending on the number of records that should not be updated)	1 min 5 sec
5000	1 min 12 sec	35 sec if all records will be updatedhttps://folio-org.atlassian.net/browse/MODEXPW-255	1min 50 sec
10k	2min 15 sec	2 min if all records will be updated	3 min 58 sec

Memory usage

The memory of mod-users increases during the tests for 2500 records 5 parallel jobs by 1%(from 42% to 43%) and during the tests for 5000 (5 parallel) and 10 000 records (5 parallel) jobs by 2%(from 43% to 45%). For mod-data-export-worker memory was about 97% all the time.

CPU utilization

2500 records per user 5 parallel jobs:

modules	max CPU utilization

modules	max CPU utilization
mod-users	116%
mod-data-export-worker	19%
nginx-okapi	13%
okapi	11%
mod-data-export-spring	7%
mod-inventory	6%
mod-agreements	4%
mod-notes	3%
mod-inventory-storage	0.5%