IN PROGRESS

Overview

Per PERF-293 - Getting issue details... STATUS the following tests were performed using the data export workflow (bulk edits, and edifacts) to determine what happens to the export workflows when mod-data-export-worker (with two nodes) fails and to what load can the exports run successfully.

Infrastructure

PTF -environment

10 m6i.2xlarge EC2 instances
2 instances of db.r6.xlarge database instances, one reader and one writer
MSK
- 4 m5.2xlarge brokers in 2 zones
- auto.create-topics.enable = true
- log.retention.minutes=120

#	modules	task definition	running tasks	CPU	memory	memoryReservation	maxMetaspaceSize	Xmx
1	mod-inventory-storage	10	2	1024	2208	1952	512m	1440m
2	okapi	3	3	1024	1684	1440	512m	922m
3	mod-invoice-storage	3	2	1024	1872	1536	512m	1024m
4	mod-orders	5	2	1024	2048	1440	512m	896m
5	mod-agreements	5	2	128	1592	1488	512m	968m
6	mod-data-export-worker	22	2	1024	3072	2800	612m	2128m
7	mod-data-export-spring	6	1	256	2048	1844	512m	1292m
8	mod-notes	3	2	128	1024	896	128m	768m
9	mod-orders-storage	4	2	128	1024	896	128m	768m
10	mod-inventory	4	2	1024	2880	2592	512m	1814m

Software Versions

mod-data-export-worker: 1.4.10
mod-data-export-spring: 1.4.5
mod-agreements: 5.2.2
mod-notes: 3.1.2
mod-users: 18.3.0
mod-inventory-storage-24.1.0
mod-inventory-18.2.2
mod-orders-12.4.3
mod-orders-storage-13.3.3
okapi-4.14.2

Test Runs (initial assumption of the data sets for combo testing)

Bulk edit files are located at https://github.com/folio-org/perf-testing/tree/Bulk-edit/workflows-scripts/Bulk-edit

Test Run	Edifact	Bulk-edit user barcodes	Bulk edit item barcodes	Memory Behavior (Trend)	Comment	Time to process	Results
#1	5 jobs (1000 orders per organisation)						5 jobs passed successfully ended
#2	-	5 jobs-5k records + 5 jobs -10k records	5 jobs-5k records + 5 jobs -10k records	memory usage increases for mod-data-export-worker at the beginning of the test but becomes stable at 96%. Stable for all other modules.	20 jobs total	40 minutes for all	18 pass 2 failed MODEXPW-264 - Getting issue details... STATUS
#3	5 jobs (1000 orders per organisation)						5 jobs passed successfully ended

Test results

Test Run	Edifact	Time to process	Results	Bulk-edit user barcodes	Time to process	Results	Bulk edit item barcodes	Time to process	Results	Comment
#1	5 jobs (1000 orders per organisation)
#1
#2	-			5 jobs -5k records	6-7 min	4 Successful -1 failed	5 jobs -5k records	20 min *	Successful	MODEXPW-264 - Getting issue details... STATUS
#2	-			5 jobs -10k records	11-12 min	4 Successful -1 failed	5 jobs -10k records	33 min *	Successful	MODEXPW-264 - Getting issue details... STATUS
#3
#3

* - average job processing time for the number of records specified ( upload identifiers + editing)

Failover testing. The behavior of the workflows when 'killing' the module

mod-data-export-worker
mod-data-export-spring

Edifact

Bulk-edit

user barcodes

Bulk edit

item barcodes

mod-data-export-worker

mod-data-export-spring

Memory Trends of Repeated concurrent Edifact jobs:

Concurrent Edifact exports (10 jobs)

Memory usage of concurrent Bulk Edit Items App + Bulk Edit Users App -20 jobs total

10 parallel bulk edits for users for 5 of 10k and 5 of 5 k records can be performed simultaneously with 10 parallel bulk edits for items - 5 of 10k records and 5 of 5k records.

18 out of 20 -successful

For 2 jobs functional issues found MODEXPW-264 - Getting issue details... STATUS