Overview

Bulk Edit -

...

Infrastructure

PTF -environment

• 9 m6i.2xlarge EC2 instances located in US West (Oregon)us-west-2 AWS region (comparing to 10 m6i.2xlarge EC2 instances located in US East (N. Virginia)us-east-1 for Morning Glory release)
• 2 instances of db.r6.xlarge database instances, one reader, and one writer
• MSK ptf-kakfka-1
  • 4 m5.2xlarge brokers in 2 zones

  • Apache Kafka version 2.8.0

  • EBS storage volume per broker 300 GiB

  • auto.create.topics.enable=truec
  • log.retention.minutes=480
  • default.replication.factor=3

...

• mod-data-export-worker-2.0.1
• mod-data-export-spring-1.5.0-SNAPSHOT.58
• mod-agreements-5.4.0-SNAPSHOT.104
• mod-notes-4.0.0-SNAPSHOT.237
• mod-users-19.0.0-SNAPSHOT.573
• mod-inventory-19.0.0-SNAPSHOT.383
• mod-inventory-storage-25.0.0-SNAPSHOT.631
• okapi-4.14.4

Summary 

Test report for Bulk Edits holdings-app functionality 2022-11-10. 

• 10k records per user, 5 users simultaneously (50k records total) can be uploaded and updated in about 12 min 30 seconds total. 100k records of 1 user can be edited in about 2 hours 17 min
• The files with identifiers should be strictly determinedcorrectly specified.
• The memory usage of mod-data-export-worker increased from 40% to 76%  at the beginning of the tests and slightly grows further PERF-359. All other modules' memory usage was stable.
• CPU - for 5 parallel jobs - for mod-data-export-worker are spiking up to 155% at the start of tests( in general usage was about 8%). For all other modules did not exceed 40%.  For 100k records jobs, CPU utilization increases only when additional jobs start.
• For all tests (100, 1k, and 10k -5 concurrent jobs, 100k records - 1job) - RDS CPU utilization did not exceed 50%.

...

mod-data-export-worker memory usage increased from 40% to 76%  at the beginning of the tests and slightly grows further

...