/
Bulk Edit Holdings App report [Nolana] 10/11/2022

Bulk Edit Holdings App report [Nolana] 10/11/2022

Jan 04, 2023

 

 

Overview

Bulk Edit - https://folio-org.atlassian.net/browse/PERF-325 Establish a performance baseline for holdings bulk updates. 

  • How long does it take to export 100, 1000, and 10k records?

  • Use it for up to 5 concurrent users.  

  • Look for a memory trend and CPU usage

 

Infrastructure

PTF -environment

  • 9 m6i.2xlarge EC2 instances located in US West (Oregon)us-west-2 AWS region (comparing to 10 m6i.2xlarge EC2 instances located in US East (N. Virginia)us-east-1 for Morning Glory release)

  • 2 instances of db.r6.xlarge database instances, one reader, and one writer

  • MSK ptf-kakfka-1

    • 4 m5.2xlarge brokers in 2 zones

    • Apache Kafka version 2.8.0

    • EBS storage volume per broker 300 GiB

    • auto.create.topics.enable=truec

    • log.retention.minutes=480

    • default.replication.factor=3

 

Modules memory and CPU parameters

#

modules

task definition

running tasks 

CPU

memory

memoryReservation

maxMetaspaceSize

Xmx

#

modules

task definition

running tasks 

CPU

memory

memoryReservation

maxMetaspaceSize

Xmx

 

mod-data-export-spring

2

1

256

2048

1844

512

1536 (1236 in MG)

 

mod-data-export-worker

3

2

1024

3072

2600

612

2128 (1536 in MG)

 

mod-users

4

2

128 (258 in MG)

1024

896

128

768

 

mod-notes

2

2

128

1024

896

128m

322 (768m in MG)

 

mod-inventory

2

2

1024

2880

2592

512m

1814m

 

mod-inventory-storage

5

2

1024

2208

1952

512m

1440m

 

okapi

2

3

1024

1684

1440

512m

922m

 

mod-orders

3

2

1024

2048

1440

512m

896m

 

mod-orders-storage

2

2

128

1024

896

128m

768m

 

mod-circulation

3

2

1024

1024

896

128m

768m

 

mod-circulation-storage

6

2

1024

1536

1440

512m

896m

 

mod-agreements

2

2

128

1592

1488

512m

968m

MG- Morning Glory release

Software Versions

  • mod-data-export-worker-2.0.1

  • mod-data-export-spring-1.5.0-SNAPSHOT.58

  • mod-agreements-5.4.0-SNAPSHOT.104

  • mod-notes-4.0.0-SNAPSHOT.237

  • mod-users-19.0.0-SNAPSHOT.573

  • mod-inventory-19.0.0-SNAPSHOT.383

  • mod-inventory-storage-25.0.0-SNAPSHOT.631

  • okapi-4.14.4

Summary 

Test report for Bulk Edits holdings-app functionality 2022-11-10. 

  • 10k records per user, 5 users simultaneously (50k records total) can be uploaded and updated in about 12 min 30 seconds total. 100k records of 1 user can be edited in about 2 hours 17 min

  • The files with identifiers should be correctly specified.

  • The memory usage of mod-data-export-worker increased from 40% to 76%  at the beginning of the tests and slightly grows further PERF-359. All other modules' memory usage was stable.

  • CPU - for 5 parallel jobs - for mod-data-export-worker are spiking up to 155% at the start of tests( in general usage was about 8%). For all other modules did not exceed 40%.  For 100k records jobs, CPU utilization increases only when additional jobs start.

  • For all tests (100, 1k, and 10k -5 concurrent jobs, 100k records - 1job) - RDS CPU utilization did not exceed 50%.

Results

Test Runs

Items App - updating item status

Record identifier files location - http://carrier-io.int.folio.ebsco.com/artifacts?q=export

1 concurrent job

Nolana

"BARCODE". Records number per 1 user

Time to upload

Time to edit - commit changes

Total time

100

45 sec

up to 5 sec

50 sec

1000

1 min 40 sec

30 sec

2 min 10 sec

10k

6 min 10 sec

6min 15 sec

12 min 25 sec

100k

1 hour 2 min

1 hour 15 min

2 hours 17 min

Tests for 5 concurrent jobs

 

1 concurrent job

Nolana

"BARCODE". Records number per 1 user

Time to upload

Time to edit - commit changes

Total time

100

46 sec

3 sec

49 sec

1000

1 min 45 sec

40 sec

2 min 25 sec

10k

6 min 10 sec

6min 30 sec

12 min 25 sec

Memory usage 5 concurrent jobs

mod-data-export-worker memory usage increased from 40% to 76%  at the beginning of the tests and slightly grows furtherhttps://folio-org.atlassian.net/browse/PERF-328. All other modules' memory usage was stable.

 

Memory usage 100k records job

CPU utilization 5 concurrent jobs

      CPU - Spike in CPU usage of mod-data-export-worker to 155% at the beginning of the tests. For all other modules did not exceed 40%.

CPU utilization 100k records job

RDS CPU utilization

For all jobs with records number 100, 1k, and 10k, of 5 concurrent jobs, 100k-1 concurrent job - RDS CPU utilization did not exceed 50%.