Versions Compared

Key

  • This line was added.
  • This line was removed.
  • Formatting was changed.

IN PROGRESS

...

Per

Jira Legacy
serverSystem JiraJIRA
serverId01505d01-b853-3c2e-90f1-ee9b165564fc
keyPERF-293
the following tests were performed using the data export workflow (bulk edits, and edifacts) to determine what happens to the export workflows when mod-data-export-worker (with two nodes) fails and to what load can the exports run successfully. 

Infrastructure

PTF -environment

  • 10 m6i.2xlarge EC2 instances
  • 2 instances of db.r6.xlarge database instances, one reader and one writer
  • MSK
    • 4 m5.2xlarge brokers in 2 zones
    • auto.create-topics.enable = true
    • log.retention.minutes=120
    • 2 partitions per DI topics

#

modules

task definition

running tasks 

CPU

memory

memoryReservation

maxMetaspaceSize

Xmx

1

mod-inventory-storage

102102422081952512m1440m
2okapi33102416841440512m922m
3mod-invoice-storage32102418721536512m1024m
4mod-orders52102420481440512m896m
5mod-agreements5212815921488512m968m
6

mod-data-export-worker

22210243072 2800612m 2128m 
7

mod-data-export-spring

6125620481844512m1292m
8mod-notes321281024896128m768m
9mod-orders-storage421281024896128m768m
10mod-inventory42102428802592512m1814m

...

  • mod-data-export-worker: 1.4.10
  • mod-data-export-spring: 1.4.5
  • mod-agreements: 5.2.2
  • mod-notes: 3.1.2
  • mod-users: 18.3.0
  • mod-inventory-storage-24.1.0
  • mod-inventory-18.2.2
  • mod-orders-12.4.3
  • mod-orders-storage-13.3.3
  • okapi-4.14.2

Test Runs (initial assumption of the data sets for combo testing)

...

Test Run

Edifact

Bulk-edit

user barcodes

Bulk edit

item barcodes

Memory Behavior (Trend)CommentTime to process Results

#1

5 jobs (1000 orders per organisation)  




5 jobs passed successfully ended

#2

-5 jobs-5k records + 5 jobs -10k records5 jobs-5k records + 5 jobs -10k recordsmemory usage increases for mod-data-export-worker at the beginning of the test but becomes stable at 96%. Stable for all other modules.20 jobs total40 minutes for all

18 pass 2 failed

Jira Legacy
serverSystem JiraJIRA
serverId01505d01-b853-3c2e-90f1-ee9b165564fc
keyMODEXPW-264

#3

5 jobs (1000 orders per organisation)  




5 jobs passed successfully ended


Test results

Test Run

EdifactTime to process Results

Bulk-edit

user barcodes

Time to process Results

Bulk edit

item barcodes

Time to process ResultsComment
#15 jobs (1000 orders per organisation)  


















#2-

5 jobs -5k records 6-7 min4 Successful -1 failed5 jobs -5k records20 min *Successful

Jira Legacy
serverSystem JiraJIRA
serverId01505d01-b853-3c2e-90f1-ee9b165564fc
keyMODEXPW-264

-

5 jobs -10k records11-12 min4 Successful -1 failed5 jobs -10k records33 min *Successful

Jira Legacy
serverSystem JiraJIRA
serverId01505d01-b853-3c2e-90f1-ee9b165564fc
keyMODEXPW-264

#3



















...

Memory usage of concurrent Bulk Edit Items App Bulk Edit Users App -20 jobs total

10 parallel bulk edits for users for 5 of 10k and 5 of 5 k records can be performed simultaneously with 10 parallel bulk edits for items - 5 of 10k records and 5 of 5k records.

...

For 2 jobs functional issues found

Jira Legacy
serverSystem JiraJIRA
serverId01505d01-b853-3c2e-90f1-ee9b165564fc
keyMODEXPW-264

  • Memory trend: memory usage increases for mod-data-export-worker at the beginning of the test but becomes stable at 96%.

Image RemovedImage Added


CPU usage

Image RemovedImage Added

Image Removed


Image Added

Test #1


pic#1 Memory usage mod-data-export-worker/spring

...