Table of Contents

Overview

This document contains the results of testing Fiscal year rollover in scope of ticket

Summary

• Service CPU utilization for mod-orders-storage increased up to 60% during orders phase of 10K rollover, while during 50K and 100K orders rollover mod-orders-storage CPU utilization reaches only 20% maximum. More details...
• DB CPU utilization shows different behavior for rollovers of different data quantity. During 10K and 50k rollover DB CPU utilization increased during orders phase (31% → 83%, 61% → 85%), while during 100K rollover DB CPU utilization decreased during orders phase (65% → 30%, 67% → 40%). More details...
• Lock wait type was observed during orders phase of rollover. Lock waiting time is higher during 10K orders rollover than during 50K and 100K orders rollover. More details...
• "Out of shared memory" error occurred during 100K orders rollover with 90K open orders. This issue was reported previously in

  Jira Legacy
  server FOLIO Issue TrackerSystem Jira serverId 6ccf3fe401505d01-3301b853-368a3c2e-983e90f1-20c466b11a49ee9b165564fc key MODFISTO-428

  .
• "504 Gateway Time-out" error occurred during 100K orders rollover with 50K open orders. This issue was reported and marked as fixed previously (

  Jira Legacy
  server FOLIO Issue TrackerSystem Jira serverId 6ccf3fe401505d01-3301b853-368a3c2e-983e90f1-20c466b11a49ee9b165564fc key MODORDERS-865

  ) but should be investigated further.
• A set of improvements was prepared by dev team:

  Jira Legacy
  server FOLIO Issue TrackerSystem Jira serverId 6ccf3fe401505d01-3301b853-368a3c2e-983e90f1-20c466b11a49ee9b165564fc key MODFISTO-439

  . When these fixes are implemented, tests will be conducted again to assess performance and compare results with current ones.

...

Investigate application behavior during fiscal year rollover:

Investigate "Out of shared memory" and "504 Gateway Time-out"  errors in scope of previously created tickets.

...

Infrastructure

PTF -environment ncp5

• 8 9 m6i.2xlarge EC2 instances located in in US East (N. Virginia)us-east-1
• 2 instances of db.r6.xlarge database instances: Writer & reader instances
• MSK ptf-kakfa-3
  
  • 4 kafka.m5.2xlarge brokers in 2 zones

  • Apache Kafka version 2.8.0

  • EBS storage volume per broker 300 GiB

  • auto.create.topics.enable=true
  • log.retention.minutes=480
  • default.replication.factor=3

...

...

For other orders quantity and data structure JMeter script should be used (perf-testing/workflows-scripts/fiscal_year/FY_rollover_preparation on the GitHub). But in this case data generation will take a lot of time.

...