Overview

The target of the test is to investigate the capacity of the Check-In/Check-Out activities.

https://folio-org.atlassian.net/browse/PERF-459

Summary

• Test results (Test #1 & Test #2) show approx. the same saturation point - ~134 vUsers

• The start point of the Performance degradation zone (Test #1 & Test#2):

  • check-in: ~110 vUsers & ~98 vUsers

  • check-out: ~75 vUsers & ~77 vUsers

Recommendations

• Up to ~75 vUsers could be used for benchmark testing if it's necessary and based on business non-functional requirements

• Run fixed load tests for the confidence of the Check-In/Check-Out activities under the 'comfort' load

Test Runs 

Results

Response Times 

Grafana: 

Test #1 http://carrier-io.int.folio.ebsco.com/grafana/d/elIt9zCnz/jmeter-performance-test-copy?orgId=1&from=1677754860000&to=1677765659000&var-percentile=95&var-test_type=baseline&var-test=circulation_checkInCheckOut_orchid&var-env=int&var-grouping=1s&var-low_limit=250&var-high_limit=750&var-db_name=jmeter&var-sampler_type=All&var-Request=Check-Out%20Controller

~134 vUsers peak - Capacity point (~683 op), throughput growth has been decreased and response issues appeared (5** status code of 'okapi' due to unavailability of related modules) (see Grafana snapshot Test #1 - Throughput)

Open

Based on Investigation of variability of test results:

• AVG/Median check-in: [400;500] ms

• AVG/Median check-out: [700;800] ms

~110 vUsers check-in - potential start point of performance degradation zone, the response time has grown up, a significant amount of requests has a response time of more than 500 ms (see Grafana snapshot Test #1 - Response time heatmap (option 'Check-in'))

~75 vUsers check-out - potential start point of performance degradation zone, the response time has grown up, a significant amount of requests has a response time of more than 800 ms (see Grafana snapshot Test #1 - Response time heatmap (option 'Check-out'))

Test #2 http://carrier-io.int.folio.ebsco.com/grafana/d/elIt9zCnz/jmeter-performance-test-copy?orgId=1&from=1677824973753&to=1677834380143&var-percentile=95&var-test_type=baseline&var-test=circulation_checkInCheckOut_orchid&var-env=int&var-grouping=1m&var-low_limit=250&var-high_limit=750&var-db_name=jmeter&var-sampler_type=All&var-Request=Check-In%20Controller

~134 vUsers peak Capacity point (~680 op), throughput growth has been decreased and response issues appeared (5** status code of 'okapi' due to unavailability of related modules) (see Grafana snapshot Test #2 - Throughput)

Open

Based on Investigation of variability of test results:

• AVG/Median check-in: [400;500] ms

• AVG/Median check-out: [700;800] ms

~98 vUsers check-in - potential start point of performance degradation zone, the response time has grown up, a significant amount of requests has a response time of more than 500 ms (see Grafana snapshot Test #2 - Response time heatmap (option 'Check-in'))

~62 vUsers check-out - potential start point of performance degradation zone, the response time has grown up, a significant amount of requests has a response time of more than 800 ms (see Grafana snapshot Test #2 - Response time heatmap (option 'Check-out'))

Memory Utilization

Test №1

Open

mod-circulation was a little grown up 58.4% > 67.5%

Test №2

Open

CPU Utilization 

Test №1

Open

nginx-okapi: 0.1% >1'012%

mod-users-b: 0.5% > 254.6%

Test №2

Open

nginx-okapi: 0.1% >853.7%

mod-users-b: 0.6% > 308%

RDS CPU Utilization 

Test №1

Open

DB Writer: up to 81.6%

Open

Test №2

DB Writer: up to 79.12%

Open

Appendix

Infrastructure

PTF -environment ncp5 [ environment name] 

• 8 m6i.2xlarge EC2 instances located in US East (N. Virginia)us-east-1 [Number of ECS instances, instance type, location region]

• 2 instances of db.r6.xlarge database instances: Writer & reader instances

• MSK ptf-kakfa-3 [ kafka configurations]

  • 4 kafka.m5.2xlarge brokers in 2 zones

  • Apache Kafka version 2.8.0

  • EBS storage volume per broker 300 GiB

  • auto.create.topics.enable=true

  • log.retention.minutes=480

  • default.replication.factor=3