Overview
This document contains the results of testing List App refreshing of 200k records with multiple parallel workflows (Check-in/Check-out and Data import) and R/W split enabled in the Poppy release. The goal of testing is to assess the performance of mod-lists along with other workflows (CICO and DI) and R/W split enabled and to measure mod-lists's impact on other modules
...
| Jira Legacy | ||||||
|---|---|---|---|---|---|---|
|
Summary
- Load tests showed that average List App refresh duration in multiple-workflow test with R/W split enabled is 11.4 min.
- ListApp duration while testig separately is 3.9 minutes less (7.5 min).
- ListApp duration while testig with other workflows but with R/W split disabled is 6.3 minutes more (17.7 min).
- According to the results, maximum CPU utilization for DI+CICO test is about 126%, while with ListApp scenario added CPU reaches 150%.
- CPU utilization of mod-lists and mod-fqm-manager for ListApp+DI+CICO test is up to 140% with R/W split enabled, while it reaches only 105% with R/W split disabled.
- RDS CPU utilization (writer instance) is about 96% for DI+CICO test and it doesn't increase with ListApp scenario added. RDS CPU utilization for reader instance reaches 95% during ListApp+DI+CICO test.
- RDS CPU utilizatoin for ListApp+DI+CICO (writer instance) didn't change in comparion with the same test but with R/W split disabled for mod-fqm-manager.
- No memory leak is suspected for all the modules.
Test runs
Query used in lists - "Item status != Available". List refresh result is 200K records.
...
| Transaction | Duration | |||
|---|---|---|---|---|
| R/W split disabled* | R/W split enabled** | |||
Testing separately (ListApp and DI+CICO) | Testing in parallel | Testing separately (ListApp and DI+CICO) | Testing in parallel | |
| ListApp refresh, avg | 8.5 min | 17.7 min | 7.5 min (-1 min) | 11.4 min (-6.3 min) |
| Check-in, avg | 1.173 sec | 1.290 sec | 0.992 (-0.181 sec) | 1.087 (-0.203 sec) |
| Check-out, avg | 1.648 sec | 1.948 sec | 1.634 (-0.014 sec) | 1.765 (-0.183 sec) |
| DI Create 25k | 18 min | 22 min | 13 min (-5 min) | 15 min (-7 min) |
* Query used in lists - "Item status == Checked out". List refresh result is 200K records. Results are taken from previous test report: [Poppy] List App with multiple workflows and R/W split disabled test report
**Query used in lists - "Item status != Available". List refresh result is about 200K records.
Grafana results
DI + CICO
ListApp + DI + CICO
...
ListApp + DI + CICO with with R/W split disabled*
*Results are taken from the previous report: [Poppy] List App with multiple workflows and R/W split disabled test report.
...
DB CPU Utilization
RDS CPU utilization (writer instance) is about 96% for writer instance 96% for DI+CICO test and it doesn't increase with ListApp scenario added. RDS CPU utilization for reader instance reaches 95% during ListApp+DI+CICO test.
RDS CPU utilizatoin for ListApp+DI+CICO (writer instance) didn't change in comparion with the same test but with R/W split disabled for mod-fqm-manager.
...
Infrastructure
PTF -environment pcp1
- 10 m6i.2xlarge EC2 instances located in US East (N. Virginia)us-east-1
1 database instance, writer
Name API Name Memory GIB vCPUs max_connections R6G Extra Large db.r6g.xlarge 32 GiB 4 vCPUs 2731 - MSK tenant
- 4 m5.2xlarge brokers in 2 zones
Apache Kafka version 2.8.0
EBS storage volume per broker 300 GiB
- auto.create.topics.enable=true
- log.retention.minutes=480
- default.replication.factor=3
...