Overview IN PROGRES
This document contains the results of testing workflows Check-in/Check-out and Data Import for MARC Bibliographic records in the Quesnelia release with a new MSK instance type. The main idea is to see how the m7g series affects FOLIO performance. Compared results for main workflows with different instance types: kafka.m5.2xlarge against kafka.m7g.2xlarge.
Ticket: PERF-921 - Getting issue details... STATUS
Summary
- Comparing kafka.m5.2xlarge against kafka.m7g.2xlarge instance type
- Check-in / Check-out perform better in ........
- DI durations ..........
- Resource utilization
- Average CPU utilization did not exceed -----
- Memory .......
- Average DB CPU usage
- Average connection count
Test Runs
Test # | MSK instance type | Scenario | Load level |
|---|---|---|---|
| 1 | kafka.m5.2xlarge | CICO + DI MARC Bib Create | 8 users + 5K, 25K sequentially |
| 2 | DI MARC Bib Create | 5K, 25K sequentially | |
| 3 | CICO + DI MARC Bib Update | 8 users + 5K, 25K sequentially | |
| 4 | DI MARC Bib Update | 5K, 25K sequentially | |
| 5 | kafka.m7g.2xlarge | CICO + DI MARC Bib Create | 8 users + 5K, 25K sequentially |
| 6 | DI MARC Bib Create | 5K, 25K sequentially | |
| 7 | CICO + DI MARC Bib Update | 8 users + 5K, 25K sequentially | |
| 8 | DI MARC Bib Update | 5K, 25K sequentially |
Test Results
This table shows results of Check-In/Check-out and Data Import create and update jobs.
The only difference between tests is MSK cluster instance type. Cluster ptf-mobius-testing2 has kafka.m5.2xlarge and cluster PERF-921 has kafka.m7g.2xlarge
| Profile | MARC File | DI Duration Quesnelia (hh:mm:ss) | Check In, Check Out Response time (8 users) Quesnelia MSK cluster: ptf-mobius-testing2 | Check In, Check Out Response time (8 users) Quesnelia MSK cluster: PERF-921 | |||
|---|---|---|---|---|---|---|---|
| MSK cluster: ptf-mobius-testing2 | MSK cluster: PERF-921 | CI Average sec | CO Average sec | CI Average sec | CO Average sec | ||
| DI MARC Bib Create (PTF - Create 2) | 5K.mrc | 00:02:54 | |||||
| 25K.mrc | 00:12:49 | ||||||
| DI MARC Bib Update (PTF - Updates Success - 1) | 5K.mrc | 00:03:14 | |||||
| 25K.mrc | 00:15:30 | ||||||
Check-in/Check-out without DI
| Scenario | Load level | Request | Response time, sec MSK cluster: ptf-mobius-testing2 | Response time, sec | ||
|---|---|---|---|---|---|---|
| 95 perc | average | 95 perc | average | |||
| Circulation Check-in/Check-out (without Data import) | 8 users | Check-in | 0.863 | 0.716 | ||
| Check-out | 1.535 | 1.269 | ||||
Comparison
CICO with DI comparison
DI duration results without Check-In and Check-Out for Quesnelia were taken from the report Data Import test report (Quesnelia)[non-ECS].
The following table compares test results of two MSK clusters.
This table show comparison results of CICO without Data Import
| Scenario | Load level | Request | Response time, sec Poppy | Response time, sec Quesnelia | ||
|---|---|---|---|---|---|---|
| 95 perc | average | 95 perc | average | |||
| Circulation Check-in/Check-out (without Data import) | 8 users | Check-in | 0.489 | 0.431 | 0.609 | 0.521 |
| Check-out | 0.969 | 0.828 | 1.070 | 0.803 | ||
Response time
DI MARC BIB Create + CICO
DI MARC BIB Update + CICO
Service CPU Utilization
DI MARC BIB Create and Update + CICO
Service Memory Utilization
DB CPU Utilization
Average DB CPU usage during data import is about 90% It decreased 5% from 95% in Poppy.
DB Connections
Average connection count during data import is about 750 connections for create jobs that is 450 connections higher than in Poppy. For update jobs - 730 connections.
DB load
Create jobs
Update jobs
Top SQL-queries:
Appendix
Infrastructure
PTF -environment qcp1
- 10 m6i.2xlarge EC2 instances located in US East (N. Virginia)us-east-1
1 database instance, writer
Name Memory GIB vCPUs max_connections db.r6g.xlarge
32 GiB 4 vCPUs 2731 - MSK tenant
- 4 m5.2xlarge brokers in 2 zones
Apache Kafka version 2.8.0
EBS storage volume per broker 300 GiB
- auto.create.topics.enable=true
- log.retention.minutes=480
- default.replication.factor=3
Task count for modules mod-agreements, mod-serials-management, mod-graphql set to 0 during tests.
Modules
Methodology/Approach
- To run CI/CO - carrier-io artefact was used
- DI tests were started from UI