...
...
...
...
...
...
...
...
...
...
...
...
...
...
...
...
| Table of Contents |
|---|
...
This document contains the results of testing workflows Check-in/Check-out and Data Import for MARC Bibliographic records in the Quesnelia release with a new MSK instance type where zookeeper instances are not required. The main idea is to see how the kafka.m7g.2xlarge with KRaft mode affects FOLIO performance. Compared results for main workflows with different instance types: kafka.m5.2xlarge against kafka.m7g.2xlarge.
...
- Comparing kafka.m5.2xlarge, zookeeper metadata mode against kafka.m7g.2xlarge, KRaft metadata mode
- Tests with KRaft mode enabled utilize less CPU resources by brokers ((5% at least, for some broker - 13%) CPU by brokers during CI/CO and during DI + CI/CO and at the same time it is the CPU utilizations are more balanced across the brokers compared to zookeeper mode
- The previous test results * showed that the performance actually the same for m5 and m7g clusters so m7g and m7g KRaft mode should perform the same way as well. Results for comparison **
- Main workflows KPIs do not degrade in tests with Check-in/Check-out with Data import
- No memory leaks. No errors during tests.
- Resource utilization
- Memory Service memory usage idoesndoesn't differ n differ in tests with both MSK cluster
- CPU utilization for modulesService CPU utilization in Zookeeper mode: mod-inventory-b - 147%, mod-quick-marc-b - 81%, mod-di-converter-storage-b - 116%, nginx-okapi - 79% the rest of modules utilized less than 50%
- Service CPU utilization in KRaft mode: mod-inventory-b - 140%133%, mod-quick-marc-b - 80%75%, mod-di-converter-storage-b - 116%103%, nginx-okapi - 75% the rest of modules utilized less than 80%50%
- DB CPU was on level of 90% during tests with both MSK cluster
**Kafka Zookeeper Mode vs KRaft Mode MSK - instance type comparison
Test Runs
Test # | MSK instance type | Scenario | Load level |
|---|---|---|---|
| 1 | kafka.m5.2xlarge | CICO + DI MARC Bib Create | 8 users + 5K, 25K sequentially |
| 2 | DI MARC Bib Create | 5K, 25K sequentially | |
| 3 | CICO + DI MARC Bib Update | 8 users + 5K, 25K sequentially | |
| 4 | DI MARC Bib Update | 5K, 25K sequentially | |
| 5 | kafka.m7g.2xlarge | CICO + DI MARC Bib Create | 8 users + 5K, 25K sequentially |
| 6 | DI MARC Bib Create | 5K, 25K sequentially | |
| 7 | CICO + DI MARC Bib Update | 8 users + 5K, 25K sequentially | |
| 8 | DI MARC Bib Update | 5K, 25K sequentially |
...
| MSK instance: kafka.m5.2xlarge, metadata mode - ZooKeeper | |||||
|---|---|---|---|---|---|
| Job profile | File size | DI Duration without CI/CO | DI Duration with CI/CO | CI with DI Average sec | CO with DI Average sec |
| PTF - Create 2 | 5k | 00:03:45 | 00:02:44 | 0.736 | 1.16 |
| 25k | 00:14:40 | 00:13:36 | 0.787 | 1.176 | |
| PTF - Updates Success - 6 | 5k | 00:04:43 | 00:04:18 | 0.764 | 1.153 |
| 25k | 00:20:21 | 00:21:25 | 0.767 | 1.179 | |
| MSK instance: kafka.m7g.2xlarge, metadata mode - KRaft | |||||
| Job profile | File size | DI Duration without CI/CO | DI Duration with CI/CO | CI with DI Average sec | CO with DI Average sec |
| PTF - Create 2 | 5k | 00:02:49 | 00:02:39 | 0.765 | 1.118 |
| 25k | 00:13:31 | 00:12:04 | 0.777 | 1.186 | |
| PTF - Updates Success - 6 | 5k | 00:04:36 | 00:04:31 | 0.706 | 1.095 |
| 25k | 00:24:07 | 00:21:50 | 0.74 | 1.16 | |
...
| Job Profile | File size | DELTA, DI without CI/CO | DELTA, DI+CI/CO | DELTA, CI with DI | DELTA, CO with DI |
|---|---|---|---|---|---|
| PTF - Create 2 | 5k | 00:00:56 | 00:00:05 | -0.029 | 0.042 |
| 25k | 00:01:09 | 00:01:32 | 0.01 | -0.01 | |
| PTF - Updates Success - 6 | 5k | 00:00:07 | -00:00:13 | 0.058 | 0.058 |
| 25k | -00:03:46 | -00:00:25 | 0.027 | 0.019 |
Check-in/Check-out without DI
...
| Scenario | Load level | Request | Response time, sec | Response time, sec | Delta | ||
|---|---|---|---|---|---|---|---|
| 95 perc | average | 95 perc | average | Average | |||
| Circulation Check-in/Check-out (without Data import) | 8 users | Check-in | 0.695 | 0.587 | 0.695 | 0.583 | -0.004 |
| Check-out | 1.148 | 0.958 | 1.151 | 0.944 | -0.014 | ||
MSK resource utilization (CPU)
...
Appendix
Infrastructure
PTF -environment qcp1
- 10 m6i.2xlarge EC2 instances located in US East (N. Virginia)us-east-1
1 database instance, writer
Name Memory GIB vCPUs max_connections db.r6g.xlarge
32 GiB 4 vCPUs 2731 - MSK ptf-mobius-testing2
- 2 m5.2xlarge brokers in 2 zones (total 2 brokers)
Apache Kafka version 2.8.0
EBS storage volume per broker 300 GiB
- auto.create.topics.enable=true
- log.retention.minutes=480
- default.replication.factor=2
- revision - 2
- metadata mode - ZooKeeper
- Total topics: 1534
- Total partitions: 12155
- MSK ptf-KRaft-mode
- 2 m7g.2xlarge brokers in 2 zones (total 2 brokers)
Apache Kafka version 3.7.x
EBS storage volume per broker 300 GiB
- auto.create.topics.enable=true
- log.retention.minutes=480
- default.replication.factor=3
- revision - 26
- metadata mode - KRaft
- Total topics: 1474
- Total partitions: 11909
Task count for module mod-graphql set to 0 before test start.
...