Kafka Zookeeper mode - Data Import with Check-ins Check-outs (Quesnelia)[non-ECS] MSK instance type comparison
- 1 Overview
- 2 Summary
- 3 Test Runs
- 4 Test Results
- 4.1 Comparison
- 4.2 Response time
- 4.3 Service CPU Utilization
- 4.4 Service Memory Utilization
- 4.5 DB CPU Utilization
- 4.6 DB Connections
- 4.7 MSK instance resource utilization
- 4.7.1 Disk usage by broker
- 4.7.1.1 MSK instance: kafka.m5.2xlarge
- 4.7.1.2 MSK instance: kafka.m7g.2xlarge
- 4.7.2 CPU (User) usage by broker
- 4.7.2.1 MSK instance: kafka.m5.2xlarge
- 4.7.2.2 MSK instance: kafka.m7g.2xlarge
- 4.7.1 Disk usage by broker
- 4.8 DB load
- 5 Appendix
Overview
This document contains the results of testing workflows Check-in/Check-out and Data Import for MARC Bibliographic records in the Quesnelia release with a new MSK instance type. The main idea is to see how the kafka.m7g.2xlarge affects FOLIO performance. Compared results for main workflows with different instance types: kafka.m5.2xlarge against kafka.m7g.2xlarge.
Ticket: 
Summary
Comparing kafka.m5.2xlarge against kafka.m7g.2xlarge instance type
The main KPI for the workflows do not differ significantly (Data Import durations and CI/CO response time). During 2 hour CI/CO with Data Import tests the number of requests were similar for both MSK clusters- 287669 in m5 and 287155 in m7g MSK instance type. Duration of data import update job with 25k records is longer for 2 minutes with m7g instance type.
MSK resources utilization. CPU decreased slowly (1% - 6%) with m7g instance type. Memory usage is on the same level.
Resource utilization
Memory utilization didn't differ a lot between two MSK clusters
Average DB CPU utilization is 85% during create jobs and 87% during update jobs for tests with different MSK instance types. DB CPU utilized 15% during Check-In/Check-Out period without DI.
Average connection count is about 850 connections for create and update jobs with CI/CO and 730 connections for CI/CO without data import for tests with different MSK instance types.
MSK instance CPU and Disk utilization is similar in both kafka.m7g.2xlarge and kafka.m5.2xlarge.
Deltas for CPU utilization shows in mod-di-converter-storage-b 20% decrease for update job and 10% decrease for mod-feesfines-b module. The rest modules CPU utilization deltas fluctuate under 10%.
Test Runs
Test # | MSK instance type | Scenario | Load level |
|---|---|---|---|
1 | kafka.m5.2xlarge | CICO + DI MARC Bib Create | 8 users + 5K, 25K sequentially |
2 | DI MARC Bib Create | 5K, 25K sequentially | |
3 | CICO + DI MARC Bib Update | 8 users + 5K, 25K sequentially | |
4 | DI MARC Bib Update | 5K, 25K sequentially | |
5 | kafka.m7g.2xlarge | CICO + DI MARC Bib Create | 8 users + 5K, 25K sequentially |
6 | DI MARC Bib Create | 5K, 25K sequentially | |
7 | CICO + DI MARC Bib Update | 8 users + 5K, 25K sequentially | |
8 | DI MARC Bib Update | 5K, 25K sequentially |
Test Results
This table shows results of Check-In/Check-out and Data Import create and update jobs.
MSK instance: kafka.m5.2xlarge | |||||
|---|---|---|---|---|---|
Job profile | File size | DI Duration without CI/CO | DI Duration with CI/CO | CI with DI Average sec | CO with DI Average sec |
PTF - Create 2 | 5k | 00:02:31 | 00:02:54 | 0.899 | 1.409 |
25k | 00:11:49 | 00:12:49 | 0.724 | 1.152 | |
PTF - Updates Success - 6 | 5k | 00:03:06 | 00:03:14 | 0.807 | 1.257 |
25k | 00:15:00 | 00:15:30 | 0.784 | 1.275 | |
MSK instance: kafka.m7g.2xlarge | |||||
Job profile | File size | DI Duration without CI/CO | DI Duration with CI/CO | CI with DI Average sec | CO with DI Average sec |
PTF - Create 2 | 5k | 00:03:05 | 00:02:39 | 0.707 | 1.104 |
25k | 00:12:03 | 00:12:08 | 0.718 | 1.129 | |
PTF - Updates Success - 6 | 5k | 00:03:36 | 00:03:34 | 0.742 | 1.124 |
25k | 00:17:05 | 00:17:33 | 0.756 | 1.148 | |
Check-in/Check-out without DI
Scenario | Load level | Request | Response time, sec | Response time, sec | ||
|---|---|---|---|---|---|---|
95 perc | average | 95 perc | average | |||
Circulation Check-in/Check-out (without Data import) | 8 users | Check-in | 0.669 | 0.570 | 0.720 | 0.606 |
Check-out | 1.152 | 0.960 | 1.241 | 0.969 | ||
Comparison
Data Import durations and Check-In/Check-Out response time comparison
Data Import durations and CI/CO response time do not differ significantly. The number of requests in 2 hour CI/CO with Data Import tests were similar for both MSK clusters- 287669 in m5 and 287155 in m7g MSK instance type.
Job Profile | File size | DELTA, DI without CI/CO | DELTA, DI+CI/CO | DELTA, CI with DI | DELTA, CO with DI |
|---|---|---|---|---|---|
PTF - Create 2 | 5k | 00:00:34 | 00:00:15 | 0.192 | 0.305 |
25k | 00:00:14 | 00:00:41 | 0.006 | 0.023 | |
PTF - Updates Success - 6 | 5k | 00:00:31 | 00:00:20 | 0.065 | 0.133 |
25k | 00:02:06 | 00:02:03 | 0.028 | 0.127 |
Check-in/Check-out without DI
Scenario | Load level | Request | Response time, sec | Response time, sec
| Delta | ||
|---|---|---|---|---|---|---|---|
95 perc | average | 95 perc | average | Average | |||
Circulation Check-in/Check-out (without Data import) | 8 users | Check-in | 0.669 | 0.570 | 0.720 | 0.606 | 0.036 |
Check-out | 1.152 | 0.960 | 1.241 | 0.969 | 0.009 | ||
MSK resource utilization (CPU)
Load scenario | Brokers | MSK instance: kafka.m5.2xlarge | MSK instance: kafka.m7g.2xlarge | Delta, % |
|---|---|---|---|---|
CICO | 1 | 13.7625025 | 10.6770835 | -3.08 |
2 | 11.94791625 | 9.87916575 | -2.06 | |
CICO+DI | 1 | 38.09166625 | 31.13749875 | -6.95 |
2 | 33.82291125 | 32.53334625 | -1.28 |
Response time
MSK instance: kafka.m5.2xlarge
MSK instance: kafka.m7g.2xlarge
Service CPU Utilization
Delta for CPU utilization shows in mod-di-converter-storage-b 20% decrease for update job and 10% decrease for mod-feesfines-b module. The most part of modules CPU utilization deltas fluctuate under 10%.