Versions Compared

Version Old Version 16 New Version 17
Changes made by

Leonid Kolesnykov

Leonid Kolesnykov

Saved on

Key

  • This line was added.
  • This line was removed.
  • Formatting was changed.

Table of Contents
Overview

...

  • Comparing kafka.m5.2xlarge against kafka.m7g.2xlarge instance type
    • Data Import durations and CI/CO response time do not differ significantly. The number of requests during in 2 hour CI/CO with Data Import test stayed the same tests were similar for both MSK clusters- 287669 in m5 and 287155 in m7g MSK instance type.
  • Resource utilization
    • Memory utilization didn't differ a lot between two MSK clusters

    • The m7g instance type consumes fewer resources in both idle and CICO+DI scenarios. CPU utilization is almost the same in the idle state but it is 3% to 18% lower under load during CI/CO + DI with the m7g instance type.

    • Delta for CPU utilization shows in mod-di-converter-storage-b 20% decrease for update job and 10% decrease for mod-feesfines-b module. The most part of Average DB CPU utilization is 85% during create jobs and 87% during update jobs for tests with different MSK instance types. DB CPU utilized 15% during Check-In/Check-Out period without DI.
    • Average connection count is about 850 connections for create and update jobs with CI/CO and 730 connections for CI/CO without data import for tests with different MSK instance types.
    • MSK instance CPU and Disk utilization is similar in both kafka.m7g.2xlarge and kafka.m5.2xlarge.
    • Deltas for CPU utilization shows in mod-di-converter-storage-b 20% decrease for update job and 10% decrease for mod-feesfines-b module. The rest modules CPU utilization deltas fluctuate under 10%. 
    • Average DB CPU usage for both MSK clusters during data import is 85% during create jobs and 87& during Update jobs. Check-In/Check-Out period without DI - 15%.
    • Average connection count for both MSK clusters during data import is about 850 connections for create and update jobs with CI/CO. And 730 connections for CI/CO without data import
    • MSK instance CPU and Disk utilization remain on the same level in kafka.m7g.2xlarge against kafka.m5.2xlarge or even decreased.

Test Runs 

...

Test #

...

Scenario

...

Test Results

This table shows results of Check-In/Check-out and Data Import create and update jobs.

The only difference between tests is MSK cluster instance type. Cluster ptf-mobius-testing2 has kafka.m5.2xlarge and cluster PERF-921 has kafka.m7g.2xlarge

...

Check-in/Check-out without DI

...

Response time, sec
MSK instance: kafka.m5.2xlarge

...

Response time, sec
MSK instance: kafka.m7g.2xlarge

...

Comparison

Data Import durations and Check-In/Check-Out response time comparison

Data Import durations and CI/CO response time do not differ significantly. The number of requests during 2 hour CI/CO with Data Import test stayed the same 287669 in m5 and 287155 in m7g MSK instance type.

...

Kafka resource utilization comparison table

The m7g instance type consumes fewer resources in both idle and CICO+DI scenarios. CPU utilization is almost the same in the idle state but it is 3% to 18% lower under load during CI/CO + DI with the m7g instance type.

...

This table show comparison results of CICO without Data Import in two MSK clusters

...

Test Runs 

Test #

MSK instance type

Scenario

Load level
1kafka.m5.2xlargeCICO + DI MARC Bib Create 8 users + 5K, 25K sequentially
2DI MARC Bib Create5K, 25K sequentially
3CICO + DI MARC Bib Update 8 users + 5K, 25K sequentially
4DI MARC Bib Update5K, 25K sequentially
5kafka.m7g.2xlargeCICO + DI MARC Bib Create 8 users + 5K, 25K sequentially
6DI MARC Bib Create5K, 25K sequentially
7CICO + DI MARC Bib Update 8 users + 5K, 25K sequentially
8DI MARC Bib Update5K, 25K sequentially

Test Results

This table shows results of Check-In/Check-out and Data Import create and update jobs.

MSK instance: kafka.m5.2xlarge
Job profileFile sizeDI Duration without CI/CODI Duration with CI/COCI with DI Average secCO with DI Average sec
PTF - Create 25k00:02:3100:02:540.8991.409
25k00:11:4900:12:490.7241.152
PTF - Updates Success - 65k00:03:0600:03:140.8071.257
25k00:15:0000:15:300.7841.275
MSK instance: kafka.m7g.2xlarge
Job profileFile sizeDI Duration without CI/CODI Duration with CI/COCI with DI Average secCO with DI Average sec
PTF - Create 25k00:03:0500:02:390.7071.104
25k00:12:0300:12:080.7181.129
PTF - Updates Success - 65k00:03:3600:03:340.7421.124
25k00:17:0500:17:330.7561.148


Check-in/Check-out without DI

ScenarioLoad levelRequest

Response time, sec
MSK instance: kafka.m5.2xlarge

Response time, sec
MSK instance: kafka.m7g.2xlarge

95 percaverage95 percaverage
Circulation Check-in/Check-out (without Data import)8 usersCheck-in0.6690.5700.7200.606
Check-out1.1520.9601.2410.969

Comparison

Data Import durations and Check-In/Check-Out response time comparison

  • Data Import durations and CI/CO response time do not differ significantly. The number of requests in 2 hour CI/CO with Data Import tests were similar for both MSK clusters- 287669 in m5 and 287155 in m7g MSK instance type.
Job ProfileFile sizeDELTA, DI without CI/CODELTA, DI+CI/CODELTA, CIDELTA, CO
PTF - Create 25k00:00:3400:00:150.1920.305
25k00:00:1400:00:410.0060.023
PTF - Updates Success - 65k00:00:3100:00:200.0650.133
25k00:02:0600:02:030.0280.127


Check-in/Check-out without DI

ScenarioLoad levelRequest

Response time, sec
MSK instance: kafka.m5.2xlarge

Response time, sec
MSK instance: kafka.m7g.2xlarge


Delta

95 percaverage95 percaverageAverage
Circulation Check-in/Check-out (without Data import)8 usersCheck-in0.6690.5700.7200.6060.036
Check-out1.1520.9601.2410.9690.009


MSK resource utilization (CPU)

0.009
Load scenarioBrokersMSK instance: kafka.m5.2xlargeMSK instance: kafka.m7g.2xlarge95 percAverage95 percAverageAverage
Circulation Check-in/Check-out (without Data import)8 usersCheck-in0.6690.570.720.6060.036
Check-out1.1520.961.2410.969Delta, %
CICO113.762502510.6770835-3.08
211.947916259.87916575-2.06
CICO+DI138.0916662531.13749875-6.95
233.8229112532.53334625-1.28

Response time

MSK instance: kafka.m5.2xlarge

...

Expand
titleMSK instance: kafka.m5.2xlarge vs MSK instance: kafka.m7g.2xlarge


ModuleMemory (kafka.m5.2xlarge)Memory (kafka.m7g.2xlarge)Delta
mod-dcb-b68.8174.375.56
mod-inventory-b68.2370.812.58
mod-users-b50.1750.370.2
mod-di-converter-storage-b48.6246.7-1.92
mod-feesfines-b45.5645.44-0.12
mod-inventory-storage-b45.3233.4-11.92
mod-source-record-storage-b44.2755.5311.26
okapi-b41.8542.50.65
mod-data-import-b41.4243.552.13
mod-patron-blocks-b41.0442.381.34
mod-search-b40.5545.585.03
mod-users-bl-b39.8245.365.54
mod-configuration-b38.7838.68-0.1
mod-source-record-manager-b38.4541.913.46
mod-pubsub-b36.8635.94-0.92
mod-quick-marc-b31.2542.611.35
mod-patron-b31.1930.52-0.67
mod-entities-links-b27.1234.497.37
mod-authtoken-b26.1727.321.15
mod-circulation-b24.1725.150.98
edge-patron-b22.7722.38-0.39
mod-circulation-storage-b20.0222.342.32
nginx-okapi4.694.58-0.11
pub-okapi4.524.46-0.06
pub-edge4.464.41-0.05

MSK instance: kafka.m5.2xlarge

Image Removed

MSK instance: kafka.m7g.2xlarge

Image Removed

DB CPU Utilization

Average DB CPU usage for both MSK clusters during data import is 85% during create jobs and 87& during Update jobs. Check-In/Check-Out period without DI - 15%.

MSK instance: kafka.m5.2xlarge

Image Removed

MSK instance: kafka.m7g.2xlarge

Image Removed

DB Connections

...

.342.32
nginx-okapi4.694.58-0.11
pub-okapi4.524.46-0.06
pub-edge4.464.41-0.05


MSK instance: kafka.m5.2xlarge

Image RemovedImage Added

MSK instance:

...

 kafka.m7g.2xlarge

Image Removed

MSK instance resource utilization

...

titleMSK resources table

...

Image Added

DB CPU Utilization

Average DB CPU utilization is 85% during create jobs and 87% during update jobs for tests with different MSK instance types. DB CPU utilized 15% during Check-In/Check-Out period without DI.

MSK instance: kafka.m5.2xlarge

Image Added

MSK instance: kafka.m7g.2xlarge

...

Image Added

...


...

DB Connections

Average connection count is about 850 connections for create and update jobs with CI/CO and 730 connections for CI/CO without data import for tests with different MSK instance types.

MSK instance: kafka.m5.2xlarge

Image Added

MSK instance: kafka.m7g.2xlarge

Image Added

MSK instance resource utilization

Expand
titleMSK resources table

MSK resource utilization (CPU)

Load scenarioBrokersMSK instance: kafka.m5.2xlargeMSK instance: kafka.m7g.2xlargeDelta, %
CICO113.762502510.6770835-3.08542-22.42%08
211.947916259.87916575-2.06875-17.31%06
CICO+DI138.0916662531.13749875-6.9541795-18.26%
233.8229112532.53334625-1.28957-3.81%28

MSK resource utilization (DIsk) was 4,6% with kafka.m5.2xlarge and 4,3% with kafka.m7g.2xlarge which may be neglected.

Disk usage by broker

MSK instance: kafka.m5.2xlarge

MSK instance: kafka.m7g.2xlarge

CPU (User) usage by broker

MSK instance: kafka.m5.2xlarge

MSK instance: kafka.m7g.2xlarge

...

Appendix

Infrastructure

PTF -environment qcp1

  • 10 m6i.2xlarge EC2 instances located in US East (N. Virginia)us-east-1

  • 1 database  instance, writer

    NameMemory GIBvCPUsmax_connections

    db.r6g.xlarge

    		32 GiB4 vCPUs2731


  • MSK ptf-mobius-testing2
    • 2 m5.2xlarge brokers in 2 zones

    • Apache Kafka version 2.8.0

    • EBS storage volume per broker 300 GiB

    • auto.create.topics.enable=true
    • log.retention.minutes=480
    • default.replication.factor=2
  • MSK perf-921-g2
    • 2 m7g.2xlarge brokers in 2 zones

    • Apache Kafka version 2.8.2.tiered

    • EBS storage volume per broker 300 GiB

    • auto.create.topics.enable=true
    • log.retention.minutes=480
    • default.replication.factor=2

...