Page Comparison

Versions Compared

Key

This line was added.
This line was removed.
Formatting was changed.

Table of Contents
Overview In Progress

...

Comparing kafka.m5.2xlarge, zookeeper metadata mode against kafka.m7g.2xlarge, KRaft metadata mode

Resource utilization

Test Runs

...

Test #

...

Scenario

...

- Tests with KRaft mode enabled utilize less (5% at least, for some broker - 13%) CPU by brokers during CI/CO and during DI + CI/CO and the same time it is more balanced compared to zookeeper mode
- Main workflows KPIs do not degrade in tests with Check-in/Check-out with Data import
Resource utilization

Test Runs

Test #	MSK instance type	Scenario	Load level
1	kafka.m5.2xlarge	CICO + DI MARC Bib Create	8 users + 5K, 25K sequentially
2		DI MARC Bib Create	5K, 25K sequentially
3		CICO + DI MARC Bib Update	8 users + 5K, 25K sequentially
4		DI MARC Bib Update	5K, 25K sequentially
5	kafka.m7g.2xlarge	CICO + DI MARC Bib Create	8 users + 5K, 25K sequentially
6		DI MARC Bib Create	5K, 25K sequentially
7		CICO + DI MARC Bib Update	8 users + 5K, 25K sequentially
8		DI MARC Bib Update	5K, 25K sequentially

...

This table shows results of Check-In/Check-out and Data Import create and update jobs.

MSK instance: kafka.m5.2xlarge, metadata mode - ZooKeeper
Job profile	File size	DI Duration without CI/CO	DI Duration with CI/CO	CI with DI Average sec	CO with DI Average sec
PTF - Create 2	5k	00:03:45	00:02:44	0.736	1.16
	25k	00:14:40	00:13:36	0.787	1.176
PTF - Updates Success - 6	5k	00:04:43	00:04:18	0.764	1.153
	25k	00:20:21	00:21:25	0.767	1.179
MSK instance: kafka.m7g.2xlarge, metadata mode - KRaft
Job profile	File size	DI Duration without CI/CO	DI Duration with CI/CO	CI with DI Average sec	CO with DI Average sec
PTF - Create 2	5k	00:02:49	00:02:39	0.765	1.118
	25k	00:13:31	00:12:04	0.777	1.186
PTF - Updates Success - 6	5k	00:04:36	00:04:31	0.706	1.095
	25k	00:24:07	00:21:50	0.74	1.16

Check-in/Check-out without DI

...

Data import durations fluctuate within a 10% range of the baseline (tests with Zookeeper metadata mode)
Response times of CI/CO with Data import do not differ in both MSK clusters

Job Profile	File size	DELTA, DI without CI/CO	DELTA, DI+CI/CO	DELTA, CI with DI	DELTA, CO with DI
PTF - Create 2	5k	00:00:56	00:00:05	-0.029	0.042
PTF - Create 2	25k	00:01:09	00:01:32	0.01	-0.01
PTF - Updates Success - 6	5k	00:00:07	00:00:13	0.058	0.058
PTF - Updates Success - 6	25k	00:03:46	00:00:25	0.027	0.019

Check-in/Check-out without DI

...

MSK resource utilization (CPU)

Load scenario	Brokers	MSK instance: kafka.m5.2xlarge	MSK instance: kafka.m7g.2xlarge	Delta, %
CICO	1
CICO	2
CICO+DI	1
CICO+DI	2

Response time

MSK instance: kafka.m5.2xlarge

...

MSK instance: kafka.m7g.2xlarge

MSK instance resource utilization

Tests with KRaft mode enabled utilize less CPU resources during CI/CO and during DI + CI/CO and the same time it is more balanced compared to zookeeper mode
The difference is 5% at least. For some brokers the difference is 13%.

Expand

title	MSK resources table

MSK resource utilization (CPU)

Load scenario

Load scenario	Brokers	MSK instance: kafka.m5.2xlarge	MSK instance: kafka.m7g.2xlarge	Delta, %
CICO	1	13	9	-4
CICO	2	13	9	-4
CICO+DI	1	45	32	-13
CICO+DI	2	34	30	-4

MSK resource utilization (DIsk) was 4,6% with kafkagrowing gradually during tests with kafka.m5.2xlarge and 4,3% with kafka.m7g.2xlarge which may be neglected.to 10%

Disk usage by broker

MSK instance: kafka.m5.2xlarge

Image Added

MSK instance: kafka.m7g.2xlarge

Image Added

CPU (User) usage by broker

MSK instance: kafka.m5.2xlarge

Image Added

MSK instance: kafka.m7g.2xlarge

Image Added

DB load

MSK instance: kafka.m5.2xlarge

...

AppendixInfrastructure

PTF -environment qcp1

10 m6i.2xlarge EC2 instances located in US East (N. Virginia)us-east-1
1 database instance, writer
Name Memory GIB vCPUs max_connections
db.r6g.xlarge
32 GiB 4 vCPUs 2731
MSK ptf-mobius-testing2
- 2 m5.2xlarge brokers in 2 zones (total 2 brokers)
- Apache Kafka version 2.8.0
- EBS storage volume per broker 300 GiB
- auto.create.topics.enable=true
- log.retention.minutes=480
- default.replication.factor=2
- revision - 2
- metadata mode - ZooKeeper
MSK ptf-KRaft-mode
- 2 m7g.2xlarge brokers in 2 zones (total 2 brokers)
- Apache Kafka version 3.7.x
- EBS storage volume per broker 300 GiB
- auto.create.topics.enable=true
- log.retention.minutes=480
- default.replication.factor=3
- revision - 26
- metadata mode - KRaft

...

Expand

title	All qcp1 modules

Module	Task Definition Revision	Module Version	Task Count	Mem Hard Limit	Mem Soft Limit	CPU Units	Xmx	Metaspace Size	Max Metaspace Size
mod-remote-storage	5	mod-remote-storage:3.2.0	2	4920	4472	1024	3960	512	512
mod-ncip	5	mod-ncip:1.14.4	2	1024	896	128	768	88	128
mod-finance-storage	5	mod-finance-storage:8.6.0	2	1024	896	1024	700	88	128
mod-agreements	5	mod-agreements:7.0.0	2	1592	1488	128	0	0	0
mod-ebsconet	5	mod-ebsconet:2.2.0	2	1248	1024	128	700	128	256
mod-organizations	5	mod-organizations:1.9.0	2	1024	896	128	768	88	128
mod-consortia	3	mod-consortia:1.1.0	2	5136	4776	1024	4416	384	512
edge-sip2	3	edge-sip2:3.2.0-SNAPSHOT.209	2	1024	896	128	768	88	128
mod-serials-management	5	mod-serials-management:1.0.0	2	2480	2312	128	1792	384	512
mod-settings	5	mod-settings:1.0.3	2	1024	896	200	768	88	128
mod-data-import	8	mod-data-import:3.1.0	1	2048	1844	256	1292	384	512
edge-dematic	5	edge-dematic:2.2.0	1	1024	896	128	768	88	128
mod-search	5	mod-search:3.2.0	2	2592	2480	2048	1440	512	1024
mod-inn-reach	3	mod-inn-reach:3.2.0-SNAPSHOT.86	2	3600	3240	1024	2880	512	1024
mod-tags	5	mod-tags:2.2.0	2	1024	896	128	768	88	128
edge-courses	5	edge-courses:1.4.0	2	1024	896	128	768	88	128
mod-authtoken	6	mod-authtoken:2.15.1	2	1440	1152	512	922	88	128
mod-inventory-update	5	mod-inventory-update:3.3.0	2	1024	896	128	768	88	128
mod-notify	5	mod-notify:3.2.0	2	1024	896	128	768	88	128
mod-configuration	5	mod-configuration:5.10.0	2	1024	896	128	768	88	128
mod-orders-storage	5	mod-orders-storage:13.7.0	2	1024	896	512	700	88	128
edge-caiasoft	5	edge-caiasoft:2.2.0	2	1024	896	128	768	88	128
mod-login-saml	5	mod-login-saml:2.8.0	2	1024	896	128	768	88	128
mod-erm-usage-harvester	5	mod-erm-usage-harvester:4.5.0	2	1024	896	128	768	88	128
mod-licenses	5	mod-licenses:6.0.0	2	2480	2312	512	1792	384	512
mod-gobi	5	mod-gobi:2.8.0	2	1024	896	128	768	88	128
mod-password-validator	5	mod-password-validator:3.2.0	2	1440	1298	128	768	384	512
mod-bulk-operations	5	mod-bulk-operations:2.0.0	2	3072	2600	1024	1536	384	512
mod-fqm-manager	5	mod-fqm-manager:2.0.1	2	1024	896	128	768	88	128
edge-dcb	5	edge-dcb:1.1.0	2	1024	896	128	768	88	128
mod-graphql	6	mod-graphql:1.12.1	2	1024	896	128	768	88	128
mod-finance	5	mod-finance:4.9.0	2	1024	896	128	768	88	128
mod-erm-usage	5	mod-erm-usage:4.7.0	2	2800	2550	1024	1800	384	512
mod-batch-print	6	mod-batch-print:1.1.0	2	1024	896	128	768	88	128
mod-copycat	5	mod-copycat:1.6.0	2	1024	896	128	768	88	128
mod-lists	5	mod-lists:2.0.0	2	1024	896	128	768	88	128
mod-entities-links	6	mod-entities-links:3.0.0	2	2592	2480	400	1440	0	1024
mod-permissions	10	mod-permissions:6.5.0	2	1684	1544	512	1024	384	512
pub-edge	3	pub-edge:2023.06.14	2	1024	896	128	768	0	0
mod-orders	5	mod-orders:12.8.0	2	2048	1740	1024	1024	384	512
edge-patron	5	edge-patron:5.1.0	2	1024	896	256	768	88	128
edge-ncip	5	edge-ncip:1.9.2	2	1024	896	128	768	88	128
edge-inn-reach	3	edge-inn-reach:3.1.1-SNAPSHOT.45	2	1024	896	128	768	88	128
mod-users-bl	5	mod-users-bl:7.7.0	2	1440	1152	512	922	88	128
mod-oa	4	mod-oa:2.1.0-SNAPSHOT.62	2	1024	896	128	768	88	256
mod-inventory-storage	5	mod-inventory-storage:27.1.0	2	4096	3690	2048	3076	384	512
mod-invoice	6	mod-invoice:5.8.0	2	1440	1152	512	922	88	128
mod-user-import	5	mod-user-import:3.8.0	2	1024	896	128	768	88	128
mod-sender	6	mod-sender:1.12.0	2	1024	896	128	768	88	128
edge-oai-pmh	5	edge-oai-pmh:2.9.0	2	1512	1360	1024	1440	384	512
mod-data-export-worker	5	mod-data-export-worker:3.2.1	2	3072	2800	1024	2048	384	512
mod-rtac	5	mod-rtac:3.6.0	2	1024	896	128	768	88	128
mod-circulation-storage	5	mod-circulation-storage:17.2.0	2	2880	2592	1536	1814	384	512
mod-calendar	5	mod-calendar:3.1.0	2	1024	896	128	768	88	128
mod-source-record-storage	7	mod-source-record-storage:5.8.0	2	5600	5000	2048	3500	384	512
mod-event-config	5	mod-event-config:2.7.0	2	1024	896	128	768	88	128
mod-courses	5	mod-courses:1.4.10	2	1024	896	128	768	88	128
mod-circulation-item	5	mod-circulation-item:1.0.0	2	1024	896	128	768	88	128
mod-inventory	6	mod-inventory:20.2.0	2	4096	3688	1024	1814	384	512
mod-email	5	mod-email:1.17.0	2	2800	2550	512	1800	384	512
mod-pubsub	5	mod-pubsub:2.13.0	2	1536	1440	1024	922	384	512
mod-circulation	5	mod-circulation:24.2.0	2	2880	2592	1536	1814	384	512
mod-di-converter-storage	5	mod-di-converter-storage:2.2.0	2	1024	896	128	768	88	128
edge-rtac	5	edge-rtac:2.7.1	2	1024	896	128	768	88	128
edge-orders	5	edge-orders:3.0.0	2	1024	896	128	768	88	128
mod-users	6	mod-users:19.3.1	2	1024	896	128	768	88	128
mod-template-engine	5	mod-template-engine:1.20.0	2	1024	896	128	768	88	128
mod-patron-blocks	5	mod-patron-blocks:1.10.0	2	1024	896	1024	768	88	128
mod-audit	5	mod-audit:2.9.0	2	1024	896	128	768	88	128
edge-fqm	5	edge-fqm:2.0.0	2	1024	896	128	768	88	128
mod-source-record-manager	6	mod-source-record-manager:3.9.0-SNAPSHOT.330	2	5600	5000	2048	3500	384	512
nginx-edge	3	nginx-edge:2023.06.14	2	1024	896	128	0	0	0
mod-quick-marc	5	mod-quick-marc:5.1.0	1	2288	2176	128	1664	384	512
nginx-okapi	3	nginx-okapi:2023.06.14	2	1024	896	128	0	0	0
okapi-b	5	okapi:5.3.0	3	1684	1440	1024	922	384	512
mod-feesfines	5	mod-feesfines:19.1.0	2	1024	896	128	768	88	128
mod-invoice-storage	5	mod-invoice-storage:5.8.0	2	1872	1536	1024	1024	384	512
mod-dcb	6	mod-dcb:1.1.0	2	1024	896	128	768	88	128
mod-service-interaction	5	mod-service-interaction:4.0.1	2	2048	1844	256	1290	384	512
mod-data-export	17	mod-data-export:5.0.4	1	2048	1844	2048	0	0	0
mod-patron	5	mod-patron:6.1.0	2	1024	896	128	768	88	128
mod-oai-pmh	5	mod-oai-pmh:3.13.0	2	4096	3690	2048	3076	384	512
edge-connexion	5	edge-connexion:1.2.0	2	1024	896	128	768	88	128
mod-kb-ebsco-java	5	mod-kb-ebsco-java:4.0.0	2	1024	896	128	768	88	128
mod-notes	5	mod-notes:5.2.0	2	1024	896	128	952	384	512
mod-data-export-spring	5	mod-data-export-spring:3.2.0	1	2048	1844	256	1536	384	512
mod-organizations-storage	5	mod-organizations-storage:4.7.0	2	1024	896	128	768	88	128
mod-login	5	mod-login:7.11.0	2	1440	1298	1024	768	384	512
pub-okapi	3	pub-okapi:2023.06.14	2	1024	896	128	768	0	0
mod-eusage-reports	5	mod-eusage-reports:2.1.1	2	1024	896	128	768	88	128

Methodology/Approach

Populate ptf-mobius-testing2 cluster with topics from tenant cluster
Run CICO for 2 hours
After 10 min delay after start of CICO Run DI Create - Export - Update for 5 and 25k
Run alone Data Imports
Create new kafka cluster
Populate NEW cluster with topics from tenant cluster
Run CICO for 2 hours
After 10 min delay after start of CICO Run DI Create - Export - Update for 5 and 25k
Run alone Data Imports
Compare resource utilization of MSK and main KPI for CICO & DI

...

Versions Compared

Old Version 3

New Version 4

Key

Table of Contents
Overview In Progress

Test Runs

Test Runs

Response time

DB load

MSK instance: kafka.m5.2xlarge

Appendix

Infrastructure

Methodology/Approach

Name	Memory GIB	vCPUs	max_connections
db.r6g.xlarge	32 GiB	4 vCPUs	2731