Overview

• This document contains performance testing results comparing new version Kafka v4.1 with Kafka v3.7 in the Sunflower Eureka consortium (SECON) environment.

• Jira PERF-1381: Verify FOLIO Performance with Kafka v4.1ClosedPreview

Summary

• Performance verification was conducted with MSK running Kafka v4.1, with baseline tests executed on Kafka v3.7 for comparison.

• Check-In/Check-Out workflow was tested with 8 and 25 concurrent users.

• Data Import tested with 25K instances (Marc BIB Create and Update).

• Comparison of results between Kafka 3.7 and Kafka 4.1 shows minor differences in response times for the Check-In/Check-Out:

  • With 8 users, the maximum difference is +34 ms for Check-In and +81 ms for Check-Out;

  • With 25 users, the maximum difference is +10 ms for Check-In and +45 ms for Check-Out;

  • Overall, no consistent performance degradation or improvement is observed across runs.

• Data Import showed reduced execution time on Kafka 4.1:

  • 25K Creates: 1 min 29 sec faster;

  • 25K Updates: 3 min 25 sec faster.

• Resource utilization remained largely consistent across both Kafka versions for identical workloads, including service CPU and Memory, DB CPU, DB Connections, and Kafka metrics.

Test Runs 

Results

This table contains CICO Response Times:

This table contains DI durations:

Comparisons

Comparison between tests with Kafka version 3.7 and Kafka version 4.1:

Resources Utilization/comparison

Resource utilization comparison for CICO tests:

Kafka version 3.7	Kafka version 4.1

Kafka version 3.7	Kafka version 4.1
Service CPU Utilization mod-circulation is the top CPU consumer. With 8 users, peak CPU is ~74.5% (Kafka 3.7) vs ~79.9% (Kafka 4.1); with 25 users, ~235% (Kafka 3.7) vs ~204% (Kafka 4.1). Minor variations are within normal run-to-run fluctuation.
Open Open	Open Open
Service Memory Usage Memory usage is stable across both Kafka versions.
8 users: Open 25 users: Open	8 users: Open 25 users: Open
RDS CPU Utilization The average RDS CPU is ~13–14% for 8 users and ~24–25% for 25 users in both Kafka versions — nearly identical.
Open Open	Open Open
RDS Database Connections Database connections remain stable at ~1,600 across all runs with no meaningful difference between Kafka versions.
8 users: Open 25 users: Open	8 users: Open 25 users: Open
Kafka metrics Kafka 4.1 shows slightly lower CPU (System max: 2.7–2.92% vs 3.46–3.48%; User max: 8.49–9.52% vs 11.2–12.3%)
8 users: Open 25 users: Open	8 users: Open 25 users: Open

Resource utilization comparison for DI tests:

Kafka version 3.7	Kafka version 4.1

Kafka version 3.7	Kafka version 4.1
Service CPU Utilization Kafka 4.1 shows a slightly lower Create peak but higher Update peak, consistent with the different execution durations (faster completion on 4.1).
Open	Open
Service Memory Usage Memory usage is stable across both Kafka versions.
Open	Open
RDS CPU Utilization RDS CPU reaches high utilization (~92%) during both phases in both versions.
Open	Open
RDS Database Connections Max count of connections ~1,860 across both Kafka versions.
Open	Open
Kafka metrics Kafka 4.1 shows comparable CPU System usage (4.28% vs 4.8%) but higher CPU User peaks during Update (up to 32.5% vs 28.5%), correlating with faster completion.
Open	Open

Appendix

Infrastructure

PTF - environment secon

• 8 r7g.2xlarge EC2 instances located in US East (N. Virginia)us-east-1

• 1 instance of db.r7g.xlarge database instance: Writer instance

• MSK
  Baseline: fse-test
  - 4 m7g.2xlarge brokers in 2 zones
  - Apache Kafka version 3.7.x
  - EBS storage volume per broker 300 GiB

• OpenSearch 2.19 ptf-test cluster

  • r7g.2xlarge.search 4 data nodes

  • r7g.large.search 3 dedicated master nodes