1 Overview
2 Summary
3 Test Results
4 Comparison
5 Resource utilization for Test #1 and Test #2
- 5.1 Instance CPU Utilization
- 5.2 Service CPU Utilization
- 5.3 Service Memory Utilization
- 5.4 DB CPU Utilization
- 5.5 DB Connections
- 5.6 Kafka metrics
- 5.7 OpenSearch Data Nodes metrics
- 5.8 DB load
- 5.9 Top SQL-queries
- 5.10 Top applications
6 Appendix
- 6.1 Infrastructure
7 Data used for export

Overview

This document contains the results of three times testing Data Export (MARC BIB) on the Sunflower[NON-ECS] CSP1 release. All test results have been summarized correctly.

PERF-1200: [Sunflower CSP1] [non-ECS] [Data Export] MARC BIBClosed

Summary

Data Export tests finished successfully(except some duplications) on using the profiles Default instances export job profile and SRS - holdings and items job profile.
Comparing with Sunflower release
- Data Export processed all files including file with 500k records with negligible duplication errors in Sunflower env.
- Data export durations are 13-32% slower compared to Sunflower releases.
- During the testing, the results were not very good, and it was decided to make a vacuum and conduct the testing again.

Test Results

This table contains durations for Data Export with two job profiles.

Profile	CSV File	Tenant (fs09000000)
Profile	CSV File	Result	Status
DE MARC Bib (Default instances export job profile)	1k.csv	0:00:08	COMPLETED
	100k.csv	0:07:00	COMPLETED
	500k.csv	0:08:43	COMPLETED
DE MARC Bib (SRS - holdings and items job profile)	1k.csv	0:00:14	COMPLETED
	100k.csv	0:18:17	COMPLETED
	500k.csv	0:22:16	COMPLETED

Test Results after vacuum

This table contains durations for Data Export with two job profiles after vacuum. Several tests were performed to obtain more results for comparison.

Profile	CSV File	Tenant (fs09000000)
Profile	CSV File	Result №1	Result №2	Result №3	Result №4	Result №5	Status
DE MARC Bib (Default instances export job profile)	1k.csv	-	-	-	-	-	COMPLETED
	100k.csv	0:01:40	00:01:39	00:01:39	00:01:39	00:01:40	COMPLETED
	500k.csv	0:05:09	00:03:11	00:03:01	00:03:00	00:02:59	COMPLETED
DE MARC Bib (SRS - holdings and items job profile)	1k.csv	-	-	-	-	-	COMPLETED
	100k.csv	0:12:52	00:07:24	00:05:29	00:05:32	00:05:22	COMPLETED
	500k.csv	0:13:29	00:14:24	00:14:29	00:14:27	00:14:19	COMPLETED

Comparison

This table contains durations comparison between Sunflower and Sunflower CSP1 releases.

Profile	CSV File	DE Duration Sunflower CSP1	DE Duration Sunflower	DE Duration, DELTA Sunflower CSP1/Sunflower
Profile	CSV File	Result hh:mm:ss	Result hh:mm:ss	hh:mm:ss / percent
DE MARC Bib (Default instances export job profile)	1k.csv	0:00:08	0:00:09	-00:00:01
	100k.csv	0:07:00	0:06:03	+00:00:57 +13.57%
	500k.csv	0:08:43	0:06:46	+00:01:57 +22.37%
DE MARC Bib (SRS - holdings and items job profile)	1k.csv	0:00:14	0:00:17	-00:00:03
	100k.csv	0:18:17	0:12:20	+00:05:57 +32.54%
	500k.csv	0:22:16	0:15:13	+00:07:03 +31.66%

Resource utilization for Tests

Resource utilization table

CPU		RAM

CPU		RAM
mod-users-keycloak-b	8.15%	mgr-applications-b	81.41%
mod-tlr-b	7.65%	mod-oa-b	76.63%
mod-login-keycloak-b	6.87%	mod-finance-b	69.99%
mod-roles-keycloak-b	6.49%	mgr-tenant-entitlements-b	68.91%
mod-data-export-b	5.44%	mod-roles-keycloak-b	68.61%
mod-requests-mediated-b	5.43%	mod-scheduler-b	67.57%
mod-consortia-keycloak-b	5.03%	mod-audit-b	66.29%
mod-inventory-storage-b	4.55%	mod-dcb-b	65.72%
mod-circulation-bff-b	3.98%	mod-users-keycloak-b	65.12%

Instance CPU Utilization

Service CPU Utilization

Here we can see that mod-data-export used 24% CPU in spike.

Service Memory Utilization

Here we can see all services have stable trends but mod-data-export which used more memory when data volume increased.

DB CPU Utilization

CPU utilization during data export peaked up to ~30% with higher loads (100k/500k), while smaller loads (1k/100k) caused moderate spikes (~15–20%).

DB Connections

DB connections was 1406.

Kafka metrics

OpenSearch Data Nodes metrics

DB load

Top SQL-queries

Top applications

Appendix

Infrastructure

PTF - environment Sunflower NON-ECS (secp1)

secp1 7 r7g.2xlarge EC2 instances located in US East (N. Virginia)us-east-1
1 instance of db.r7.xlarge database instance: Writer instance
MSK fse-test
- 4 kafka.m7g.xlarge brokers in 2 zones (2 brokers per zone)
  - Apache Kafka version 3.7.x, metadata mode - KRaft
  - EBS storage volume per broker 300 GiB
  - auto.create.topics.enable=true
  - log.retention.minutes=480
  - default.replication.factor=3
  - revision - 26
OpenSearch 2.13 ptf-test cluster
- r7g.2xlarge.search 4 data nodes
- r6g.large.search 3 dedicated master nodes

Cluster Resources - secp1 (Sep 9, 2025, 11:29:48 AM)

Module	Task Definition Revision	Module Version	Task Count	Mem Hard Limit	Mem Soft Limit	CPU Units	Xmx	Metaspace Size	Max Metaspace Size	R/W Split Enabled

Module	Task Definition Revision	Module Version	Task Count	Mem Hard Limit	Mem Soft Limit	Xmx	Metaspace Size	Max Metaspace Size	R/W Split Enabled
mod-remote-storage	11	579891902283.dkr.ecr.us-east-1.amazonaws.com/folio/mod-remote-storage:3.4.3	2	4920	4472	3960	512	512	false
mod-remote-storage - Sidecar 1	N/A	579891902283.dkr.ecr.us-east-1.amazonaws.com/folio/folio-module-sidecar:3.0.8	N/A	1024	512	256	0	96	false
mod-finance-storage	10	579891902283.dkr.ecr.us-east-1.amazonaws.com/folio/mod-finance-storage:8.8.4	2	1024	896	700	88	128	false
mod-finance-storage - Sidecar 1	N/A	579891902283.dkr.ecr.us-east-1.amazonaws.com/folio/folio-module-sidecar:3.0.8	N/A	1024	512	256	0	96	false
mod-ebsconet	10	579891902283.dkr.ecr.us-east-1.amazonaws.com/folio/mod-ebsconet:2.4.0	2	1248	1024

PTF - Data Export Test Report (Sunflower CSP1) [NON-ECS]