1 Overview
2 Summary
3 Test Results
4 Comparison
5 Resource utilization for Test #1 and Test #2
- 5.1 Instance CPU Utilization
- 5.2 Service CPU Utilization
- 5.3 Service Memory Utilization
- 5.4 DB CPU Utilization
- 5.5 DB Connections
- 5.6 Kafka metrics
- 5.7 OpenSearch Data Nodes metrics
- 5.8 DB load
- 5.9 Top SQL-queries
- 5.10 Top applications
6 Appendix
- 6.1 Infrastructure
7 Methodology/Approach
- 7.1 SQL Query

Overview

This document contains the results of testing Data Export (MARC BIB) on the Ramsons [NON-ECS] release on rcp1 environment.

PERF-968: [Ramsons] [non-ECS] [Data Export] MARC BIBClosed

Summary

Data Export tests finished successfully on rcp1 environment using the profiles Default instances export job profile and SRS - holdings and items job profile.
Comparing with previous testing results Quesnelia and Ramsons releases
- Data Export processed all files including file with 500k records without errors for Ramsons releases.
- Data Export durations improved - around 10% in Average for Ramsons releases.

Test Results

This table contains durations for Data Export with two job profiles.

Profile	CSV File	Tenant (fs09000000)
Profile	CSV File	Result	Status
DE MARC Bib (Default instances export job profile)	1k.csv	0:00:03	COMPLETED
	100k.csv	0:02:19	COMPLETED
	500k.csv	0:04:33	COMPLETED
DE MARC Bib (SRS - holdings and items job profile)	1k.csv	0:00:06	COMPLETED
	100k.csv	0:07:02	COMPLETED
	500k.csv	0:08:57	COMPLETED

Comparison

This table contains durations comparison between Ramsons and Quesnelia releases.

Profile	CSV File	DE Duration Ramsons	DE Duration Quesnelia	DE Duration, DELTA Poppy/Quesnelia
Profile	CSV File	Result hh:mm:ss	Result hh:mm:ss	hh:mm:ss / percent
DE MARC Bib (Default instances export job profile)	1k.csv	0:00:03	0:00:02	+00:00:01
	100k.csv	0:02:19	0:02:39	-00:00:20 -12%
	500k.csv	0:04:33	0:05:21	-00:00:48 -15%
DE MARC Bib (SRS - holdings and items job profile)	1k.csv	0:00:06	0:00:05	+00:00:01
	100k.csv	0:07:02	0:08:15	-00:01:13 -15%
	500k.csv	0:08:57	0:09:22	-00:00:25 -5%

Resource utilization for Test #1 and Test #2

CPU		RAM

CPU		RAM
mod-data-export-b	42%	mod-data-export-b	79%
mod-inventory-b	1.40%	mod-inventory-b	64%
mod-source-record-storage-b	0.40%	okapi-b	51%
mod-source-record-manager-b	0.10%	mod-users-bl-b	42%
mod-inventory-storage-b	0.10%	mod-source-record-storage-b	28%
okapi-b	0.10%	mod-inventory-storage-b	27%
mod-authtoken-b	0.10%	mod-authtoken-b	18%
mod-users-bl-b	0.10%	mod-source-record-manager-b	18%
nginx-okapi	0.10%	nginx-okapi	4%

Instance CPU Utilization

Service CPU Utilization

Here we can see that mod-data-export used 42% CPU in spike.

Service Memory Utilization

Here we can see that all modules show a stable trend.

DB CPU Utilization

DB CPU spike was 45%.

DB Connections

DB connections was 1134.

Kafka metrics

OpenSearch Data Nodes metrics

DB load

Top SQL-queries

Top applications

Appendix

Infrastructure

PTF - environment Ramsons NON-ECS (rcp1)

Cluster Resources - rcp1-pvt (Thu Jan 30 12:56:18 UTC 2025)

Task Definition Revision	Module Version	Task Count	Mem Hard Limit	Mem Soft Limit	CPU Units	Xmx	Metaspace Size	Max Metaspace Size
12	mod-remote-storage:3.3.5	2	4920	4472	0	3960	512	512
9	mod-ncip:1.15.6	2	1024	896	0	768	88	128
10	mod-finance-storage:8.7.3	2	1024	896	0	700	88	128
12	mod-agreements:7.1.5	2	1592	1488	0	0	0	0
11	mod-ebsconet:2.3.1	2	1248	1024	0	700	128	256
9	mod-organizations:2.0.0	2	1024	896	0	700	88	128
4	mod-consortia:1.2.2	2	5136	4776	0	4416	512	1024
10	edge-sip2:3.3.1	2	1024	896	0	768	88	128
10	mod-settings:1.1.0	2	1024	896	0	768	88	128
12	mod-serials-management:1.1.3	2	2480	2312	0	1792	384	512
9	edge-dematic:2.3.1	1	1024	896	0	768	88	128
10	mod-data-import:3.2.4	1	2048	1844	0	1292	384	512
24	mod-search:4.0.8	2	2592	2480	0	1440	512	1024
5	mod-inn-reach:3.2.1-SNAPSHOT.102	2	3600	3240	0	2880	512	1024
18	mod-record-specifications:1.0.2	2	1024	896	0	768	88	128
9	mod-tags:2.3.0	2	1024	896	0	768	88	128
10	mod-authtoken:2.16.1	2	1440	1152	0	922	88	128

PTF - Data Export Test Report (Ramsons) [NON-ECS]