Table of Contents

minLevel	1
maxLevel	6
outline	false
style	default
type	list
printable	true

...

This document contains the results of testing Data Import for MARC Bibliographic records creates and updates with different file sizes (10K, 25K, 50K, 100K, 500K) at Ramsons release Okapi [ECS].
Ticket:

Jira Legacy

server	System Jira
serverId	01505d01-b853-3c2e-90f1-ee9b165564fc
key	PERF-964

...

mod-search deadlocks ticket
Jira Legacy
server System Jira
serverId 01505d01-b853-3c2e-90f1-ee9b165564fc
key MSEARCH-932

Results

Test #	Data-import test	Profile	Duration Ramsons (rcon)	Duration Quesnelia (qcon)	Duration Quesnelia (qcp1)	Results
1	10k MARC BIB Create	PTF - Create 2	5 min 10 s	4 min 14 sec	6 minutes	Completed
2	25k MARC BIB Create	PTF - Create 2	10 min 30 s	9 min 41 sec	13 min 41 sec	Completed
3	50k MARC BIB Create	PTF - Create 2	15 min 43 s	18 min 18 sec	21 min 59 sec	Completed
4	100k MARC BIB Create	PTF - Create 2	31 min 51 s	38 min 36 sec	40 min 16 sec	Completed
5	500k MARC BIB Create	PTF - Create 2	2 hr 37 min	3 hours 30 min	3 hours 27 min	Completed
6	10k MARC BIB Update	PTF - Updates Success - 6	7 min 10 s	5 min 59 sec	10 min 27 sec	Completed
7	25k MARC BIB Update	PTF - Updates Success - 6	19 min 3 s	19 min 52 sec	23 min 16 sec	Completed
8	50k MARC BIB Update	PTF - Updates Success - 6	38 min 53 sec	37 min 53 sec	40 min 52 sec	Completed
9	100k MARC BIB Update	PTF - Updates Success - 6	1 hr 23 min	1 hrs 14 min	1 hrs 2 min	Completed
10	500k MARC BIB Update	PTF - Updates Success - 6		6 hrs 39 min	5 hrs 31 min	Completed

Memory Utilization

Memory usage for both sets of tests (Creates and Updates) showing stable trend. Memory of all modules returned to normal state after tests finished. No memory leak suspects observed.

...

Expand

title	Cluster Resources - rcon-pvt

Module	Task Definition Revision	Module Version	Task Count	Mem Hard Limit	Mem Soft Limit	CPU Units	Xmx	Metaspace Size	Max Metaspace Size	R/W Split Enabled
mod-remote-storage	8	579891902283.dkr.ecr.us-east-1.amazonaws.com/folio/mod-remote-storage:3.3.3 mod-remote-storage:3.3.3	2	4920	4472	0	3960	512	512false
mod-finance-storage	7	579891902283.dkr.ecr.us-east-1.amazonaws.com/folio/mod-finance-storage:8.7.3	2	1024	896	0	700	88	128false
mod-ncip	7	579891902283.dkr.ecr.us-east-1.amazonaws.com/folio/mod-ncip:1.15.6	2	1024	896	0	768	88	128false
mod-agreements	8	579891902283.dkr.ecr.us-east-1.amazonaws.com/folio/mod-agreements:7.1.4	2	1592	1488	0	0	0	0false
mod-ebsconet	8	579891902283.dkr.ecr.us-east-1.amazonaws.com/folio/mod-ebsconet:2.3.1	2	1248	1024	0	700	128	256	false	mod-organizations	7	579891902283.dkr.ecr.us-east-1.amazonaws.com/folio/
mod-organizations	7	mod-organizations:2.0.0	2	1024	896	0	700	88	128	false
mod-consortia	10	579891902283.dkr.ecr.us-east-1.amazonaws.com/folio/mod-consortia:1.2.2	2	5136	4776	0	2048	512	1024	false
edge-sip2	7	579891902283.dkr.ecr.us-east-1.amazonaws.com/folio/edge-sip2:3.3.1	2	1024	896	0	768	88	128false
mod-serials-management	8	579891902283.dkr.ecr.us-east-1.amazonaws.com/folio/mod-serials-management:1.1.2	2	2480	2312	0	1792	384	512false
mod-settings	7	579891902283.dkr.ecr.us-east-1.amazonaws.com/folio/mod-settings:1.1.0	2	1024	896	0	768	88	128false
mod-data-import	10	579891902283.dkr.ecr.us-east-1.amazonaws.com/folio/mod-data-import:3.2.4	1	2048	1844	0	1292	384	512	false
mod-search	18	579891902283.dkr.ecr.us-east-1.amazonaws.com/folio/mod-search:4.0.7	2	2592	2480	0	1440	512	1024	false	edge-dematic	7	579891902283.dkr.ecr.us-east-1.amazonaws.com/folio/
edge-dematic	7	edge-dematic:2.3.1	1	1024	896	0	768	88	128	false
mod-inn-reach	4	579891902283.dkr.ecr.us-east-1.amazonaws.com/folio/mod-inn-reach:3.2.1-SNAPSHOT.102	2	3600	3240	0	2880	512	1024	false
mod-record-specifications	7	579891902283.dkr.ecr.us-east-1.amazonaws.com/folio/mod-record-specifications:1.0.2	2	1024	896	0	768	88	128false
mod-tags	7	579891902283.dkr.ecr.us-east-1.amazonaws.com/folio/mod-tags:2.3.0	2	1024	896	0	768	88	128false
mod-authtoken	9	579891902283.dkr.ecr.us-east-1.amazonaws.com/folio/mod-authtoken:2.16.1	2	1440	1152	0	922	88	128false
edge-courses	8	579891902283.dkr.ecr.us-east-1.amazonaws.com/folio/edge-courses:1.5.1	2	1024	896	0	768	88	128	false
mod-notify	7	579891902283.dkr.ecr.us-east-1.amazonaws.com/folio/mod-notify:3.3.0	2	1024	896	0	768	88	128	false
mod-inventory-update	7	579891902283.dkr.ecr.us-east-1.amazonaws.com/folio/-inventory-update	7	mod-inventory-update:4.0.0	2	1024	896	0	768	88	128	false
mod-configuration	7	579891902283.dkr.ecr.us-east-1.amazonaws.com/folio/mod-configuration:5.11.0	2	1024	896	0	768	88	128	false
mod-orders-storage	7	579891902283.dkr.ecr.us-east-1.amazonaws.com/folio/mod-orders-storage:13.8.3	2	1024	896	0	700	88	128false
edge-caiasoft	7	579891902283.dkr.ecr.us-east-1.amazonaws.com/folio/edge-caiasoft:2.3.2	2	1024	896	0	768	88	128false
mod-login-saml	7	579891902283.dkr.ecr.us-east-1.amazonaws.com/folio/mod-login-saml:2.9.3	2	1024	896	0	768	88	128false
mod-erm-usage-harvester	7	579891902283.dkr.ecr.us-east-1.amazonaws.com/folio/mod-erm-usage-harvester:5.0.1	2	1024	896	0	768	88	128	false
mod-gobi	7	579891902283.dkr.ecr.us-east-1.amazonaws.com/folio/mod-gobi:2.9.0	2	1024	896	0	700	88	128	false	mod-licenses	7	579891902283.dkr.ecr.us-east-1.amazonaws.com/folio/
mod-licenses	7	mod-licenses:6.1.2	2	2480	2312	0	1792	384	512	false
mod-password-validator	7	579891902283.dkr.ecr.us-east-1.amazonaws.com/folio/mod-password-validator:3.3.0	2	1440	1298	0	768	384	512	false
edge-dcb	8	579891902283.dkr.ecr.us-east-1.amazonaws.com/folio/edge-dcb:1.2.1	2	1024	896	0	768	88	128false
mod-bulk-operations	8	579891902283.dkr.ecr.us-east-1.amazonaws.com/folio/mod-bulk-operations:2.1.8	2	3072	2600	0	1536	384	512false
mod-fqm-manager	10	579891902283.dkr.ecr.us-east-1.amazonaws.com/folio/mod-fqm-manager:3.0.7	2	3000	2600	0	768	88	128false
mod-graphql	9	579891902283.dkr.ecr.us-east-1.amazonaws.com/folio/mod-graphql:1.13.1	2	1024	896	0	768	88	128	false
mod-finance	8	579891902283.dkr.ecr.us-east-1.amazonaws.com/folio/mod-finance:5.0.1	2	1024	896	0	700	88	128	false
mod-erm-usage	7	579891902283.dkr.ecr.us-east-1.amazonaws.com/folio/-erm-usage	7	mod-erm-usage:5.0.0	2	2800	2550	0	1800	384	512	false
mod-batch-print	7	579891902283.dkr.ecr.us-east-1.amazonaws.com/folio/mod-batch-print:1.2.0	2	1024	896	0	768	88	128	false
mod-tlr	4	579891902283.dkr.ecr.us-east-1.amazonaws.com/folio/mod-tlr:1.0.0-SNAPSHOT.8	2	1024	896	0	768	88	128false
mod-lists	12	579891902283.dkr.ecr.us-east-1.amazonaws.com/folio/mod-lists:3.0.5	2	6000	2600	0	768	88	128false
mod-copycat	7	579891902283.dkr.ecr.us-east-1.amazonaws.com/folio/mod-copycat:1.7.0	2	1024	512	0	768	88	128false
mod-entities-links	11	579891902283.dkr.ecr.us-east-1.amazonaws.com/folio/mod-entities-links:3.1.3	2	2592	2480	0	1440	0	1024	false
mod-permissions	13	579891902283.dkr.ecr.us-east-1.amazonaws.com/folio/mod-permissions:6.6.1	2	1684	1544	512	1024	384	512	false	pub-edge	7	579891902283.dkr.ecr.us-east-1.amazonaws.com/folio/
pub-edge	7	pub-edge:2023.06.14	2	1024	896	0	768	0	0	false
mod-orders	9	579891902283.dkr.ecr.us-east-1.amazonaws.com/folio/mod-orders:12.9.9	2	2048	1740	0	1024	384	512	false
edge-patron	8	579891902283.dkr.ecr.us-east-1.amazonaws.com/folio/edge-patron:5.2.1	2	1024	896	0	768	88	128false
mod-marc-migrations	26	579891902283.dkr.ecr.us-east-1.amazonaws.com/folio/mod-marc-migrations:1.0.0	2	1024	896	0	768	88	128false
edge-ncip	8	579891902283.dkr.ecr.us-east-1.amazonaws.com/folio/edge-ncip:1.10.1	2	1024	896	0	768	88	128false
edge-inn-reach	5	579891902283.dkr.ecr.us-east-1.amazonaws.com/folio/edge-inn-reach:3.3.0-SNAPSHOT.69	2	1024	896	0	768	88	128	false
mod-users-bl	7	579891902283.dkr.ecr.us-east-1.amazonaws.com/folio/mod-users-bl:7.9.3	2	1440	1152	0	922	88	128	false	mod-oa	4	579891902283.dkr.ecr.us-east-1.amazonaws.com/folio/
mod-oa	4	mod-oa:2.1.0-SNAPSHOT.66	2	1024	896	0	768	88	128	false
mod-inventory-storage	12	579891902283.dkr.ecr.us-east-1.amazonaws.com/folio/mod-inventory-storage:28.0.4	2	4096	3690	0	3076	512	1024	false
mod-invoice	8	579891902283.dkr.ecr.us-east-1.amazonaws.com/folio/mod-invoice:5.9.2	2	1440	1152	0	922	88	128false
mod-user-import	7	579891902283.dkr.ecr.us-east-1.amazonaws.com/folio/mod-user-import:3.9.0	2	1024	896	0	768	88	128false
mod-sender	7	579891902283.dkr.ecr.us-east-1.amazonaws.com/folio/mod-sender:1.13.0	2	1024	896	0	768	88	128false
edge-oai-pmh	7	579891902283.dkr.ecr.us-east-1.amazonaws.com/folio/edge-oai-pmh:2.10.0	2	1512	1360	0	1440	384	512	false
mod-data-export-worker	10	579891902283.dkr.ecr.us-east-1.amazonaws.com/folio/mod-data-export-worker:3.3.6	2	3072	2048	0	2048	384	512	false	mod-rtac	7	579891902283.dkr.ecr.us-east-1.amazonaws.com/folio/
mod-rtac	7	mod-rtac:3.7.0	2	1024	896	0	768	88	128	false
mod-circulation-storage	8	579891902283.dkr.ecr.us-east-1.amazonaws.com/folio/mod-circulation-storage:17.3.3	2	2880	2592	0	1814	384	512	false
mod-source-record-storage	13	579891902283.dkr.ecr.us-east-1.amazonaws.com/folio/mod-source-record-storage:5.9.5	2	5600	5000	0	3500	384	512false
mod-calendar	7	579891902283.dkr.ecr.us-east-1.amazonaws.com/folio/mod-calendar:3.2.0	2	1024	896	0	768	88	128false
mod-event-config	7	579891902283.dkr.ecr.us-east-1.amazonaws.com/folio/mod-event-config:2.8.0	2	1024	896	0	768	88	128false
mod-courses	8	579891902283.dkr.ecr.us-east-1.amazonaws.com/folio/mod-courses:1.4.11	2	1024	896	0	768	88	128	false
mod-circulation-item	7	579891902283.dkr.ecr.us-east-1.amazonaws.com/folio/mod-circulation-item:1.1.0	2	1024	896	0	0	0	0	false	mod-inventory	9	579891902283.dkr.ecr.us-east-1.amazonaws.com/folio/
mod-inventory	9	mod-inventory:21.0.5	2	2880	2592	0	1814	384	512	false
mod-email	8	579891902283.dkr.ecr.us-east-1.amazonaws.com/folio/mod-email:1.18.1	2	2800	2550	0	1800	384	512	false
mod-requests-mediated	4	579891902283.dkr.ecr.us-east-1.amazonaws.com/folio/mod-requests-mediated:1.0.0-SNAPSHOT.4	2	1024	896	0	768	88	128false
mod-circulation	8	579891902283.dkr.ecr.us-east-1.amazonaws.com/folio/mod-circulation:24.3.8	2	2880	2592	0	1814	384	512false
mod-pubsub	8	579891902283.dkr.ecr.us-east-1.amazonaws.com/folio/mod-pubsub:2.15.3	2	1536	1440	0	922	384	512false
mod-di-converter-storage	9	579891902283.dkr.ecr.us-east-1.amazonaws.com/folio/mod-di-converter-storage:2.3.1	2	1024	896	0	768	88	128	false
edge-rtac	7	579891902283.dkr.ecr.us-east-1.amazonaws.com/folio/edge-rtac:2.8.0	2	1024	896	0	768	88	128	false	edge-orders	7	579891902283.dkr.ecr.us-east-1.amazonaws.com/folio/
edge-orders	7	edge-orders:3.1.0	2	1024	896	0	768	88	128	false
mod-users	8	579891902283.dkr.ecr.us-east-1.amazonaws.com/folio/mod-users:19.4.5	2	1024	896	0	768	88	128	false
mod-template-engine	7	579891902283.dkr.ecr.us-east-1.amazonaws.com/folio/mod-template-engine:1.21.0	2	1024	896	0	768	88	128false
mod-patron-blocks	7	579891902283.dkr.ecr.us-east-1.amazonaws.com/folio/mod-patron-blocks:1.11.1	2	1024	896	0	768	88	128false
mod-audit	8	579891902283.dkr.ecr.us-east-1.amazonaws.com/folio/mod-audit:2.10.2	2	1024	896	0	768	88	128false
edge-fqm	9	579891902283.dkr.ecr.us-east-1.amazonaws.com/folio/edge-fqm:3.0.2	2	1024	896	0	768	88	128	false
mod-source-record-manager	8	579891902283.dkr.ecr.us-east-1.amazonaws.com/folio/mod-source-record-manager:3.9.5	2	5600	5000	0	3500	384	512	false	nginx-edge	7	579891902283.dkr.ecr.us-east-1.amazonaws.com/folio/
nginx-edge	7	nginx-edge:2023.06.14	2	1024	896	0	0	0	0	false
mod-quick-marc	7	579891902283.dkr.ecr.us-east-1.amazonaws.com/folio/mod-quick-marc:6.0.0	1	2288	2176	0	1664	384	512	false
nginx-okapi	7	579891902283.dkr.ecr.us-east-1.amazonaws.com/folio/nginx-okapi:2023.06.14	2	1024	896	0	0	0	0false
okapi-b	8	579891902283.dkr.ecr.us-east-1.amazonaws.com/folio/okapi:6.1.1	3	1684	1440	1024	922	384	512false
mod-feesfines	7	579891902283.dkr.ecr.us-east-1.amazonaws.com/folio/mod-feesfines:19.2.1	2	1024	896	0	768	88	128false
mod-invoice-storage	7	579891902283.dkr.ecr.us-east-1.amazonaws.com/folio/mod-invoice-storage:5.9.1	2	1872	1536	0	1024	384	512	false
mod-reading-room	7	579891902283.dkr.ecr.us-east-1.amazonaws.com/folio/mod-reading-room:1.0.0	2	1024	896	0	768	88	128	false	mod-dcb	8	579891902283.dkr.ecr.us-east-1.amazonaws.com/folio/
mod-dcb	8	mod-dcb:1.2.4	2	1024	896	0	768	88	128	false
mod-service-interaction	7	579891902283.dkr.ecr.us-east-1.amazonaws.com/folio/mod-service-interaction:4.1.1	2	2048	1844	0	1290	384	512	false
mod-patron	8	579891902283.dkr.ecr.us-east-1.amazonaws.com/folio/mod-patron:6.2.5	2	1024	896	0	768	88	128false
mod-data-export	13	579891902283.dkr.ecr.us-east-1.amazonaws.com/folio/mod-data-export:5.1.5	1	2048	1844	0	0	0	0false
mod-oai-pmh	7	579891902283.dkr.ecr.us-east-1.amazonaws.com/folio/mod-oai-pmh:3.14.3	2	4096	3690	0	3076	384	512false
edge-connexion	7	579891902283.dkr.ecr.us-east-1.amazonaws.com/folio/edge-connexion:1.3.1	2	1024	896	0	768	88	128	false
mod-notes	7	579891902283.dkr.ecr.us-east-1.amazonaws.com/folio/mod-notes:6.0.0	2	1024	896	0	952	384	512	false
mod-kb-ebsco-java	8	579891902283.dkr.ecr.us-east-1.amazonaws.com/folio/kb-ebsco-java	8	mod-kb-ebsco-java:5.0.0	2	1024	896	0	768	88	128false
mod-organizations-storage	7	579891902283.dkr.ecr.us-east-1.amazonaws.com/folio/mod-organizations-storage:4.8.1	2	1024	896	0	700	88	128false
mod-data-export-spring	8	579891902283.dkr.ecr.us-east-1.amazonaws.com/folio/mod-data-export-spring:3.4.3	1	2048	1844	0	1536	384	512	false
mod-login	7	579891902283.dkr.ecr.us-east-1.amazonaws.com/folio/	7	mod-login:7.12.1	2	1440	1298	0	768	384	512	false
pub-okapi	7	579891902283.dkr.ecr.us-east-1.amazonaws.com/folio/pub-okapi:2023.06.14	2	1024	896	0	768	0	0	false
edge-erm	5	579891902283.dkr.ecr.us-east-1.amazonaws.com/folio/edge-erm:1.3.0	2	1024	896	0	768	88	128	false
mod-eusage-reports	7	579891902283.dkr.ecr.us-east-1.amazonaws.com/folio/mod-eusage-reports:3.0.0	2	1024	896	0	768	88	128	false

R - Ramsons release

Dataset Size:

...

Code Block

language	sql

select file_name,started_date,completed_date, completed_date - started_date as duration ,status
from cs00000int_0001[tenant]_mod_source_record_manager.job_execution order by started_date desc limit 1000;

Version	Old Version 13	New Version Current
Changes made by	Roman_Fedynyshyn	Roman_Fedynyshyn
Saved on	Jan 21, 2025	Feb 04, 2025

Versions Compared

Key

Results

Memory Utilization

Content Comparison

Versions Compared

Key

Results

Memory Utilization