Overview The purpose of the document is getting results of testing Data Import Create MARC holdings records and to detect performance trends in scope of Trillium regression ECS testing https://folio-org.atlassian.net/browse/PERF-1386
Previous report https://folio-org.atlassian.net/wiki/x/BIAPEg
Summary All tests completed successfully.
For all file sizes performance degraded comparing to previous (Quesnelia) results by 30 -30%. Main reason of degradation is 10M records larger DB and higher MSK CPU usage
No memory leaks or additional anomalies found during tests.
Recommendations & Jiras Test Runs/Results Profile used for testing - Default - Create Holdings and SRS MARC Holdings
Test Results
1
1k
25 s
37 s
48 %
Success
2
5k
1 min 24 s
2 min 4 s
47 %
Success
3
10k
2 min 40 s
4 min 12 s
57 %
Success
4
80k
21 min 44s
28 min 40 s
31 %
Success
Cluster resource utilization Service CPU Utilization Most used modules (CPU %):
Memory Utilization Memory utilization for all related modules stayed stable, without spikes and anomalies.
Instance CPU utilization Maximum loaded EC2 instance has 25%. which mean that there's enough room for other workflows.
MSK tenant cluster Disk usage by broker
CPU (User) usage by broker
CPU usage is ±15 % higher than in the same process in ECS env. That may explain slowness.
DB resources utilization RDS CPU Utilization
DB Connections
DB Load
SQL queries
Infrastructure PTF - environment Trillium(tecp1)
Data Set size
fs09000000
Number of records Trillium
Number of records Quesnelia
Number of records Trillium
Number of records Quesnelia
Instances
35 769 193
25 901 331
Holdings
36 638 287
27 074 913
Items
35 566 574
25 871 735
tecp1 resources Module Task Definition Revision Module Version Task Count Mem Hard Limit Mem Soft Limit CPU Units Xmx Metaspace Size Max Metaspace Size mod-remote-storage 1 mod-remote-storage:4.0.0 2 4920 4472 64 3960 512 512 mod-remote-storage - Sidecar 1 N/A folio-module-sidecar:4.0.1.nb N/A 1024 512 64 256 0 96 mod-finance-storage 1 mod-finance-storage:8.9.1 2 1024 896 64 700 88 128 mod-finance-storage - Sidecar 1 N/A folio-module-sidecar:4.0.1.nb N/A 1024 512 64 256 0 96 mod-ebsconet 1 mod-ebsconet:2.5.0 2 1248 1024 64 700 128 256 mod-ebsconet - Sidecar 1 N/A folio-module-sidecar:4.0.1.nb N/A 1024 512 64 256 0 96 mod-mosaic 1 mod-mosaic:1.1.0 2 1024 896 64 768 88 128 mod-mosaic - Sidecar 1 N/A folio-module-sidecar:4.0.1.nb N/A 1024 512 64 256 0 96 edge-sip2 2 edge-sip2:4.0.2 2 1024 896 64 768 88 128 mod-tags 1 mod-tags:3.0.0 2 1024 896 64 768 88 128 mod-tags - Sidecar 1 N/A folio-module-sidecar:4.0.1.nb N/A 1024 512 64 256 0 96 edge-courses 1 edge-courses:1.7.0 2 1024 896 64 768 88 128 mod-inventory-update 3 mod-inventory-update:5.1.3 2 1024 896 64 768 88 128 mod-inventory-update - Sidecar 1 N/A folio-module-sidecar:4.0.1.nb N/A 1024 512 64 256 0 96 mod-notify 1 mod-notify:3.5.0 2 1024 896 64 768 88 128 mod-notify - Sidecar 1 N/A folio-module-sidecar:4.0.1.nb N/A 1024 512 64 256 0 96 mod-configuration 1 mod-configuration:5.13.0 2 1024 896 64 768 88 128 mod-configuration - Sidecar 1 N/A folio-module-sidecar:4.0.1.nb N/A 1024 512 64 256 0 96 edge-caiasoft 1 edge-caiasoft:3.0.0 2 1024 896 64 768 88 128 folio-keycloak 2 folio-keycloak:26.6.0 3 2750 2250 1536 1536 0 0 mod-gobi 1 mod-gobi:3.1.0 2 1024 896 64 768 88 128 mod-gobi - Sidecar 1 N/A folio-module-sidecar:4.0.1.nb N/A 1024 512 64 256 0 96 mod-licenses 2 mod-licenses:6.3.3 2 2480 2312 64 1792 384 512 mod-licenses - Sidecar 1 N/A folio-module-sidecar:4.0.1.nb N/A 1024 512 64 256 0 96 mod-graphql 2 mod-graphql:1.15.1 2 1024 896 64 768 88 128 mod-graphql - Sidecar 1 N/A folio-module-sidecar:4.0.1.nb N/A 1024 512 64 256 0 96 mod-erm-usage 1 mod-erm-usage:5.2.0 2 2800 2550 64 1800 384 512 mod-erm-usage - Sidecar 1 N/A folio-module-sidecar:4.0.1.nb N/A 1024 512 64 256 0 96 mod-roles-keycloak 1 mod-roles-keycloak:4.0.3 4 4096 3688 128 3076 88 256 mod-roles-keycloak - Sidecar 1 N/A folio-module-sidecar:4.0.1.nb N/A 1024 512 128 256 0 96 mod-batch-print 1 mod-batch-print:1.4.0 2 1024 896 64 768 88 128 mod-batch-print - Sidecar 1 N/A folio-module-sidecar:4.0.1.nb N/A 1024 512 64 256 0 96 mod-copycat 1 mod-copycat:1.9.0 2 1024 896 64 768 88 128 mod-copycat - Sidecar 1 N/A folio-module-sidecar:4.0.1.nb N/A 1024 512 64 256 0 96 mod-entities-links 1 mod-entities-links:5.0.2 2 2592 2483 64 1440 0 1024 mod-entities-links - Sidecar 1 N/A folio-module-sidecar:4.0.1.nb N/A 1024 512 64 256 0 96 pub-edge 1 pub-edge:2025.09.19 2 1024 896 64 768 0 0 mod-orders 2 mod-orders:13.1.6 2 2048 1740 128 1024 384 512 mod-orders - Sidecar 1 N/A folio-module-sidecar:4.0.1.nb N/A 1024 512 64 256 0 96 kong 2 kong:3.9.6 2 6144 6144 64 768 88 128 edge-patron 1 edge-patron:5.4.0 2 1024 896 64 768 88 128 edge-ncip 2 edge-ncip:1.11.3 2 1024 896 64 768 88 128 edge-inn-reach 1 edge-inn-reach:4.0.2 2 1024 896 64 768 88 128 mod-users-bl 1 mod-users-bl:8.0.0 2 1440 1152 64 922 88 128 mod-users-bl - Sidecar 1 N/A folio-module-sidecar:4.0.1.nb N/A 1024 512 64 256 0 96 mod-invoice 1 mod-invoice:6.1.3 2 1440 1152 64 922 88 128 mod-invoice - Sidecar 1 N/A folio-module-sidecar:4.0.1.nb N/A 1024 512 64 256 0 96 mod-inventory-storage 1 mod-inventory-storage:30.0.3 2 4300 3690 512 3067 384 512 mod-inventory-storage - Sidecar 1 N/A folio-module-sidecar:4.0.1.nb N/A 1024 768 512 256 0 128 mod-user-import 1 mod-user-import:3.11.0 2 1024 896 64 768 88 128 mod-user-import - Sidecar 1 N/A folio-module-sidecar:4.0.1.nb N/A 1024 512 64 256 0 96 mod-sender 1 mod-sender:1.15.0 2 1024 896 64 768 88 128 mod-sender - Sidecar 1 N/A folio-module-sidecar:4.0.1.nb N/A 1024 512 64 256 0 96 mgr-tenant-entitlements 4 mgr-tenant-entitlements:4.0.1 2 1024 896 64 512 88 256 mod-login-keycloak 1 mod-login-keycloak:4.0.1 2 1024 896 128 512 88 256 mod-login-keycloak - Sidecar 1 N/A folio-module-sidecar:4.0.1.nb N/A 1024 512 64 256 0 96 mod-data-export-worker 2 mod-data-export-worker:3.5.2 2 3072 2800 128 2048 384 512 mod-data-export-worker - Sidecar 1 N/A folio-module-sidecar:4.0.1.nb N/A 1024 768 128 256 0 128 mod-circulation-storage 1 mod-circulation-storage:17.5.0 2 2880 2592 64 1814 384 512 mod-circulation-storage - Sidecar 1 N/A folio-module-sidecar:4.0.1.nb N/A 1024 512 64 256 0 96 mod-calendar 1 mod-calendar:4.0.0 2 2048 1536 64 1024 96 256 mod-calendar - Sidecar 1 N/A folio-module-sidecar:4.0.1.nb N/A 1024 512 64 256 0 96 mod-event-config 1 mod-event-config:2.10.0 2 1024 896 64 768 88 128 mod-event-config - Sidecar 1 N/A folio-module-sidecar:4.0.1.nb N/A 1024 512 64 256 0 96 mod-courses 1 mod-courses:1.6.0 2 1024 896 64 768 88 128 mod-courses - Sidecar 1 N/A folio-module-sidecar:4.0.1.nb N/A 1024 512 64 256 0 96 mod-circulation-item 1 mod-circulation-item:2.0.0 2 1024 896 64 0 0 0 mod-circulation-item - Sidecar 1 N/A folio-module-sidecar:4.0.1.nb N/A 1024 512 64 256 0 96 mod-email 1 mod-email:1.20.0 2 2800 2550 64 1800 384 512 mod-email - Sidecar 1 N/A folio-module-sidecar:4.0.1.nb N/A 1024 512 64 256 0 96 mod-di-converter-storage 3 mod-di-converter-storage:2.5.0 2 1024 896 64 768 88 128 mod-di-converter-storage - Sidecar 1 N/A folio-module-sidecar:4.0.1.nb N/A 1024 512 64 256 0 96 mod-pubsub 1 mod-pubsub:2.17.0 2 1536 1440 1024 922 384 512 mod-pubsub - Sidecar 1 N/A folio-module-sidecar:4.0.1.nb N/A 1024 512 512 256 0 96 mod-circulation 1 mod-circulation:24.5.0 2 2880 2592 64 1814 384 512 mod-circulation - Sidecar 1 N/A folio-module-sidecar:4.0.1.nb N/A 1024 512 64 256 0 96 edge-orders 1 edge-orders:3.3.0 2 1024 896 64 768 88 128 edge-rtac 1 edge-rtac:3.1.0 2 1024 896 64 768 88 128 mod-linked-data-import 1 mod-linked-data-import:1.0.1 2 2048 1792 64 768 88 128 mod-linked-data-import - Sidecar 1 N/A folio-module-sidecar:4.0.1.nb N/A 1024 512 64 256 0 96 mod-template-engine 1 mod-template-engine:1.23.0 2 1024 896 64 768 88 128 mod-template-engine - Sidecar 1 N/A folio-module-sidecar:4.0.1.nb N/A 1024 512 64 256 0 96 mod-audit 2 mod-audit:3.0.1 2 1024 896 64 768 88 128 mod-audit - Sidecar 1 N/A folio-module-sidecar:4.0.1.nb N/A 1024 512 64 256 0 96 mod-source-record-manager 4 mod-source-record-manager:4.0.5 2 4500 4000 64 3000 384 512 mod-source-record-manager - Sidecar 1 N/A folio-module-sidecar:4.0.1.nb N/A 1024 512 64 256 0 96 mod-quick-marc 1 mod-quick-marc:8.0.1 1 2288 2176 64 1664 384 512 mod-quick-marc - Sidecar 1 N/A folio-module-sidecar:4.0.1.nb N/A 1024 512 64 256 0 96 mod-feesfines 1 mod-feesfines:19.4.0 2 1024 896 64 768 88 128 mod-feesfines - Sidecar 1 N/A folio-module-sidecar:4.0.1.nb N/A 1024 512 64 256 0 96 edge-users 1 edge-users:2.2.0 2 1024 896 64 768 88 128 mod-dcb 1 mod-dcb:2.0.0 2 2048 1844 64 768 88 192 mod-dcb - Sidecar 1 N/A folio-module-sidecar:4.0.1.nb N/A 1024 512 64 256 0 96 mod-service-interaction 2 mod-service-interaction:4.3.2 2 2048 1844 64 1290 384 512 mod-service-interaction - Sidecar 1 N/A folio-module-sidecar:4.0.1.nb N/A 1024 512 64 256 0 96 mod-patron 2 mod-patron:6.4.1 2 1024 896 64 768 88 128 mod-patron - Sidecar 1 N/A folio-module-sidecar:4.0.1.nb N/A 1024 512 64 256 0 96 edge-connexion 1 edge-connexion:1.5.0 2 1024 896 64 768 88 128 mod-data-export-spring 1 mod-data-export-spring:3.6.1 1 2048 1844 64 1536 384 512 mod-data-export-spring - Sidecar 1 N/A folio-module-sidecar:4.0.1.nb N/A 1024 512 64 256 0 96 mod-organizations-storage 1 mod-organizations-storage:5.0.0 2 1024 896 64 620 88 128 mod-organizations-storage - Sidecar 1 N/A folio-module-sidecar:4.0.1.nb N/A 1024 512 64 256 0 96 edge-erm 1 edge-erm:1.5.0 2 1024 896 64 768 88 128 mod-ncip 1 mod-ncip:1.17.3 2 1024 896 64 768 88 128 mod-ncip - Sidecar 1 N/A folio-module-sidecar:4.0.1.nb N/A 1024 512 64 256 0 96 mod-agreements 2 mod-agreements:7.3.4 2 4096 4096 64 0 0 0 mod-agreements - Sidecar 1 N/A folio-module-sidecar:4.0.1.nb N/A 1024 512 64 256 0 96 mod-organizations 1 mod-organizations:2.2.0 2 1024 896 64 768 88 128 mod-organizations - Sidecar 1 N/A folio-module-sidecar:4.0.1.nb N/A 1024 512 64 256 0 96 mod-settings 1 mod-settings:1.3.1 2 1024 896 64 768 88 128 mod-settings - Sidecar 1 N/A folio-module-sidecar:4.0.1.nb N/A 1024 512 64 256 0 96 mod-serials-management 2 mod-serials-management:2.1.2 2 2780 2312 64 1792 384 896 mod-serials-management - Sidecar 1 N/A folio-module-sidecar:4.0.1.nb N/A 1024 512 64 256 0 96 mod-search 2 mod-search:6.0.6 2 2592 2480 64 1440 512 1024 mod-search - Sidecar 1 N/A folio-module-sidecar:4.0.1.nb N/A 1024 512 128 256 0 96 edge-dematic 1 edge-dematic:3.0.0 1 1024 896 64 768 88 128 mod-data-import 4 mod-data-import:3.4.0 1 2048 1844 64 1292 384 512 mod-data-import - Sidecar 1 N/A folio-module-sidecar:4.0.1.nb N/A 1024 512 64 256 0 96 mod-inn-reach 3 mod-inn-reach:4.0.5 2 5000 4000 64 2880 512 1024 mod-inn-reach - Sidecar 1 N/A folio-module-sidecar:4.0.1.nb N/A 1024 512 64 256 0 96 mod-record-specifications 1 mod-record-specifications:3.0.0 2 2048 1592 64 1440 256 512 mod-record-specifications - Sidecar 1 N/A folio-module-sidecar:4.0.1.nb N/A 1024 512 64 256 0 96 edge-inventory 1 edge-inventory:2.3.0 2 1024 896 64 768 88 128 mod-orders-storage 1 mod-orders-storage:14.0.2 2 1024 896 64 700 88 128 mod-orders-storage - Sidecar 1 N/A folio-module-sidecar:4.0.1.nb N/A 1024 512 64 256 0 96 mod-erm-usage-harvester 1 mod-erm-usage-harvester:5.2.0 2 1024 896 64 768 88 128 mod-erm-usage-harvester - Sidecar 1 N/A folio-module-sidecar:4.0.1.nb N/A 1024 512 64 256 0 96 mod-password-validator 1 mod-password-validator:4.0.0 2 1440 1298 64 768 384 512 mod-password-validator - Sidecar 1 N/A folio-module-sidecar:4.0.1.nb N/A 1024 512 64 256 0 96 mod-bulk-operations 2 mod-bulk-operations:2.3.4 2 5600 5200 64 2880 384 512 mod-bulk-operations - Sidecar 1 N/A folio-module-sidecar:4.0.1.nb N/A 1500 768 64 256 0 128 edge-dcb 1 edge-dcb:2.0.0 2 1024 896 64 768 88 128 mod-fqm-manager 2 mod-fqm-manager:4.1.6 2 3000 2600 64 2048 384 512 mod-fqm-manager - Sidecar 1 N/A folio-module-sidecar:4.0.1.nb N/A 1024 512 64 256 0 96 mod-finance 1 mod-finance:5.2.1 2 1024 896 64 768 88 128 mod-finance - Sidecar 1 N/A folio-module-sidecar:4.0.1.nb N/A 1024 512 64 256 0 96 mod-lists 2 mod-lists:4.0.1 2 6000 2600 64 2048 256 512 mod-lists - Sidecar 1 N/A folio-module-sidecar:4.0.1.nb N/A 1024 512 64 256 0 96 mod-users-keycloak 1 mod-users-keycloak:4.0.2 4 4096 3688 128 3076 88 256 mod-users-keycloak - Sidecar 1 N/A folio-module-sidecar:4.0.1.nb N/A 1024 512 128 256 0 96 mod-marc-migrations 3 mod-marc-migrations:3.0.2 2 2880 2592 64 1814 384 512 mod-marc-migrations - Sidecar 1 N/A folio-module-sidecar:4.0.1.nb N/A 1024 512 64 256 0 96 mgr-applications 1 mgr-applications:4.0.1 2 1024 896 64 512 88 256 mgr-tenants 1 mgr-tenants:4.0.0 2 1024 896 64 512 88 256 edge-oai-pmh 1 edge-oai-pmh:2.12.0 2 1512 1360 64 1440 384 512 mod-okapi-facade 1 mod-okapi-facade:4.0.0 2 1024 896 64 768 88 128 mod-okapi-facade - Sidecar 1 N/A folio-module-sidecar:4.0.1.nb N/A 1024 512 64 256 0 96 mod-rtac 1 mod-rtac:3.9.1 2 1024 896 64 768 88 128 mod-rtac - Sidecar 1 N/A folio-module-sidecar:4.0.1.nb N/A 1024 512 64 256 0 96 mod-scheduler 2 mod-scheduler:4.0.2 2 1024 896 64 512 88 256 mod-scheduler - Sidecar 1 N/A folio-module-sidecar:4.0.1.nb N/A 1024 512 64 256 0 96 mod-source-record-storage 2 mod-source-record-storage:6.0.1 2 4500 4000 64 3000 384 512 mod-source-record-storage - Sidecar 1 N/A folio-module-sidecar:4.0.1.nb N/A 1024 512 64 256 0 96 mod-inventory 3 mod-inventory:22.0.2 2 4096 3688 512 3076 384 512 mod-inventory - Sidecar 1 N/A folio-module-sidecar:4.0.1.nb N/A 1024 768 512 256 0 128 mod-linked-data 2 mod-linked-data:2.0.3 2 2048 1792 64 768 88 128 mod-linked-data - Sidecar 1 N/A folio-module-sidecar:4.0.1.nb N/A 1024 512 64 256 0 96 mod-patron-blocks 1 mod-patron-blocks:1.13.0 2 1024 896 64 768 88 128 mod-patron-blocks - Sidecar 1 N/A folio-module-sidecar:4.0.1.nb N/A 1024 512 64 256 0 96 mod-reporting 1 mod-reporting:1.6.1 2 1024 896 64 768 88 128 mod-reporting - Sidecar 1 N/A folio-module-sidecar:4.0.1.nb N/A 1024 512 64 256 0 96 edge-fqm 2 edge-fqm:4.1.2 2 1024 896 64 768 88 128 nginx-edge 1 nginx-edge:2025.09.19 2 1024 896 64 0 0 0 mod-invoice-storage 1 mod-invoice-storage:6.1.1 2 1872 1536 64 1024 384 512 mod-invoice-storage - Sidecar 1 N/A folio-module-sidecar:4.0.1.nb N/A 1024 512 64 256 0 96 mod-reading-room 1 mod-reading-room:2.0.0 2 1024 896 64 768 88 128 mod-reading-room - Sidecar 1 N/A folio-module-sidecar:4.0.1.nb N/A 1024 512 64 256 0 96 mod-data-export 3 mod-data-export:5.4.4 1 2592 2480 64 1440 88 1024 mod-data-export - Sidecar 1 N/A folio-module-sidecar:4.0.1.nb N/A 1024 512 64 256 0 96 mod-oai-pmh 1 mod-oai-pmh:3.16.2 2 4096 3690 64 3076 384 512 mod-oai-pmh - Sidecar 1 N/A folio-module-sidecar:4.0.1.nb N/A 1024 512 64 256 0 96 mod-kb-ebsco-java 2 mod-kb-ebsco-java:7.0.1 2 1024 896 64 768 88 128 mod-kb-ebsco-java - Sidecar 1 N/A folio-module-sidecar:4.0.1.nb N/A 1024 512 64 256 0 96 mod-notes 1 mod-notes:8.0.0 2 1024 896 64 952 384 512 mod-notes - Sidecar 1 N/A folio-module-sidecar:4.0.1.nb N/A 1024 512 64 256 0 96 mod-eusage-reports 1 mod-eusage-reports:3.1.0 2 1024 896 64 768 88 128 mod-eusage-reports - Sidecar 1 N/A folio-module-sidecar:4.0.1.nb N/A 1024 512 64 256 0 96
Methodology/Approach Prepare Data Import Files 1k, 5k, 10k, 80k with defined number of holding records associated with instance HRID (1 instance HRID for all records or 1 per 1000 records)
run Data imports using default SRS profile
If everything is successful - collect durations of DI with :
select file_name,total_records_in_file,started_date,completed_date, completed_date - started_date as duration ,status,error_status
from [tenant]_mod_source_record_manager.job_execution
where subordination_type = 'COMPOSITE_PARENT'
-- where started_date > '2024-06-13 14:47:54' and completed_date < '2024-06-13 19:01:50.832'
order by started_date desc
limit 10 