Table of Contents

Jira Legacy

server	System Jira
serverId	01505d01-b853-3c2e-90f1-ee9b165564fc
key	MODDATAIMP-924

Results

Test #		Profile	Splitting Feature Enabled	Results	Before Splitting Feature released	Results
1	100K MARC Create	PTF - Create 2	37 min -39 min	Completed	32-33 minutes	Completed
1	250K MARC Create	PTF - Create 2	1 hour 32 min	Completed	1 hour 33 min - 1 hour 57 min	Completed
1	500K MARC Create	PTF - Create 2	3 hours 29 min	Completed	3 hours 33 min	Completed
2	Multitenant MARC Create (100k, 50k, and 1 record)	PTF - Create 2

3 hours

2 hours 40 min	Completed			3 hours 1 min	Completed
3	CI/CO + DI MARC Create (20 users CI/CO, 25k records DI on 3 tenants)	PTF - Create 2					24 min	Completed *
4	100K MARC Update (Create new file)	PTF - Updates Success - 1	58 min 25 sec 57 min 19 sec	Completed
4	250K MARC Update	PTF - Updates Success - 1	2 hours 2 min ** 2 hours 12 min	Completed with errors ** Completed	1 hour 53 min	Completed
4	500K MARC Update	PTF - Updates Success - 1	4 hours 43 min 4 hours 38 minutes	Completed Completed

* - One record on one tenant could be discarded with error: io.netty.channel.StacklessClosedChannelException

...

** - up to 10 items were discarded with the error: io.vertx.core.impl.NoStackTraceThrowable: Cannot get actual Item by id: org.folio.inventory.exceptions.InternalServerErrorException: Access for user 'data-import-system-user' (f3486d35-f7f7-4a69-bcd0-d8e5a35cb292) requires permission: inventory-storage.items.item.get

Test#2 Multitenant MARC Create (100k, 50k, and 1 record)

Test #3 With CI/CO 20 users and DI 25k records on each of the 3 tenants

...

tenant0_mod_source_record_storage.marc_records_lb = 9674629
tenant2_mod_source_record_storage.marc_records_lb = 0
tenant3_mod_source_record_storage.marc_records_lb = 0
tenant0_mod_source_record_storage.raw_records_lb = 9604805
tenant2_mod_source_record_storage.raw_records_lb = 0
tenant3_mod_source_record_storage.raw_records_lb = 0
tenant0_mod_source_record_storage.records_lb = 9674677
tenant2_mod_source_record_storage.records_lb = 0
tenant3_mod_source_record_storage.records_lb = 0
tenant0_mod_source_record_storage.marc_indexers = 620042011
tenant2_mod_source_record_storage.marc_indexers = 0
tenant3_mod_source_record_storage.marc_indexers = 0
tenant0_mod_source_record_storage.marc_indexers with field_no 010 = 3285833
tenant2_mod_source_record_storage.marc_indexers with field_no 010 = 0
tenant3_mod_source_record_storage.marc_indexers with field_no 010 = 0
tenant0_mod_source_record_storage.marc_indexers with field_no 035 = 19241844
tenant2_mod_source_record_storage.marc_indexers with field_no 035 = 0
tenant3_mod_source_record_storage.marc_indexers with field_no 035 = 0
tenant0_mod_inventory_storage.authority = 4
tenant2_mod_inventory_storage.authority = 0
tenant3_mod_inventory_storage.authority = 0
tenant0_mod_inventory_storage.holdings_record = 9592559
tenant2_mod_inventory_storage.holdings_record = 16
tenant3_mod_inventory_storage.holdings_record = 16
tenant0_mod_inventory_storage.instance = 9976519
tenant2_mod_inventory_storage.instance = 32
tenant3_mod_inventory_storage.instance = 32
tenant0_mod_inventory_storage.item = 10787893
tenant2_mod_inventory_storage.item = 19
tenant3_mod_inventory_storage.item = 19

PTF -environment ocp3

10 m6i.2xlarge EC2 instances located in US East (N. Virginia)us-east-1
2 database instances, one reader, and one writer
Name API Name Memory GIB vCPUs max_connections
R6G Extra Large db.r6g.xlarge 32 GiB 4 vCPUs 2731
MSK ptf-kakfa-3
- 4 m5.2xlarge brokers in 2 zones
- Apache Kafka version 2.8.0
- EBS storage volume per broker 300 GiB
- auto.create.topics.enable=true
- log.retention.minutes=480
- default.replication.factor=3
Kafka topics partitioning: - 2 partitions for DI topics

...

Module ocp3-pvt Mon Sep 11 09:33:28 UTC 2023	Task Def. Revision	Module Version	Task Count	Mem Hard Limit	Mem Soft limit	CPU units	Xmx	MetaspaceSize	MaxMetaspaceSize	R/W split enabled
mod-remote-storage	13	mod-remote-storage:2.0.3	2	4920	4472	1024	3960	512	512	false
mod-agreements	8	mod-agreements:5.5.2	2	1592	1488	128	968	384	512	false
mod-data-import	7	mod-data-import:2.7.1	1	2048	1844	256	1292	384	512	false
mod-search	30	mod-search:2.0.1	2	2592	2480	2048	1440	512	1024	false
mod-authtoken	7	mod-authtoken:2.13.0	2	1440	1152	512	922	88	128	false
mod-configuration	7	mod-configuration:5.9.1	2	1024	896	128	768	88	128	false
mod-inventory-storage	1	mod-inventory-storage:26.1.0-SNAPSHOT.665	0	2208	1952	1024	1440	384	512	false
mod-circulation-storage	15	mod-circulation-storage:16.0.1	2	2880	2592	1536	1814	384	512	false
mod-source-record-storage	11	mod-source-record-storage:5.6.7	2	5600	5000	2048	3500	384	512	false
mod-calendar	7	mod-calendar:2.4.2	2	1024	896	128	768	88	128	false
mod-inventory	12	mod-inventory:20.0.6	2	2880	2592	1024	1814	384	512	false
mod-circulation	9	mod-circulation:23.5.6	2	2880	2592	1536	1814	384	512	false
mod-di-converter-storage	8	mod-di-converter-storage:2.0.5	2	1024	896	128	768	88	128	false
mod-pubsub	8	mod-pubsub:2.9.1	2	1536	1440	1024	922	384	512	false
mod-users	8	mod-users:19.1.1	2	1024	896	128	768	88	128	false
mod-patron-blocks	8	mod-patron-blocks:1.8.0	2	1024	896	1024	768	88	128	false
mod-source-record-manager	9	mod-source-record-manager:3.6.4	2	5600	5000	2048	3500	384	512	false
nginx-edge	7	nginx-edge:2023.06.14	2	1024	896	128	0	0	0	false
mod-quick-marc	7	mod-quick-marc:3.0.0	1	2288	2176	128	1664	384	512	false
nginx-okapi	7	nginx-okapi:2023.06.14	2	1024	896	128	0	0	0	false
okapi-b	8	okapi:5.0.1	3	1684	1440	1024	922	384	512	false
mod-feesfines	7	mod-feesfines:18.2.1	2	1024	896	128	768	88	128	false
mod-patron	7	mod-patron:5.5.2	2	1024	896	128	768	88	128	false
mod-notes	7	mod-notes:5.0.1	2	1024	896	128	952	384	512	false
pub-okapi	7	pub-okapi:2023.06.14	2	1024	896	128	768	0	0	false

Service versions for Splitting Feature test

Module ocp3-pvt Mon Sep 25 12:43:06 UTC 2023	Task Def. Revision	Module Version	Task Count	Mem Hard Limit	Mem Soft limit	CPU units	Xmx	MetaspaceSize	MaxMetaspaceSize	R/W split enabled
mod-data-import	10	mod-data-import:2.7.2-SNAPSHOT.137	1	2048	1844	256	1292	384	512	false
mod-search	30	mod-search:2.0.1	2	2592	2480	2048	1440	512	1024	false
mod-configuration	8	579891902283.dkr.ecr.us-east-1.amazonaws.com/folio/mod-configuration:5.9.1	2	1024	896	128	768	88	128	false
mod-bulk-operations	7	579891902283.dkr.ecr.us-east-1.amazonaws.com/folio/mod-bulk-operations:1.0.6	2	3072	2600	1024	1536	384	512	false
mod-inventory-storage	1	579891902283.dkr.ecr.us-east-1.amazonaws.com/folio/mod-inventory-storage:26.1.0-SNAPSHOT.665	0	2208	1952	1024	1440	384	512	false
mod-circulation-storage	15	579891902283.dkr.ecr.us-east-1.amazonaws.com/folio/mod-circulation-storage:16.0.1	2	2880	2592	1536	1814	384	512	false
mod-source-record-storage	12	579891902283.dkr.ecr.us-east-1.amazonaws.com/folio/mod-source-record-storage:5.6.7	2	5600	5000	2048	3500	384	512	false
mod-calendar	7	579891902283.dkr.ecr.us-east-1.amazonaws.com/folio/mod-calendar:2.4.2	2	1024	896	128	768	88	128	false
mod-inventory	12	579891902283.dkr.ecr.us-east-1.amazonaws.com/folio/mod-inventory:20.0.6	2	2880	2592	1024	1814	384	512	false
mod-circulation	9	579891902283.dkr.ecr.us-east-1.amazonaws.com/folio/mod-circulation:23.5.6	2	2880	2592	1536	1814	384	512	false
mod-di-converter-storage	8	579891902283.dkr.ecr.us-east-1.amazonaws.com/folio/mod-di-converter-storage:2.0.5	2	1024	896	128	768	88	128	false
mod-pubsub	9	579891902283.dkr.ecr.us-east-1.amazonaws.com/folio/mod-pubsub:2.9.1	2	1536	1440	1024	922	384	512	false
mod-users	9	579891902283.dkr.ecr.us-east-1.amazonaws.com/folio/mod-users:19.1.1	2	1024	896	128	768	88	128	false
mod-patron-blocks	9	579891902283.dkr.ecr.us-east-1.amazonaws.com/folio/mod-patron-blocks:1.8.0	2	1024	896	1024	768	88	128	false
mod-source-record-manager	12	579891902283.dkr.ecr.us-east-1.amazonaws.com/folio/mod-source-record-manager:3.6.5-SNAPSHOT.245	2	5600	5000	2048	3500	384	512	false
mod-quick-marc	7	579891902283.dkr.ecr.us-east-1.amazonaws.com/folio/mod-quick-marc:3.0.0	1	2288	2176	128	1664	384	512	false
nginx-okapi	7	579891902283.dkr.ecr.us-east-1.amazonaws.com/folio/nginx-okapi:2023.06.14	2	1024	896	128	0	0	0	false
okapi-b	8	579891902283.dkr.ecr.us-east-1.amazonaws.com/folio/okapi:5.0.1	3	1684	1440	1024	922	384	512	false
mod-feesfines	8	579891902283.dkr.ecr.us-east-1.amazonaws.com/folio/mod-feesfines:18.2.1	2	1024	896	128	768	88	128	false
mod-notes	7	579891902283.dkr.ecr.us-east-1.amazonaws.com/folio/mod-notes:5.0.1	2	1024	896	128	952	384	512	false
pub-okapi	7	579891902283.dkr.ecr.us-east-1.amazonaws.com/folio/pub-okapi:2023.06.14	2	1024	896	128	768	0	0	false

Methodology/Approach

Test 1: Manually tested 100k, 250k, and 500k records files started one by one on one tenant only.

...

Version	Old Version 15	New Version 16
Changes made by	Olga Kondratenko	Mykhailo Petryshyn
Saved on	Sep 22, 2023	Sep 25, 2023

Versions Compared

Key

Results

Methodology/Approach

Name	API Name	Memory GIB	vCPUs	max_connections
R6G Extra Large	db.r6g.xlarge	32 GiB	4 vCPUs	2731

Page Comparison

Versions Compared

Key

Results

Methodology/Approach