It's been found after testing that the actual durations of the imports performed were about 2 (two) times longer than what was reported. This is due to the PTF environment missing a DB trigger that, when restored, doubled the imports' durations.

Overview

This document contains the results of testing Data Import in Morning Glory to detect performance trends. PERF-265

Infrastructure

10 m6i.2xlarge EC2 instances (changed. In Lotus it was m5.xlarge)
2 instances of db.r6.xlarge database instances, one reader and one writer
MSK
- 4 m5.2xlarge brokers in 2 zones
- auto.create-topics.enable = true
- log.retention.minutes=120
- 2 partitions per DI topics
mod-inventory memory
- 1024 CPU units, 2592MB mem
- inventory.kafka.DataImportConsumerVerticle.instancesNumber=10
- inventory.kafka.MarcBibInstanceHridSetConsumerVerticle.instancesNumber=10
- kafka.consumer.max.poll.records=10
mod-inventory-storage
- 1024 CPU units, 1684MB mem
mod-source-record-storage
- 1024 CPU units, 1296MB mem
mod-source-record-manager
- 1024 CPU units, 1844MB mem
mod-data-import
- 256 CPU units, 1844MB mem
mod-data-import-cs
- 128 CPU units, 896MB mem

Infrastructure comparison with MG vs Lotus

Please note that infrastructure has being migrated from 6 m5.xlarge EC2 instances to 10 m6i.2xlarge. Key differences below:

Instance	CPU	RAM
m5.xlarge	4	16
m6i.2xlarge	8	32

Differences in modules memory and CPU parameters

Module	CPU	RAM	CPU	RAM
Module	Lotus		Morning Glory
mod-inventory	256	1814MB	1024	2592MB
mod-inventory-storage	128	544MB	1024	1684MB
mod-SRS	128	908MB	1024	1296MB
mod-SRM	128	1292MB	1024	1844MB
mod-data-import	128	1024MB	256	1844MB
mod-data-import-cs	-	-	128	896MB

Software versions

mod-data-import v2.5.0
mod-data-import-converter-storage v1.14.0
mod-source-record-manager v3.4.0
mod-source-record-storage v5.4.0
mod-inventory v18.2.0
mod-inventory-storage v24.0.0

Results

Summary

Morning Glory release is faster than lotus (with taking into account that most of memory and CPU parameters per modules containers has being changed).

So far we can only compare results of PTF-Create-2 job profile, while Update-success-2 is not available on our Morning-Glory env.

	Profile	Duration Morning Glory	Duration Lotus	delta
1K MARC Create	PTF - Create 2	50 s	1 min 9 s	38%
1K MARC Update	PTF - Updates Success - 1	39s	1 min 30 s
2K MARC Create	PTF - Create 2	1 min 2s	1 min 34 s	51%
2K MARC Update	PTF - Updates Success - 1	1 min 11 s	1 min 54 s
5K MARC Create	PTF - Create 2	2 min 20s	3 min 54 s	67%
5K MARC Update	PTF - Updates Success - 1	3 min 4 s	4 min 12 s
10K MARC Create	PTF - Create 2	4 min 33 s	6 min 45 s	48%
10K MARC Update	PTF - Updates Success - 1	5 min 29 s	8 min 4 s
25K MARC Create	PTF - Create 2	10 min 55 s	16 min 8s	47%
25K MARC Update	PTF - Updates Success - 1	13 min 37 s	19 min 50s
50K MARC Create	PTF - Create 2	21 min 37 s	32 min 28 s	50%
50K MARC Update	PTF - Updates Success - 1	26 min 10 s	39 min 5 s
100K MARC Create	PTF - Create 2	44 min 4 s	1 hr 11 min	86%
100K MARC Update	PTF - Updates Success - 1	55 min 33 s	1 hr 19 min
500K MARC Create	PTF - Create 2	3 hr 55 min Completed with errors	7 hr 4 min (Completed with errors)

Resources usage

Note: Service CPU utilisation is much lower than it was in Lotus release. It's affected by changing of CPU and Memory parameters.

Note:

mod-source-record-manager memory consumption started to grow severely after restarting (and adding additional partition to kafka topics). However after establishing all needed connections it staed on te same level for all next tests.

Note:

Instance CPU usage is lower comparing to Lotus as well because of changing instance types for Morning Glory release. (4VCPU vs 8 VCPU).

Note: RDS CPU usage is more or less the same for both releases and it's ±80%.

Folio Development Teams

Data Import test report (Morning Glory)