Data Import test report (Morning Glory)
It's been found after testing that the actual durations of the imports performed were about 2 (two) times longer than what was reported. This is due to the PTF environment missing a DB trigger that, when restored, doubled the imports' durations.
Overview
This document contains the results of testing Data Import in Morning Glory to detect performance trends. PERF-265
Infrastructure
10 m6i.2xlarge EC2 instances (changed. In Lotus it was m5.xlarge)
2 instances of db.r6.xlarge database instances, one reader and one writer
MSK
4 m5.2xlarge brokers in 2 zones
auto.create-topics.enable = true
log.retention.minutes=120
2 partitions per DI topics
mod-inventory memory
1024 CPU units, 2592MB mem
inventory.kafka.DataImportConsumerVerticle.instancesNumber=10
inventory.kafka.MarcBibInstanceHridSetConsumerVerticle.instancesNumber=10
kafka.consumer.max.poll.records=10
mod-inventory-storage
1024 CPU units, 1684MB mem
mod-source-record-storage
1024 CPU units, 1296MB mem
mod-source-record-manager
1024 CPU units, 1844MB mem
mod-data-import
256 CPU units, 1844MB mem
mod-data-import-cs
128 CPU units, 896MB mem
Infrastructure comparison with MG vs Lotus
Please note that infrastructure has being migrated from 6 m5.xlarge EC2 instances to 10 m6i.2xlarge. Key differences below:
Instance | CPU | RAM |
|---|---|---|
m5.xlarge | 4 | 16 |
m6i.2xlarge | 8 | 32 |
Differences in modules memory and CPU parameters
Module | Lotus | Morning Glory | ||
CPU | RAM | CPU | RAM | |
|---|---|---|---|---|
mod-inventory | 256 | 1814MB | 1024 | 2592MB |
mod-inventory-storage | 128 | 544MB | 1024 | 1684MB |
mod-SRS | 128 | 908MB | 1024 | 1296MB |
mod-SRM | 128 | 1292MB | 1024 | 1844MB |
mod-data-import | 128 | 1024MB | 256 | 1844MB |
mod-data-import-cs | - | - | 128 | 896MB |
Software versions
mod-data-import v2.5.0
mod-data-import-converter-storage v1.14.0
mod-source-record-manager v3.4.0
mod-source-record-storage v5.4.0
mod-inventory v18.2.0
mod-inventory-storage v24.0.0
Results
Summary
Morning Glory release is faster than lotus (with taking into account that most of memory and CPU parameters per modules containers has being changed).
So far we can only compare results of PTF-Create-2 job profile, while Update-success-2 is not available on our Morning-Glory env.
| Duration Morning Glory | Duration Lotus | delta | |
|---|---|---|---|---|
1K MARC Create | PTF - Create 2 | 50 s | 1 min 9 s | 38% |
1K MARC Update | PTF - Updates Success - 1 | 39s | 1 min 30 s |
|
2K MARC Create | PTF - Create 2 | 1 min 2s | 1 min 34 s | 51% |
2K MARC Update | PTF - Updates Success - 1 | 1 min 11 s | 1 min 54 s |
|
5K MARC Create | PTF - Create 2 | 2 min 20s | 3 min 54 s | 67% |
5K MARC Update | PTF - Updates Success - 1 | 3 min 4 s | 4 min 12 s |
|
10K MARC Create | PTF - Create 2 | 4 min 33 s | 6 min 45 s | 48% |
10K MARC Update | PTF - Updates Success - 1 | 5 min 29 s | 8 min 4 s |
|
25K MARC Create | PTF - Create 2 | 10 min 55 s | 16 min 8s | 47% |
25K MARC Update | PTF - Updates Success - 1 | 13 min 37 s | 19 min 50s |
|
50K MARC Create | PTF - Create 2 | 21 min 37 s | 32 min 28 s | 50% |
50K MARC Update | PTF - Updates Success - 1 | 26 min 10 s | 39 min 5 s |
|
100K MARC Create | PTF - Create 2 | 44 min 4 s | 1 hr 11 min | 86% |
100K MARC Update | PTF - Updates Success - 1 | 55 min 33 s | 1 hr 19 min |
|
500K MARC Create | PTF - Create 2 | 3 hr 55 min Completed with errors | 7 hr 4 min (Completed with errors) |
|
Resources usage
Note: Service CPU utilisation is much lower than it was in Lotus release. It's affected by changing of CPU and Memory parameters.
Note:
mod-source-record-manager memory consumption started to grow severely after restarting (and adding additional partition to kafka topics). However after establishing all needed connections it staed on te same level for all next tests.
Note:
Instance CPU usage is lower comparing to Lotus as well because of changing instance types for Morning Glory release. (4VCPU vs 8 VCPU).
Note: RDS CPU usage is more or less the same for both releases and it's ±80%.
Update success 1 resource usage