...
...
...
...
...
...
...
...
...
...
...
...
...
...
...
...
...
...
| Info |
|---|
It's been found after testing that the actual durations of the imports performed were about 2 (two) times longer than what was reported. This is due to the PTF environment missing a DB trigger that, when restored, doubled the imports' durations. |
| Table of Contents |
|---|
Overview
This is a test report of MARC Authorities Data Import running on the Morning Glory software.
Infrastructure
- 10 m6i.2xlarge EC2 instances (changed. In Lotus it was m5.xlarge)
- 2 instances of db.r6.xlarge database instances, one reader and one writer
- MSK
- 4 m5.2xlarge brokers in 2 zones
- auto.create-topics.enable = true
- log.retention.minutes=120
- 2 partitions per DI topics
- mod-inventory memory
- 1024 CPU units, 2592MB mem
- inventory.kafka.DataImportConsumerVerticle.instancesNumber=10
- inventory.kafka.MarcBibInstanceHridSetConsumerVerticle.instancesNumber=10
- kafka.consumer.max.poll.records=10
- mod-inventory-storage
- 1024 CPU units, 1684MB mem
- mod-source-record-storage
- 1024 CPU units, 1296MB mem
- mod-source-record-manager
- 1024 CPU units, 1844MB mem
- mod-data-import
- 256 CPU units, 1844MB mem
- mod-data-import-cs
- 128 CPU units, 896MB mem
...
Please note that infrastructure has being migrated from 6 m5.xlarge EC2 instances to 10 m6i.2xlarge. Key differences below:
Instance | CPU | RAM |
|---|---|---|
| m5.xlarge | 4 | 16 |
| m6i.2xlarge | 8 | 32 |
Differences in modules memory and CPU parameters
...
Profile | Duration | |
|---|---|---|
| 1K | Default - Create SRS MARC Authority | 24 sec |
5K | Default - Create SRS MARC Authority | 1 min 21 sec |
| 10K | Default - Create SRS MARC Authority | 2 min 32 sec |
| 25K | Default - Create SRS MARC Authority | 11 min 14 s |
| 50K | Default - Create SRS MARC Authority | 22 min |
Comparison to Lotus
MG Duration | Lotus Duration | Delta | |
|---|---|---|---|
| 1K | 24 sec | ||
5K | 1 min 21 sec | ||
| 10K | 2 min 32 sec | ||
| 25K | 11 min 14 s | ||
| 50K | 22 min |
Resource Usages
- CPU usage on mod-data-import-converter-storage increases over time
Jira Legacy server System JiraJIRA serverId 01505d01-b853-3c2e-90f1-ee9b165564fc key PERF-282
...
- during a 1 week of testing memory usage increased for mod-srm, mod-srs, mod-data-import-cs, and mod-inventory-storage - JIRA ticket was created
Jira Legacy server System JiraJIRA serverId 01505d01-b853-3c2e-90f1-ee9b165564fc key PERF-281 - Tasks for mod-srm, and mod-srs were unexpectedly restarted.
...