| Table of Contents |
|---|
Overview
This document contains the results of testing Data Import for MARC Bibliographic records at Poppy release.
| Jira Legacy | ||||||||||
|---|---|---|---|---|---|---|---|---|---|---|
|
...
- Investigate Timeout issues. Ticket created
Jira Legacy server System JIRAJira serverId 01505d01-b853-3c2e-90f1-ee9b165564fc key MODINV-924 - Check memory trends for mod-source-record-storage-b and mod-inventory during additional DI tests without cluster night shut down
- Increase CPU units allocation for mod-inventory, mod-di-converter-storage, mod-quick-marc services
- Use higher DB instance type (scale up from db.r6g.xlarge to db.r6g.2xlarge).
...
| Test # | Duration Orchid with R/W split enabled (07/09/2023) | Duration Poppy with R/W split enabled | Difference, % / sec | Results | ||
|---|---|---|---|---|---|---|
| 1 | 1k MARC BIB Create | PTF - Create 2 | 39 sec | Completed | ||
| 2 | 2k MARC BIB Create | PTF - Create 2 | 1 min 01 sec | Completed | ||
| 3 | 5k MARC BIB Create | PTF - Create 2 | 2 min 23 sec | 2 min 22 sec | ↓ 0.88% / 1 sec | Completed |
| 4 | 10k MARC BIB Create | PTF - Create 2 | 5 min 12 sec | 4 min 29 sec | ↓ 18.86% / 43 sec | Completed |
| 5 | 25k MARC BIB Create | PTF - Create 2 | 11 min 45 sec | 10 min 38 sec | ↓ 11.38% / 67 sec | Completed |
| 6 | 50k MARC BIB Create | PTF - Create 2 | 23 min 36 sec | 20 min 26 sec | ↓ 15.18% / 190 sec | Completed |
7 | 100k MARC BIB Create | PTF - Create 2 | 49 min 28 sec | 2 hours 46 min | Cancelled (stopped by user) * | |
| 8 | 1k MARC BIB Update | PTF - Updates Success - 1 | 34 sec | Completed | ||
| 9 | 2k MARC BIB Update | PTF - Updates Success - 1 | 1 min 09 sec | Completed | ||
| 10 | 5k MARC BIB Update | PTF - Updates Success - 1 | 2 min 48 sec | 2 min 31 sec | ↓ 6.66% / 17 sec | Completed |
| 11 | 10k MARC BIB Update | PTF - Updates Success - 1 | 5 min 23 sec | 5 min 13 sec | ↓ 1.84% / 10 sec | Completed |
| 12 | 25k MARC BIB Update | PTF - Updates Success - 1 | 14 min 12 sec | 12 min 27 sec | ↓ 14% / 105 sec | Completed with errors * |
| 13 | 25k MARC BIB Update | PTF - Updates Success - 1 | 2 min 15 sec | Completed with errors * | ||
| 14 | 25k MARC BIB Update | PTF - Updates Success - 1 | 12 min | Cancelled (stopped by user) * |
...
Average active sessions (AAS)
MARC BIB CREATE
Top SQL
MARC BIB UPDATE
Top SQL
INSERT INTO fs09000000_mod_source_record_manager.events_processed
INSERT INTO fs09000000_mod_source_record_manager.journal_records
MSK CPU utilization (Percent) OpenSearch
...
Infrastructure
PTF -environment pcp1
- 10 m6i.2xlarge EC2 instances located in US East (N. Virginia)us-east-1
2 database instances, writer/reader
Name Memory GIB vCPUs max_connections db.r6g.xlarge
32 GiB 4 vCPUs 2731 - MSK tenant
- 4 m5.2xlarge brokers in 2 zones
Apache Kafka version 2.8.0
EBS storage volume per broker 300 GiB
- auto.create.topics.enable=true
- log.retention.minutes=480
- default.replication.factor=3
...