Overview
This document contains the results of testing Data Import MARC Authorities in Nolana release to detect performance trends.
Ticket:
| Jira Legacy | ||||||
|---|---|---|---|---|---|---|
|
Summary
Test Results
Profile used for testing - Default - Create SRS MARC Authority
Duration:
Morning glory
Duration:
Nolana
Duration:
Orchid| Test | File | Duration | Diff_absolute | Diff_percentage | ||
|---|---|---|---|---|---|---|
| Morning glory | Nolana | Orchid | |||
| 1 | 1K | 24 s | 27 s | 41s | +14s |
| +51% | |||||
| 2 | 5K | 1 min 21 s | 1 min 15 s | 1min 21s | +6s |
|---|
| +8% | |||||
| 3 | 10K | 2 min 32 s | 2 min 31 s | 2min 53s | +22s |
|---|
| +14% | |||||
| 4 | 25K | 11 min 14 s | 7 min 7 s | 5 min 42s | -1min 25s |
|---|
| -19% | ||
| 5 | 50K | 22 |
|---|
| min | 11 min 24 s | 11 min 11s | -13s | -1% |
Instance CPU Utilization
Service CPU Utilization
...
PTF -environment ncp5
- 11 m6i.2xlarge EC2 instances located in US East (N. Virginia)us-east-1
2 database instances, one reader, and one writer
Name API Name Memory GIB vCPUs max_connections R6G Extra Large db.r6g.xlarge 32 GiB 4 vCPUs 2731 - number of connections for mod-source-record-manager and mod-source-record-storage - 30 connections.
- MSK ptf-kakfa-3
- 4 m5.2xlarge brokers in 2 zones
Apache Kafka version 2.8.0
EBS storage volume per broker 300 GiB
- auto.create.topics.enable=true
- log.retention.minutes=480
- default.replication.factor=3
- Kafka topics partitioning: - 2 partitions for DI topics
...
Modules memory and CPU parameters
Modules | Version | Task Definition | Running Tasks | CPU | Memory | MemoryReservation | MaxMetaspaceSize | Xmx |
|---|---|---|---|---|---|---|---|---|
| mod-inventory-storage | 26.0.0 | 12 | 2 | 2048 | 4096 | 3690 | 512 | 3076 |
| mod-inventory | 20.0.6 | 12 | 2 | 1024 | 2880 | 2592 | 512 | 1814 |
| mod-source-record-storage | 5.6.7 | 27 | 2 | 2048 | 5600 | 5000 | 512 | 3500 |
| mod-quick-marc | 3.0.0 | 6 | 1 | 128 | 2288 | 2176 | 512 | 1664 |
| mod-source-record-manager | 3.6.4 | 18 | 2 | 2048 | 5600 | 5000 | 512 | 3500 |
| mod-di-converter-storage | 2.0.5 | 6 | 2 | 128 | 1024 | 896 | 128 | 768 |
| mod-data-import | 2.7.1 | 10 | 1 | 256 | 2048 | 1844 | 512 | 1292 |
| okapi | 5.0.1 | 8 | 3 | 1024 | 1684 | 1440 | 512 | 922 |
| nginx-okapi | 2022.06.14 | 7 | 2 | 128 | 1024 | 896 | - | - |
| pub-okapi | 2022.06.14 | 7 | 2 | 128 | 1024 | 896 | - | 768 |