Table of Contents |
---|
...
Previous test report: Data Import MARC Authorities (Nolana)
Summary
- All of a the tests was were done successfully without errors and or issues.
- There is no specific trend for DI performance and resources utilization for Orchid release compared to Nonala. There is degradation for smaller file jobs, while for bigger file jobs processing time improved (up to 19%). See the details...
- Max CPU utilization was observed for mod-inventory module, it reached 43% during 50K DI import. See the details...
- Max DB CPU reached 86% during 50K DI import. No memory leaks found. See the details...
...
PTF -environment ncp5
- 11 m6i.2xlarge EC2 instances located in US East (N. Virginia)us-east-1
2 database instances, one reader, and one writer
Name API Name Memory GIB vCPUs max_connections R6G Extra Large db.r6g.xlarge 32 GiB 4 vCPUs 2731 - number of connections for mod-source-record-manager and mod-source-record-storage - 30 connections.
- MSK ptf-kakfa-3
- 4 m5.2xlarge brokers in 2 zones
Apache Kafka version 2.8.0
EBS storage volume per broker 300 GiB
- auto.create.topics.enable=true
- log.retention.minutes=480
- default.replication.factor=3
- Kafka topics partitioning: - 2 partitions for DI topics
...
Modules | Version | Task Definition | Running Tasks | CPU | Memory | MemoryReservation | MaxMetaspaceSize | Xmx |
---|---|---|---|---|---|---|---|---|
mod-inventory-storage | 26.0.0 | 12 | 2 | 2048 | 4096 | 3690 | 512 | 3076 |
mod-inventory | 20.0.6 | 12 | 2 | 1024 | 2880 | 2592 | 512 | 1814 |
mod-source-record-storage | 5.6.7 | 27 | 2 | 2048 | 5600 | 5000 | 512 | 3500 |
mod-quick-marc | 3.0.0 | 6 | 1 | 128 | 2288 | 2176 | 512 | 1664 |
mod-source-record-manager | 3.6.4 | 18 | 2 | 2048 | 5600 | 5000 | 512 | 3500 |
mod-di-converter-storage | 2.0.5 | 6 | 2 | 128 | 1024 | 896 | 128 | 768 |
mod-data-import | 2.7.1 | 10 | 1 | 256 | 2048 | 1844 | 512 | 1292 |
okapi | 5.0.1 | 8 | 3 | 1024 | 1684 | 1440 | 512 | 922 |
nginx-okapi | 2022.06.14 | 7 | 2 | 128 | 1024 | 896 | - | - |
pub-okapi | 2022.06.14 | 7 | 2 | 128 | 1024 | 896 | - | 768 |
...