| Table of Contents |
|---|
...
- mod-inventory-storage - from 11% to 79%
- mod-source-record-manager - from 33% to 61%
- mod-source-record-storage - from 37% to 63%
- mod-data-import - from 22% to 35%
mod-inventory-storage memory utilizatoin (maximum) during 80k DI job:
DB CPU Utilization
During 80k DI job there is a switch of instrances for reader and writer role. This might happen because of overload and restart of the writer instance.
...
Infrastructure
PTF -environment ncp5
- 11 m6i.2xlarge EC2 instances located in US East (N. Virginia)us-east-1
2 database instances, one reader, and one writer
Name API Name Memory GIB vCPUs max_connections R6G Extra Large db.r6g.xlarge 32 GiB 4 vCPUs 2731 - MSK ptf-kakfa-3
- 4 m5.2xlarge brokers in 2 zones
Apache Kafka version 2.8.0
EBS storage volume per broker 300 GiB
- auto.create.topics.enable=true
- log.retention.minutes=480
- default.replication.factor=3
- Kafka topics partitioning: - 2 partitions for DI topics
...
Modules | Version | Task Definition | Running Tasks | CPU | Memory | MemoryReservation | MaxMetaspaceSize | Xmx |
|---|---|---|---|---|---|---|---|---|
| mod-inventory-storage | 26.0.0 | 12 | 2 | 2048 | 4096 | 3690 | 512 | 3076 |
| mod-inventory | 20.0.6 | 12 | 2 | 1024 | 2880 | 2592 | 512 | 1814 |
| mod-source-record-storage | 5.6.7 | 27 | 2 | 2048 | 5600 | 5000 | 512 | 3500 |
| mod-quick-marc | 3.0.0 | 6 | 1 | 128 | 2288 | 2176 | 512 | 1664 |
| mod-source-record-manager | 3.6.4 | 18 | 2 | 2048 | 5600 | 5000 | 512 | 3500 |
| mod-di-converter-storage | 2.0.5 | 6 | 2 | 128 | 1024 | 896 | 128 | 768 |
| mod-data-import | 2.7.1 | 10 | 1 | 256 | 2048 | 1844 | 512 | 1292 |
| okapi | 5.0.1 | 8 | 3 | 1024 | 1684 | 1440 | 512 | 922 |
| nginx-okapi | 2022.06.14 | 7 | 2 | 128 | 1024 | 896 | - | - |
| pub-okapi | 2022.06.14 | 7 | 2 | 128 | 1024 | 896 | - | 768 |
...