| Table of Contents |
|---|
...
- Investigate memory growing trend for mod-inventory-storage in tests set #1 (using 1 instance HRID to create all Holdings).
Test Runs
Profile used for testing - Default - Create Holdings and SRS MARC Holdings
| Set of tests № | Scenario | Test Conditions |
|---|---|---|
| 1 | DI Holdings Create (previous* approach) | 1K, 5K, 10K, 80K sequentially |
| 2 | DI Holdings Create (new** approach) | 1K, 5K, 10K, 80K sequentially |
*previous approach - Data import Holdings with mrc file where 1 instance HRID is associated to all holdings (1k, 5k, 10k, 80k)
**new approach - Data import Holdings with mrc file where 1 instance HRID is associated to 1000 holdings
Test Results
...
Set 2 - Files used to test DI create Holdings had 1 unique instance HRID for every 1000 created Holdings (new approach)
Test | Number of records in File | Duration: Quesnelia [non-ECS] Set #1 | Duration: Quesnelia [non-ECS] Set #2 | Status and Errors Quesnelia [non-ECS] Set #1, Set #2 |
|---|---|---|---|---|
| 1 | 1k | 19 sec | 25 sec | Success |
| 2 | 5k | 1 min 17 sec | 1 min 24 sec | Success |
| 3 | 10k | 2 min 32 sec | 2 min 40 sec | Success |
| 4 | 80k | 19 min 54 sec | 21 min 44 sec | Success |
Comparison
Test | Number of records in File | Duration: Poppy [non-ECS] | Duration: Quesnelia [non-ECS] Set #1 | Delta, sec | %, Delta/Poppy Duration |
|---|---|---|---|---|---|
| 1 | 1k | 32 sec | 19 sec | 13 sec | 40.63% |
| 2 | 5k | 2 min 14 sec | 1 min 17 sec | 57 sec | 42.54% |
| 3 | 10k | 4 min 35 sec | 2 min 32 sec | 2 min 3 sec | 44.73% |
| 4 | 80k | 36 min 25 sec | 19 min 54 sec | 16 min 31 sec | 45.35% |
Compared with results in previous test report: Data Import Create MARC holdings records [non-ECS] [Poppy]
...
CPU (User) usage by broker
Open Search
...
CPU
...
For all tests - 95% utilization master node
CPU utilization data node
Minimum free storage space (GiB)
Maximuim memory utilization data node
Indexing rate
RDS CPU Utilization
For all tests - 95% except of DI Holdings with 1k file - 25%.
DB Connections
DB connections - 860.
DB Load
Set #1
Set #2
SQL queries
...
| Name | Memory GIB | vCPUs | Engine version |
|---|---|---|---|
db.r6g.xlarge | 32 GB | 4 vCPUs | 16.1 |
- MSK tenant
- 2 m5.2xlarge brokers in 2 zones
- Apache Kafka version 2.8.0
- EBS storage volume per broker 300 GiB
- auto.create.topics.enable=true
- log.retention.minutes=480
- default.replication.factor=2
- Open Search ptf-test
- version OpenSearch_2_7_R20240502
- Data nodes
- Instance type - r6g.2xlarge.search
- Number of nodes - 4
- Storage type - EBS
- EBS volume size (GiB) - 500
- Dedicated master nodes
- Instance type - r6g.large.search
- Number of nodes - 3
- DB records
- fs09000000
- instances Instances - 25901331
- items Items - 27074913
- holdings Holdings - 25871735
- fs07000001
- nstances Instances - 10100620
- items Items - 1484850
- holdings Holdings - 10522266
- fs07000002
- nstances Instances - 1161275
- items Items - 1153548
- holdings Holdings - 1153548
- fs09000000
| Expand | |||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| |||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
|
...