...
...
...
...
| Table of Contents |
|---|
...
The purpose of the document is getting results of testing Data Import Create MARC holdings records and to detect performance trends in Quesnelia in scope of ticket Jira Legacy server System Jira columnIds issuekey,summary,issuetype,created,updated,duedate,assignee,reporter,priority,status,resolution columns key,summary,type,created,updated,due,assignee,reporter,priority,status,resolution serverId 01505d01-b853-3c2e-90f1-ee9b165564fc key PERF-833
...
- Investigate memory growing trend for mod-inventory-storage in tests set #1 (using 1 instance HRID to create all Holdings).
Test Runs
Profile used for testing - Default - Create Holdings and SRS MARC Holdings
| Set of tests № | Scenario | Test Conditions |
|---|---|---|
| 1 | DI Holdings Create (previous* approach) | 1K, 5K, 10K, 80K sequentially |
| 2 | DI Holdings Create (new** approach) | 1K, 5K, 10K, 80K sequentially |
*previous approach - Data import Holdings with mrc file where 1 instance HRID is associated to all holdings (1k, 5k, 10k, 80k)
**new approach - Data import Holdings with mrc file where 1 instance HRID is associated to 1000 holdings
Test Results
...
Set 2 - Files used to test DI create Holdings had 1 unique instance HRID for every 1000 created Holdings (new approach)
Test | Number of records in File | Duration: Quesnelia [non-ECS] Set #1 | Duration: Quesnelia [non-ECS] Set #2 | Status and Errors Quesnelia [non-ECS] Set #1, Set #2 |
|---|---|---|---|---|
| 1 | 1k | 19 sec | 25 sec | Success |
| 2 | 5k | 1 min 17 sec | 1 min 24 sec | Success |
| 3 | 10k | 2 min 32 sec | 2 min 40 sec | Success |
| 4 | 80k | 19 min 54 sec | 21 min 44 sec | Success |
Comparison
Test | Number of records in File | Duration: Poppy [non-ECS] | Duration: Quesnelia [non-ECS] Set #1 | Delta, sec | %, Delta/Poppy Duration |
|---|---|---|---|---|---|
| 1 | 1k | 32 sec | 19 sec | 13 sec | 40.63% |
| 2 | 5k | 2 min 14 sec | 1 min 17 sec | 57 sec | 42.54% |
| 3 | 10k | 4 min 35 sec | 2 min 32 sec | 2 min 3 sec | 44.73% |
| 4 | 80k | 36 min 25 sec | 19 min 54 sec | 16 min 31 sec | 45.35% |
Compared with results in previous test report: Data Import Create MARC holdings records [non-ECS] [Poppy]
...
CPU (User) usage by broker
RDS CPU Utilization
...
Open Search
CPU utilization master node
CPU utilization data node
Minimum free storage space (GiB)
Maximuim memory utilization data node
Indexing rate
RDS CPU Utilization
For all tests - 95% except of DI Holdings with 1k file - 25%.
DB Connections
DB connections - 860.
DB Load
Set #1
Set #2
SQL queries
...
| Name | Memory GIB | vCPUs | Engine version |
|---|---|---|---|
db.r6g.xlarge | 32 GB | 4 vCPUs | 16.1 |
- MSK tenant
- 2 m5.2xlarge brokers in 2 zones
- Apache Kafka version 2.8.0
- EBS storage volume per broker 300 GiB
- auto.create.topics.enable=true
- log.retention.minutes=480
- default.replication.factor=2
- Open Search ptf-test
- version OpenSearch_2_7_R20240502
- Data nodes
- Instance type - r6g.2xlarge.search
- Number of nodes - 4
- Storage type - EBS
- EBS volume size (GiB) - 500
- Dedicated master nodes
- Instance type - r6g.large.search
- Number of nodes - 3
- DB records
- fs09000000
- instances Instances - 25901331
- items Items - 27074913
- holdings Holdings - 25871735
- fs07000001
- nstances Instances - 10100620
- items Items - 1484850
- holdings Holdings - 10522266
- fs07000002
- nstances Instances - 1161275
- items Items - 1153548
- holdings Holdings - 1153548
- fs09000000
| Expand | |||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| |||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
|
...