Table of Contents | ||
---|---|---|
|
Overview In progress
- This document contains the results of testing Check-in/Check-out and Data Import for MARC Bibliographic records on the Ramsons[ECS] release environment.
Jira Legacy | ||||||||||
---|---|---|---|---|---|---|---|---|---|---|
|
Summary
- Data Import with Check In/Check Out tests finished successfully with PTF - Create 2 and PTF - Updates Success - 2 job profiles with files 5K, 10K, 25K, 50K, 100K records.
- Comparing with previous testing results Quesnelia and Ramsons releases
Test Runs
...
Test Results
This table contains durations for Data Import with Check-in/Check-out.
...
DI Duration
Ramsons
(hh:mm:ss)
...
Check In, Check Out Response time (8 users)
Ramsons
...
DI MARC Bib Create
(PTF - Create 2)
...
DI MARC Bib Update
(PTF - Updates Success - 2)
...
Check-in/Check-out without Data Import
...
Comparison
This table contains DI durations with CICO comparison between Quesnelia and Ramsons releases.
...
DI Delta
Quesnelia/Ramsons
with CI/CO
(hh:mm:ss)
...
0:00:32
+11%
...
0:18:24
+40%
...
0:04:01
+118%
...
Detailed CICO response time comparison without DI
...
Table of Contents | ||
---|---|---|
|
Overview
- This document contains the results of testing Check-in/Check-out and Data Import for MARC Bibliographic records on the Ramsons[NON-ECS] release environment.
Jira Legacy | ||||||||||
---|---|---|---|---|---|---|---|---|---|---|
|
Summary
- Data Import with Check In/Check Out tests finished successfully with PTF - Create 2 and PTF - Updates Success - 2 job profiles with files 5K, 10K, 25K, 50K, 100K records.
- Instance type was updated from m6i.2xlarge to r7g.2xlarge
- Comparing with previous testing results Quesnelia and Ramsons releases
- Check-in duration degraded by 20% and Check-out duration degraded by 40% on average without Data Import
- Data Import Create jobs duration with CI/CO degraded by 15%-40%, depending on file size
- Data Import Update jobs durations with CI/CO degraded by two times.
- Check-in duration degraded by 20% and Check-out duration degraded by 40% on average for Data Import Create
- Check-in duration degraded by 40% and Check-out duration degraded by 45% on average for Data Import Update
- During the test, a slow SQL query was detected with a latency of 28,32 seconds.
Test Runs
Test № | Scenario | Test Conditions | Results |
---|---|---|---|
1 | DI MARC Bib Create | 5K, 10K, 25K, 50K, 100K consequentially | Completed |
CICO | 8 users | ||
2 | DI MARC Bib Update | 5K, 10K, 25K, 50K, 100K consequentially |
|
CICO | 8 users |
Test Results
This table contains durations for Data Import with Check-in/Check-out.
Profile | MARC File | DI Duration Ramsons (hh:mm:ss) | Check In, Check Out Response time (8 users) Ramsons | |
---|---|---|---|---|
CI Average ms | CO Average ms | |||
DI MARC Bib Create (PTF - Create 2) | 5K.mrc | 0:02:53 | 837 | 1.628 |
10K.mrc | 0:05:35 | 809 | 1.460 | |
25K.mrc | 0:15:13 | 895 | 1.545 | |
50K.mrc | 0:27:17 | 901 | 1.495 | |
100K.mrc | 1:04:31 | 896 | 1.459 | |
DI MARC Bib Update (PTF - Updates Success - 2) | 5K.mrc | 0:07:25 | 992 | 1.767 |
10K.mrc | 0:12:52 | 1.100 | 1.903 | |
25K.mrc | 0:37:02 | 1.225 | 1.957 | |
50K.mrc | 1:13:16 | 1.299 | 1.957 | |
100K.mrc | 2:32:25 | 1.297 | 2.027 |
Check-in/Check-out without Data Import
Scenario | Load level | Request | Response time, ms Quesnelia | |
---|---|---|---|---|
95 perc | average | |||
Circulation Check-in/Check-out (without Data import) | 8 users | Check-in | 767 | 625 |
Check-out |
Test №1-2
Resource utilization
...
title | Resource utilization table |
---|
...
Response times
...
title | Response times |
---|
Service CPU Utilization
Service Memory Utilization
DB CPU Utilization
DB Connections
Kafka metrics
OpenSearch Data Nodes metrics
DB load
Top SQL-queries
Top applications
...
insert into "marc_records_lb" ("id", "content") values (cast($1 as uuid), cast($2 as jsonb)) on conflict ("id") do update set "content" = cast($3 as jsonb)
...
WITH cte AS (SELECT id,
name,
name_type_id,
authority_id,
last_updated_date
FROM fs09000000_mod_search.contributor
WHERE last_updated_date > $1
ORDER BY last_updated_date
)
SELECT c.id,
c.name,
c.name_type_id,
c.authority_id,
c.last_updated_date,
json_agg(
CASE
WHEN sub.instance_count IS NULL THEN NULL
ELSE json_build_object(
'count', sub.instance_count,
'typeId', sub.type_ids,
'shared', sub.shared,
'tenantId', sub.tenant_id
)
END
) AS instances
FROM cte c
LEFT JOIN
(SELECT cte.id,
ins.tenant_id,
ins.shared,
array_agg(DISTINCT ins.type_id) FILTER (WHERE ins.type_id <> '') AS type_ids,
count(DISTINCT ins.instance_id) AS instance_count
FROM fs09000000_mod_search.instance_contributor ins
INNER JOIN cte
ON ins.contributor_id = cte.id
GROUP BY cte.id,
ins.tenant_id,
ins.shared) sub ON c.id = sub.id
GROUP BY c.id,
c.name,
c.name_type_id,
c.authority_id,
c.last_updated_date
ORDER BY last_updated_date ASC
Appendix
Infrastructure
...
- rcp1 5 r7g.2xlarge EC2 instances located in US East (N. Virginia)us-east-1
- 1 instance of db.r6g.xlarge database instance: Writer instance
- MSK fse-test
- 4 kafka.m7g.xlarge brokers in 2 zones (2 brokers per zone)
Apache Kafka version 3.7.x, metadata mode - KRaft
EBS storage volume per broker 300 GiB
- auto.create.topics.enable=true
- log.retention.minutes=480
- default.replication.factor=2
- revision - 26
r6g.2xlarge.search 4 data nodes
r6g.large.search 3 dedicated master nodes
DB tables records size:
...
Methodology/Approach
DI tests scenario (DI MARC Bib Create and Update) were started on Ramsons (rcp1) env with file splitting features enabled on a NON-ECS environment.
- To run CI/CO - Ubuntu AWS instance was used as load generator
- DI tests were started from UI
Test runs:
...
1347 | 1138 |
Comparison
This table contains DI durations with CICO comparison between Quesnelia and Ramsons releases.
Profile | MARC File | DI Duration | DI Delta with CI/CO | Check In, Check Out Response time (8 users) | Check In, Check Out Response time (8 users) | Delta, % | ||||||
without CI/CO | with CI/CO | Quesnelia | Ramsons | Quesnelia/Ramsons | Quesnelia/Ramsons | |||||||
Quesnelia | Ramsons | Quesnelia | Ramsons | CI Average sec | CO Average sec | CI Average sec | CO Average sec | CI | CO | |||
DI MARC Bib Create (PTF - Create 2) | 5K.mrc | 00:03:20 | 00:02:32 | 0:02:53 | 0:00:21 +14% | 0.645 | 0.901 | 0.837 | 1.628 | +23% | +45% | |
10K.mrc | 00:06:00 | 00:05:33 | 00:05:03 | 0:05:35 | 0:00:32 | 0.628 | 0.922 | 0.809 | 1.460 | +22% | +37% | |
25K.mrc | 00:13:41 | 0:14:16 | 00:11:58 | 0:15:13 | 0:03:15 +27% | 0.639 | 0.96 | 0.895 | 1.545 | +29% | +38% | |
50K.mrc | 00:21:59 | 0:28:23 | 00:23:29 | 0:27:17 | 0:03:48 +16% | 0.678 | 1.003 | 0.901 | 1.495 | +25% | +33% | |
100K.mrc | 00:40:16 | 0:49:03 | 00:46:07 | 1:04:31 | 0:18:24 | 0.686 | 0.998 | 0.896 | 1.459 | +23% | +32% | |
DI MARC Bib Update (PTF - Updates Success - 2) | 5K.mrc | 00:07:10 | 00:03:24 | 0:07:25 | 0:04:01 | 0.628 | 0.975 | 0.992 | 1.767 | +37% | +45% | |
10K.mrc | 00:10:27 | 0:15:00 | 00:06:29 | 0:12:52 | 0:04:01 +98% | 0.664 | 1.018 | 1.100 | 1.903 | +40% | +47% | |
25K.mrc | 00:23:16 | 0:34:41 | 00:16:15 | 0:37:02 | 0:20:47 +128% | 0.717 | 1.062 | 1.225 | 1.957 | +41% | +46% | |
50K.mrc | 00:40:52 | 1:10:01 | 00:33:33 | 1:13:16 | 0:39:43 +118% | 0.721 | 1.071 | 1.299 | 1.957 | +44% | +45% | |
100K.mrc | 01:02:00 | 2:27:05 | 01:10:14 | 2:32:25 | 1:22:11 +117% | 0.739 | 1.081 | 1.297 | 2.027 | +43% | +47% |
Detailed CICO response time comparison without DI
Scenario | Load level | Request | Response time, ms Ramsons | Response time, ms Quesnelia | Delta, % | ||
---|---|---|---|---|---|---|---|
95 perc | average | 95 perc | average | Ramsons/Quesnelia average | |||
Circulation Check-in/Check-out (without Data import) | 8 users | Check-in | 767 | 625 | 609 | 521 | +19.96% |
Check-out | 1.347 | 1.138 | 1.070 | 803 | +41.72% |
Test №1-2
Resource utilization
Expand | ||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| ||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
|
Response times
Expand | ||
---|---|---|
| ||
Service CPU Utilization
Service Memory Utilization
DB CPU Utilization
DB Connections
Kafka metrics
OpenSearch Data Nodes metrics
DB load
Top SQL-queries
Top applications
# | TOP SQL statement | ||||||
---|---|---|---|---|---|---|---|
1 |
| ||||||
# | TOP SLOW
| ||||||
1 |
|
Appendix
Infrastructure
PTF -environment rcp1 |
---|
|
DB tables records size:
Tennent | instance count | holdings count | item count | authority count |
---|---|---|---|---|
fs09000000 | 30.587.260 | 30.646.075 | 31.734.818 | 16.535.572 |
Methodology/Approach
DI tests scenario (DI MARC Bib Create and Update) were started on Ramsons (rcp1) env with file splitting features enabled on a NON-ECS environment.
- To run CI/CO - Ubuntu AWS instance was used as load generator
- DI tests were started from UI
Test runs:
- Test 1: Manually tested 5K, 10K, 25K, 50K, 100K consequentially records files, DI (DI MARC Bib Create ) started on Main tenant (fs09000000), and CICO with 8 users on background.
- Test 2: Manually tested 5K, 10K, 25K, 50K, 100K consequentially records files, DI (DI MARC Bib Update) started on Main tenant (fs09000000), and CICO with 8 users on background.
Expand | ||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| ||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
Cluster Resources - rcp1-pvt (Wed Jan 22 12:28:18 UTC 2025)
|
Test artifacts:
View file | ||||
---|---|---|---|---|
|