...
Retest after this issue will be fixed
Jira Legacy |
---|
server | System Jira |
---|
serverId | 01505d01-b853-3c2e-90f1-ee9b165564fc |
---|
key | MODDATAIMP-1064 |
---|
|
Test Runs
Important! Before each test, data-import MRC files were prepared using python script ensure creation of MARC Authority records.
...
DB connections were changing values from 550 connection to 870
...
Database loadÂ
...
Top SQL
...
Open Search metrics
Master node CPU utilization
CPU utilization percentage for the master node reached maximum 35%
...
Maximum CPU utilization percentage for all data nodes.
...
Maximum memory usage percentage for all data nodes.
...
Amazon Managed Streaming
...
...
Test 2. DI Central tenant 1k-5K-10K-25K-50K + CI/CO 5VU
...
Expand |
---|
title | 0.22 WITH cte AS (SELECT count(*) AS total_count FROM cs00000int_mod_source_recor |
---|
|
Code Block |
---|
WITH cte AS (SELECT count(*) AS total_count FROM cs00000int_mod_source_record_manager.job_execution WHERE subordination_type <> 'PARENT_MULTIPLE' AND TRUE AND status IN ('COMMITTED', 'ERROR', 'CANCELLED') AND job_profile_id NOT IN ('d0ebb7b0-2f0f-11eb-adc1-0242ac120002', '91f9b8d6-d80e-4727-9783-73fb53e3c786') AND file_name NOT IN ('No file name') AND subordination_type NOT IN ('COMPOSITE_PARENT') AND NOT job_profile_hidden AND NOT is_deleted) SELECT j.*, cte.*, p.total_records_count total, p.succeeded_records_count + p.error_records_count currently_processed, (select jsonb_agg(x) composite_data from (select status, count(1) cnt, sum(p1.total_records_count) total_records_count, sum(p1.succeeded_records_count + p1.error_records_count) currently_processed FROM cs00000int_mod_source_record_manager.job_execution j1 LEFT JOIN cs00000int_mod_source_record_manager.job_execution_progress p1 ON j1.id = p1.job_execution_id where j1.parent_job_id = j.id and j1.id != j1.parent_job_id and j1.subordination_type = 'COMPOSITE_CHILD' group by status) x) composite_data FROM cs00000int_mod_source_record_manager.job_execution j LEFT JOIN cs00000int_mod_source_record_manager.job_execution_progress p ON j.id = p.job_execution_id LEFT JOIN cte ON true WHERE subordination_type <> 'PARENT_MULTIPLE' AND TRUE AND status IN ('COMMITTED', 'ERROR', 'CANCELLED') AND job_profile_id NOT IN ('d0ebb7b0-2f0f-11eb-adc1-0242ac120002', '91f9b8d6-d80e-4727-9783-73fb53e3c786') AND file_name NOT IN ('No file name') AND subordination_type NOT IN ('COMPOSITE_PARENT') AND NOT job_profile_hidden AND NOT is_deleted ORDER BY completed_date desc LIMIT $1 OFFSET $2 |
|
Image Added
Maximum CPU utilization percentage for all data nodes.
...
Maximum memory usage percentage for all data nodes.
...
Amazon Managed Streaming
...
Test 3. Multitenant DI 50K-22K-10K-5K-1K.
...
Expand |
---|
|

Code Block |
---|
| select distinct on ("records_lb"."id") "records_lb"."id", "records_lb"."snapshot_id", "records_lb"."matched_id", "records_lb"."generation", "records_lb"."record_type", "records_lb"."external_id", "records_lb"."state", "records_lb"."leader_record_status", "records_lb"."order", "records_lb"."suppress_discovery", "records_lb"."created_by_user_id", "records_lb"."created_date", "records_lb"."updated_by_user_id", "records_lb"."updated_date", "records_lb"."external_hrid", "marc_records_lb"."content" as "parsed_record_content", "raw_records_lb"."content" as "raw_record_content", "error_records_lb"."content" as "error_record_content", "error_records_lb"."description" from "records_lb" left outer join "marc_records_lb" on "records_lb"."id" = "marc_records_lb"."id" left outer join "raw_records_lb" on "records_lb"."id" = "raw_records_lb"."id" left outer join "error_records_lb" on "records_lb"."id" = "error_records_lb"."id" join "marc_indexers_010" on "records_lb"."id" = "marc_indexers_010"."marc_id" join "marc_records_tracking" on ("marc_records_tracking"."marc_id" = "marc_indexers_010"."marc_id" and "marc_records_tracking"."version" = "marc_indexers_010"."version") where ("records_lb"."record_type" = $1::"record_type" and "records_lb"."state" = $2::"record_state" and ("marc_indexers_010"."value" in ('n 027310230486 ') and "marc_indexers_010"."ind1" LIKE '#' and "marc_indexers_010"."ind2" LIKE '#' and "marc_indexers_010"."subfield_no" = 'a')) offset $3 rows fetch next $4 rows only |
|
Open Search metrics
Master node CPU utilization
CPU utilization percentage for the master node reached maximum 47%
...
Maximum CPU utilization percentage for all data nodes.
...
Maximum memory usage percentage for all data nodes.
...
Amazon Managed Streaming
...
Image Added
Appendix
Infrastructure
PTF - environment Quesnelia (QCON)
...