Versions Compared

Old Version 25

changes.mady.by.user Stanislav Nehrii

Saved on

compared with

New Version 26

changes.mady.by.user Stanislav Nehrii

Saved on

Key

  • This line was added.
  • This line was removed.
  • Formatting was changed.

Table of Contents
outlinetrue

...

  • Through a series of experiments involving different placement strategies, instance types, and total instance counts, we found that the performance remained consistent when using this configurationthese configurations: 
    • three c7g.large instances dedicated to the okapi service alongside five r7g.2xlarge instances for all other services, with the CPU parameter set to 2 for all services.
    • five r7g.2xlarge instances for all services, with the CPU parameter set to 2 for all services.
  • Optimized environment configurations offers a 20-40% cost reduction compared to the existing setup, making it a more economical option without compromising on performance.
  • Configurations with three c7g.large instances for the okapi service and five r7g.2xlarge instances for all other services show the best performance across all experiments.
  • In fact, some workflows show better performance with this new setup than correct infrastructures.
  • The CPU utilization on EC2 level better now - around 30-60%, previously it was under 20%.

...

ClusterInstance TypeCost per Month
(USD)		Number of InstancesTotal Cost per Cluster
(USD)
QCP1m6g.2xlarge$221.7610$2,217.60
MCPTm6g.2xlarge$221.7614$3,104.64
Optimized Infrastructure
Two Auto Scaling Groups		c7g.large$52.203$1,698.84
r7g.2xlarge$308.455
Optimized Infrastructure
One Auto Scaling Groups		r7g.2xlarge$308.455$1,542.25

...

Test #DescriptionStatus
Test 1Instance type: m6g.2xlarge. Instances count: 10. Completed
Test 2Instance type: m6g.2xlarge. Instances count: 10 (Repeat Test 1). Completed
Test 3Used two autoscaling groups, 1st with 3 Instance Type: c7g.large for okapi service and 5 Instance Type: m6g.2xlarge for others services.Completed
Test 4Used two autoscaling groups, 1st with 3 Instance Type: c7g.large for okapi service and 5 Instance Type: m6g.2xlarge for others services (Repeat Test 3).Completed
Test 5CPU=2 was set for all modules, used two autoscaling groups, 1st with 3 Instance Type: c7g.large, 3 of them for okapi service and 5 Instance Type: r7g.xlarge for others modules.Completed
Test 6CPU=2 was set for all modules, used two autoscaling groups, 1st with 3 Instance Type: c7g.large, 3 of them for okapi service and 5 Instance Type: r7g.xlarge for others modules (Repeat Test 5).Completed
Test 7CPU=2 was set for all modules except CPU=2048 for mod-search, used two autoscaling groups, 1st with 3 Instance Type: c7g.large, 3 of them for okapi service and 5 Instance Type: r7g.xlarge for others modules.Completed
Test 8

CPU=2 was set for all modules, used ONE autoscaling group with 5 Instance Type:  c7g.large for all services.


Test 9CPU=2 was set for all modules, used ONE autoscaling group with 5 Instance Type:  c7g.large for all services (Repeat Test 8).
Test 10CPU=2 was set for all modules except CPU=2048 for mod-search, used ONE autoscaling group with 5 Instance Type:  c7g.large for all services.
Test 11

CPU=2 was set for all modules except CPU=2048 for mod-search, used ONE autoscaling group with 5 Instance Type:  c7g.large for all services (Repeat Test 10).


Test 12CPU=2 was set for all modules except CPU=2048 for mod-search, used ONE autoscaling group with 5 Instance Type:  c7g.large for all services (Repeat Test 11).

...

This graph shows the durations of all workflows compared between the best test results.

Part 1.

 Image RemovedImage Added

Part 2.

...

Image Added


Part 3

Image Added



Test №1

Introduction: Test 1: The Baseline QCP1 Environment configuration was applied, qcp1_MiniMaster.jmx script was run. 

Objective: The objective of test was to evaluate the performance of the MCPT environment by applying the baseline configuration.

...

Service CPU Utilization

Here we can see that mod-inventory-b modules used 73% CPU.

Service Memory Utilization

Here we can't see any sign of memory leaks on every module. Memory shows stable trend.


Kafka metrics


OpenSearch metrics


DB CPU Utilization

DB CPU was 97% average.

...

Service CPU Utilization

Here we can see that mod-inventory-b modules used 102% CPU in maximum.

...

Here we can't see any sign of memory leaks on every module. Memory shows stable trend.


Kafka metrics


OpenSearch metrics

DB CPU Utilization

DB CPU was 95% average 

...

Service CPU Utilization

Here we can see that mod-inventory-b used 78k% of the CPU.

Service Memory Utilization

Here we can't see any sign of memory leaks on every module. Memory shows stable trend.


Kafka metrics


OpenSearch metrics

DB CPU Utilization

DB CPU was 97% maximum.

...

Service CPU Utilization

Here we can see that mod-data-export-b used 82k% of the CPU power of parameter CPU=2.

...

Here we can't see any sign of memory leaks on every module. Memory shows stable trend.

Kafka metrics



OpenSearch metrics

DB CPU Utilization

DB CPU was 97%.

...

Service CPU Utilization

Here we can see that mod-data-export-b used 36k% AVG of the CPU power of parameter CPU=2.

...

Here we can't see any sign of memory leaks on every module. Memory shows stable trend.

Kafka metrics



OpenSearch metrics


DB CPU Utilization

DB CPU was 94%.

...

Service CPU Utilization

Here we can see that mod-data-export-b used 116k% MAX of the CPU power of parameter CPU=2.

...

Here we can't see any sign of memory leaks on every module. Memory shows stable trend.

Kafka metrics



OpenSearch metrics


DB CPU Utilization

DB CPU was 94%.

...

Service CPU Utilization

Here we can see that mod-data-export-b used 36k% AVG of the CPU power of parameter CPU=2.

...

Here we can't see any sign of memory leaks on every module. Memory shows stable trend.

Kafka metrics



OpenSearch metrics


DB CPU Utilization

DB CPU was 94%.

...

Service CPU Utilization

Here we can see that mod-data-export-b used 36k% AVG of the CPU power of parameter CPU=2.

...

Here we can't see any sign of memory leaks on every module. Memory shows stable trend.

Kafka metrics



OpenSearch metrics


DB CPU Utilization

DB CPU was 94%.

...

PTF - Baseline QCP1 environment configuration (was changed during testing)

  • 10 m6g.2xlarge EC2 instances located in US East (N. Virginia)us-east-1

  • 1 database  instance, writer


    NameMemory GIBvCPUs

    db.r6g.xlarge

    		32 GB4 vCPUs


  • Open Search ptf-test 
    • Data nodes
      • Instance type - r6g.2xlarge.search
      • Number of nodes - 4
      • Version: OpenSearch_2_7_R20240502
    • Dedicated master nodes
      • Instance type - r6g.large.search
      • Number of nodes - 3
  • MSK fse-tenant
    • 2 brokers, kafka.m7g.xlarge brokers in 2 zones

    • Apache Kafka version 3.7.x 

    • EBS storage volume per broker 300 GiB

    • auto.create.topics.enable=true
    • log.retention.minutes=480
    • default.replication.factor=3

...

Baseline QCP1 Environment configuration: Parameter srs.marcIndexers.delete.interval.seconds=86400 for mod-source-record-storage,  number of tasks to launch for service mod-marc-migrations-b was set zero. Instance type: m6g.2xlarge. Instances count: 10. Database db.r6g.xlarge, Amazon OpenSearch Service  ptf-test: r6g.2хlarge.search (4 nodes).

...