...

• Through a series of experiments involving different placement strategies, instance types, and total instance counts, we found that the performance remained consistent when using these two configuration: 
  • six c7g.large instances (three dedicated to the okapi service and three allocated to mod-courses, mod-sender, mod-tasks-list, mod-gobi, edge-dematic, mod-erm-usage, mod-eusage-reports, mod-notify, and mod-data-import services) alongside ten r7g.xlarge instances for all other services, with the CPU parameter set to 2 for all services.
  • three c7g.large instances dedicated to the okapi service alongside five r7g.2xlarge instances for all other services, with the CPU parameter set to 2 for all services.
• Notably, this both environment configuration offers a 40% cost reduction compared to the existing setup, making it a more economical option without compromising on performance.Tests had 100% errors count for AIE_TC: Create
• Configurations with three c7g.large instances for the okapi service and five r7g.2xlarge instances for all other services show the best performance across all experiments. So it will be more tests in this ticket.
• Tests had 100% errors count for AIE_TC: Create Invoices, AIE_TC: Invoices Approve, AIE_TC: Paying Invoices, TC: Receiving-an-Order-Line, Unreceiving-a-Piece and Unreceiving-a-Piece Workflows because data was not regenerated.  

Test Runs

Test Results

This table contains durations for all Workflows. 

Comparison

This graph shows the durations of all workflows compared between the best test result from Test №8, MCPT Basile configuration with 14 instances m6g.2xlarge and result from mcpt tunning report.

Average Case workflows: Part 1.

 Image RemovedImage Added

Average Case workflows: Part 2.

Image RemovedImage Added

Average Case workflows: Part 3.

Image RemovedImage Added

...

Service CPU Utilization

Here we can see that okapi modules used 42k% CPU power parameter CPU=2 for module.

...

Here we can't see any sign of memory leaks on every module. Memory shows stable trend.

Kafka metrics

DB CPU Utilization

DB CPU was 99% average with ERW: Exporting Receiving Information.

...

Here we can't see any sign of memory leaks on every module. Memory shows stable trend.

Kafka metrics

DB CPU Utilization

DB CPU was 99% average with ERW: Exporting Receiving Information

...

Here we can't see any sign of memory leaks on every module. Memory shows stable trend.

Kafka metrics

DB CPU Utilization

DB CPU was 99% maximum.

...

Service CPU Utilization

Here we can see that okapi used 43k% of the CPU power of parameter CPU=2.

...

Here we can't see any sign of memory leaks on every module. Memory shows stable trend.

Kafka metrics

DB CPU Utilization

DB CPU was 99%.

...

Service CPU Utilization

Here we can see that okapi used 46k% of the CPU power of parameter CPU=2.

...

Here we can't see any sign of memory leaks on every module. Memory shows stable trend.

Kafka metrics

DB CPU Utilization

DB CPU was 99%.

...

Service CPU Utilization

Here we can see that okapi used 46k% of the CPU power of parameter CPU=2.

...

Here we can't see any sign of memory leaks on every module. Memory shows stable trend.

Kafka metrics

DB CPU Utilization

DB CPU was 99%.

...

...

...

...

...

...

Infrastructure

PTF - Baseline MCPT environment configurationenvironment configuration (was changed during testing)

• 14 m6g.2xlarge EC2 instances located in US East (N. Virginia)us-east-1

• 1 database  instance, writer

  
  

  Name	Memory GIB	vCPUs
  db.r6g.4xlarge	128 GiB	16 vCPUs

  
  

• Open Search ptf-test 
  • Data nodes
    • Instance type - r6g.2xlarge.search
    • Number of nodes - 4
    • Version: OpenSearch_2_7_R20240502
  • Dedicated master nodes
    • Instance type - r6g.large.search
    • Number of nodes - 3
• MSK ptf-KRaft-mode
  
  • 2 brokers, kafka.m7g.xlarge brokers in 2 zones

  • Apache Kafka version 3.7.x 

  • EBS storage volume per broker 300 GiB

  • auto.create.topics.enable=true
  • log.retention.minutes=480
  • default.replication.factor=3

...

Baseline MCPT Environment configuration according to tunning environment from previous report:  task count: 4 for services: mod-permissions, mod-search, mod-patron, mod-inventory, mod-inventory-storage, mod-circulation, mod-circulation-storage, mod-order, mod-order-storage, mod-invoice, mod-invoice-storage, for mod-users and mod-authtoken task count 6. Parameter srs.marcIndexers.delete.interval.seconds=86400 for mod-source-record-storage. Instance type: m6g.2xlarge. Instances count: 14. Database r6g.4xlarge, Amazon OpenSearch Service  ptf-test: r6g.2хlarge.search (4 nodes).

...