Table of Contents Table of Contents outline true
Overview
- In this report, PTF investigates the impact of setting CPU allocations to 0 units across all tasks within an AWS ECS cluster. The purpose of this study is to determine whether removing CPU constraints reveals the actual CPU usage of the tasks and to assess how this adjustment affects overall performance. By comparing key workflows across different environments, we aim to identify any potential changes in efficiency, throughput, or resource utilization that may result from setting CPU = 0. The findings from these tests will help inform best practices for resource allocation and performance optimization within ECS clusters.
...
Service CPU Utilization
Here we can see that nginx-okapi modules used 20% CPU for Test-2 (high load case).
...
Here we can see that Memory exceed 113% of the unit power mod-data-export-b modules for Test-2 (high load case).
Kafka metrics
DB CPU Utilization
DB CPU was 99% average with ERW: Exporting Receiving Information
...
Here we can't see any sign of memory leaks on every module. Memory shows stable trend.
Kafka metrics
DB CPU Utilization
DB CPU was 99% average with ERW: Exporting Receiving Information
...
Here we can't see any sign of memory leaks on every module. Memory shows stable trend.
Kafka metrics
DB CPU Utilization
DB CPU was 64% maximum.
...
Service CPU Utilization
Here we can see that mod-source-record-storage-b used 72000% of the CPU power of parameter CPU=2.
...
Here we can't see any sign of memory leaks on every module. Memory shows stable trend.
Kafka metrics
DB CPU Utilization
DB CPU was 99%.
...
Here we can't see any sign of memory leaks on every module. Memory shows stable trend.
Kafka metrics
DB CPU Utilization
DB CPU was 57%.
...
Here we can't see any sign of memory leaks on every module. Memory shows stable trend.
Kafka metrics
DB CPU Utilization
DB CPU was 98%.
...
Max number of DB connections was 971.
Test №9
The Baseline MCPT QCP1 Environment configuration was applied, the instance type was changed to r6g.xlarge number of instances, the was changed to 14, placement strategy was updated to "one task per host", and CPU=20 was set for all servicesmodules, Check In\Check Out with 20 users for one tenant on 30 minutes test was run.
Instance CPU Utilization
Service CPU Utilization
Here we can see that okapi used 44000% 15% of the absolute CPU power of the unit CPU power.container instance.
Service Memory Utilization
Here we can't see any sign of memory leaks on every module. Memory shows stable trend.
Inctanse CPU Utilization
Kafka metrics
DB CPU Utilization
DB CPU was 98%.
DB Connections
Max number of DB connections was 5150.
DB load
Top SQL-queries
Kafka metrics
DB CPU Utilization
DB CPU was 98%.
DB Connections
Max number of DB connections was 5150.
Test №10
The Baseline MCPT QCP1 Environment configuration was applied, the instance type was changed to x2gd.large, the number of instances was changed to 14, placement strategy was updated to "one task per host", and CPU=2 was set for all servicesand CPU=0 was set for all modules, Data Import - Create with 25k and 100k records files tests were run.
Instance CPU Utilization
Service CPU Utilization
Here we can see that okapi used 38% of the unit CPU power.
Service Memory Utilization
Here we can't see any sign of memory leaks on every module. Memory shows stable trend.
Kafka metrics
DB CPU Utilization
DB CPU was maximum 53%92%.
DB Connections
Max number of DB connections was 3842.
DB load
Top SQL-queries
921.
Appendix
Infrastructure
PTF - Baseline MCPT environment configuration
- 14 m6g.2xlarge EC2 instances located in US East (N. Virginia)us-east-1
1 database instance, writer
Name Memory GIB vCPUs db.r6g.4xlarge
128 GiB 16 vCPUs - Open Search ptf-test
- Data nodes
- Instance type - r6g.2xlarge.search
- Number of nodes - 4
- Version: OpenSearch_2_7_R20240502
- Dedicated master nodes
- Instance type - r6g.large.search
- Number of nodes - 3
- Data nodes
- MSK ptf-KRaft-mode
- 2 brokers, kafka.m7g.xlarge brokers in 2 zones
Apache Kafka version 3.7.x
EBS storage volume per broker 300 GiB
- auto.create.topics.enable=true
- log.retention.minutes=480
- default.replication.factor=3
...
Baseline MCPT Environment configuration according to tunning environment from previous report: task count: 4 for services: mod-permissions, mod-search, mod-patron, mod-inventory, mod-inventory-storage, mod-circulation, mod-circulation-storage, mod-order, mod-order-storage, mod-invoice, mod-invoice-storage, for mod-users and mod-authtoken task count 6. Parameter srs.marcIndexers.delete.interval.seconds=86400 for mod-source-record-storage. Instance type: m6g.2xlarge. Instances count: 14. Database r6g.4xlarge, Amazon OpenSearch Service ptf-test: r6g.2хlarge.search (4 nodes).
...