outline

Table of Contents

outline	true

Overview

In this report, PTF investigates the impact of setting CPU allocations to 0 units across all tasks within an AWS ECS cluster. The purpose of this study is to determine whether removing CPU constraints reveals the actual CPU usage of the tasks and to assess how this adjustment affects overall performance. By comparing key workflows across different environments, we aim to identify any potential changes in efficiency, throughput, or resource utilization that may result from setting CPU = 0. The findings from these tests will help inform best practices for resource allocation and performance optimization within ECS clusters.

Jira Legacy

server	System Jira
columnIds	issuekey,summary,issuetype,created,updated,duedate,assignee,reporter,priority,status,resolution
columns	key,summary,type,created,updated,due,assignee,reporter,priority,status,resolution
serverId	01505d01-b853-3c2e-90f1-ee9b165564fc
key	PERF-959

Summary

...

During the tests Parameter CPU=0 improved performance for a several workflows, so this point we are going to investigate more in this ticket.No memory leaks, memory consumption like MSF_TC: mod search by auth query, MSF_TC: mod search by contributors, MSF_TC: mod search by title query and ULR_TC: Users loan Renewal Transaction.
No memory leaks, memory consumption was stable during all of the tests.
Tests had 100% errors count for AIE_TC: Create Invoices, AIE_TC: Invoices Approve, AIE_TC: Paying Invoices, TC: Receiving-an-Order-Line, Unreceiving-a-Piece and Unreceiving-a-Piece Workflows because data was not regenerated.

Test Runs

...

This table contains durations for all Workflows.

...

Test # Description Status
Test 1 The Baseline MCPT Environment configuration was applied, and CPU=0 was set for all modules,  Fixed Load (average case) MOBIUS test was run. Completed
Test 2 The Baseline MCPT Environment configuration was applied, and CPU=0 was set for all modules,  Fixed Load (high load case) MOBIUS test was run. Completed
Test 3 The Baseline MCPT Environment configuration was applied, and CPU=0 was set for all modules,  Fixed Load (average case)
Average
(milliseconds)ErrorsAverage
(milliseconds)ErrorsAverage
(milliseconds)ErrorsAIE_TC: Create Invoices8350100%13458100%7421100%AIE_TC: Invoices Approve3180100%5251100%2960100%AIE_TC: Paying Invoices3289100%4919100%3060100%CICO_TC_Check-In Controller23230%33227%22280%CICO_TC_Check-Out Controller41540%570010%38960%CSI_TC:Share local instance1300819%1478114%1296920%DE_Exporting MARC Bib records custom workflow529830%37991295%787590%DE_Exporting MARC Bib records workflow427850%40202698%739570%EVA_TC: View Account8033%9623%7001%ILR_TC: Create ILR15270%22154%14510%MSF_TC: mod search by auth query6720%18307%30964%MSF_TC: mod search by boolean query1650%4852%7371%MSF_TC: mod search by contributors3980%10633%16642%MSF_TC: mod search by filter query2860%7132%10702%MSF_TC: mod search by keyword query2840%6581%10061%MSF_TC: mod search by subject query4070%11121%15302%MSF_TC: mod search by title query10310%24491%29071%OPIH_/oai/records60420%4649100%35870%POO_TC: Add Order Lines553340%9907619%544400%POO_TC: Approve Order425670%7902212%421910%POO_TC Create Order319330%5646813%314060%RTAC_TC: edge-rtac41500%47260%40840%SDIC_Single Record Import (Create)1377719%2047513%1377320%SDIU_Single Record Import (Update)195490%3663916%195821%TC: Receiving-an-Order-Line45895100%83735100%45888100%Serials-Receiving-Workflow48584100%86692100%47338100%Unreceiving-a-Piece8080100%13729100%8013100%ULR_TC: Users loan Renewal Transaction31890%48186%29700%

Comparison

This graph shows the total durations of all workflows compared between tests.

Resource utilization for Test №1

Baseline MCPT Environment configuration

Image Removed

Service CPU Utilization

Here we can see that CPU did not exceed 95% of the unit CPU power for all of the modules.

Service Memory Utilization

Here we can't see any sign of memory leaks on every module. Memory shows stable trend.

Kafka metrics

Image Removed

DB CPU Utilization

DB CPU was 99% average with ERW: Exporting Receiving Information

DB Connections

Max number of DB connections was 5568 in maximum.

Resource utilization for Test №2

The Baseline MCPT Environment configuration was applied, and CPU=0 was set for all modules.

Service CPU Utilization

Here we can see that okapi used 13% of the absolute CPU power of the container instance.

Service Memory Utilization

Here we can't see any sign of memory leaks on every module. Memory shows stable trend.

Kafka metrics

DB CPU Utilization

DB CPU was 99% average with ERW: Exporting Receiving Information

DB Connections

Max number of DB connections was 5788.

DB load

Top SQL-queries

Resource utilization for Test №3

The Baseline MCPT Environment configuration was applied, the instance type was changed to x2gd.xlarge, the number of instances was changed to 6, and CPU=0 was set for all services.

Service CPU Utilization

Here we can see that mod-permissions used 20% of the absolute CPU power of the container instance.

Service Memory Utilization

Here we can't see any sign of memory leaks on every module. Memory shows stable trend.

Kafka metrics

DB CPU Utilization

DB CPU was 64% maximum.

DB Connections

Max number of DB connections was 2040.

Resource utilization for Test №4

The Baseline MCPT Environment configuration was applied, the instance type was changed to x2gd.xlarge, the number of instances was changed to 8, and CPU=0 was set for all services.

Service CPU Utilization

Here we can see that okapi used 20% of the absolute CPU power of the container instance.

Service Memory Utilization

Here we can't see any sign of memory leaks on every module. Memory shows stable trend.

Kafka metrics

DB CPU Utilization

DB CPU was 91%.

DB Connections

Max number of DB connections was 4840.

DB load

Top SQL-queries

Resource utilization for Test №5

The Baseline MCPT Environment configuration was applied, the instance type was changed to x2gd.large, the number of instances was changed to 10, and CPU=0 was set for all services.TT

Service CPU Utilization

Here we can see that okapi used 36% of the absolute CPU power of the container instance.

Service Memory Utilization

Here we can't see any sign of memory leaks on every module. Memory shows stable trend.

Kafka metrics

DB CPU Utilization

DB CPU was 42%.

DB Connections

Max number of DB connections was 3650.

DB load

Top SQL-queries

Resource utilization for Test №6

The Baseline MCPT Environment configuration was applied, the instance type was changed to r6g.xlarge, the number of instances was changed to 14 but 12 were used, and CPU=2 was set for all services.

Service CPU Utilization

Here we can see that okapi used 46000% CPU of unit power.

Service Memory Utilization

Here we can't see any sign of memory leaks on every module. Memory shows stable trend.

Inctanse CPU Utilization

Kafka metrics

DB CPU Utilization

DB CPU was 98%.

DB Connections

Max number of DB connections was 5150.

DB load

Top SQL-queries

Resource utilization for Test №7

The Baseline MCPT Environment configuration was applied, the instance type was changed to r6g.xlarge, the number of instances was changed to 14, placement strategy was updated to "one task per host", and CPU=2 was set for all services.

Service CPU Utilization

Here we can see that okapi used 44000% of the unit CPU power.

Service Memory Utilization

Here we can't see any sign of memory leaks on every module. Memory shows stable trend.

Inctanse CPU Utilization

Kafka metrics

DB CPU Utilization

DB CPU was 98%.

DB Connections

Max number of DB connections was 5150.

DB load

MOBIUS test was run (rerun Test 1).	Completed
Test 4	The Baseline MCPT Environment configuration was applied, and CPU=1 was set for all modules, Fixed Load (average case) MOBIUS test was run.	Completed
Test 5	The Baseline MCPT Environment configuration was applied, and CPU=2 was set for all modules, Check In\Check Out with 20 users for one tenant on 30 minutes test was run.	Completed
Test 6	The Baseline MCPT Environment configuration was applied, and CPU=2 was set for all modules, Data Import - Create with 25k and 100k records files tests were run.	Completed
Test 7	The Baseline QCP1 Environment configuration was applied, and CPU=2 was set for all modules, Check In\Check Out with 20 users for one tenant on 30 minutes test was run.	Completed
Test 8	The Baseline QCP1 Environment configuration was applied, and CPU=2 was set for all modules, Data Import - Create with 25k and 100k records files tests were run.	Completed
Test 9	The Baseline QCP1 Environment configuration was applied, and CPU=0 was set for all modules, Check In\Check Out with 20 users for one tenant on 30 minutes test was run.	Completed
Test 10	The Baseline QCP1 Environment configuration was applied, and CPU=0 was set for all modules, Data Import - Create with 25k and 100k records files tests were run.	Completed

Test Results

This table contains durations for all Workflows.

Workflow	Test 1 (average case) CPU=0		Test 2 (high load) CPU=0		Test 3 (average case) CPU=0		Test 4 (average case) CPU=1
	Average (milliseconds)	Errors	Average (milliseconds)	Errors	Average (milliseconds)	Errors	Average (milliseconds)	Errors
AIE_TC: Create Invoices	8350	100%	13458	100%	7421	100%	8251	100%
AIE_TC: Invoices Approve	3180	100%	5251	100%	2960	100%	3088	100%
AIE_TC: Paying Invoices	3289	100%	4919	100%	3060	100%	3285	100%
CICO_TC_Check-In Controller	2323	0%	3322	7%	2228	0%	2313	0%
CICO_TC_Check-Out Controller	4154	0%	5700	10%	3896	0%	3985	0%
CSI_TC:Share local instance	13008	19%	14781	14%	12969	20%	13022	19%
DE_Exporting MARC Bib records custom workflow	52983	0%	379912	95%	78759	0%	100700	0%
DE_Exporting MARC Bib records workflow	42785	0%	402026	98%	73957	0%	109794	0%
EVA_TC: View Account	803	3%	962	3%	700	1%	832	3%
ILR_TC: Create ILR	1527	0%	2215	4%	1451	0%	1518	0%
MSF_TC: mod search by auth query	672	0%	1830	7%	3096	4%	1023	0%
MSF_TC: mod search by boolean query	165	0%	485	2%	737	1%	212	0%
MSF_TC: mod search by contributors	398	0%	1063	3%	1664	2%	604	0%
MSF_TC: mod search by filter query	286	0%	713	2%	1070	2%	417	0%
MSF_TC: mod search by keyword query	284	0%	658	1%	1006	1%	422	0%
MSF_TC: mod search by subject query	407	0%	1112	1%	1530	2%	623	0%
MSF_TC: mod search by title query	1031	0%	2449	1%	2907	1%	1758	0%
OPIH_/oai/records	6042	0%	4649	100%	3587	0%	5448	0%
POO_TC: Add Order Lines	55334	0%	99076	19%	54440	0%	55224	0%
POO_TC: Approve Order	42567	0%	79022	12%	42191	0%	42907	0%
POO_TC Create Order	31933	0%	56468	13%	31406	0%	32166	0%
RTAC_TC: edge-rtac	4150	0%	4726	0%	4084	0%	4099	0%
SDIC_Single Record Import (Create)	13777	19%	20475	13%	13773	20%	13873	19%
SDIU_Single Record Import (Update)	19549	0%	36639	16%	19582	1%	19568	0%
TC: Receiving-an-Order-Line	45895	100%	83735	100%	45888	100%	46890	100%
Serials-Receiving-Workflow	48584	100%	86692	100%	47338	100%	48019	100%
Unreceiving-a-Piece	8080	100%	13729	100%	8013	100%	8297	100%
ULR_TC: Users loan Renewal Transaction	3189	0%	4818	6%	2970	0%	3125	0%

This table contains durations for Check In\Check Out tests.

Requests	Executions		Response Times (ms)
	#Samples	Error %	Average	90th pct	95th pct	99th pct
Check-In Controller (MCPT) CPU=2	2480	0%	490	579	666	1576
Check-Out Controller (MCPT) CPU=2	2407	0%	1018	1139	1277	2246
Check-In Controller (QCP1) CPU=2	2214	0.63%	583	659	702	880
Check-Out Controller (QCP1) CPU=2	2975	0.94%	953	1077	1166	1448
Check-In Controller (QCP1) CPU=0	2191	0.68%	612	672	719	887
Check-Out Controller (QCP1) CPU=0	2937	0.89%	984	1104	1192	1442

This table contains durations for Data Import tests.

Job Profile	File	Duration (h:mm:ss)	Status
PTF Create 2 (MCPT) CPU=2	25k.mrc	0:15:21	COMMITTED
PTF Create 2 (MCPT) CPU=2	100k.mrc	1:01:25	COMMITTED
PTF - Create 2 (QCP1) CPU=2	25k.mrc	0:14:52	COMMITTED
PTF - Create 2 (QCP1) CPU=2	100k.mrc	0:50:36	COMMITTED
PTF - Create 2 (QCP1) CPU=0	25k.mrc	0:14:47	COMMITTED
PTF - Create 2 (QCP1) CPU=0	100k.mrc	0:52:34	COMMITTED

Comparison

This graph shows the durations of all workflows compared between tests.

Average Case workflows: Part 1.

Image Added

Average Case workflows: Part 2.

Image Added

High Load workflows: Part 1.

Image Added

High Load workflows: Part 2.

Image Added

High Load workflows: Part 3.

Image Added

Test №1-2-3

The Baseline MCPT Environment configuration was applied, and CPU=0 was set for all modules

Instance CPU Utilization

Image Added

Service CPU Utilization

Here we can see that nginx-okapi modules used 20% CPU for Test-2 (high load case).

Image Added

Service Memory Utilization

Here we can see that Memory exceed 113% of the unit power mod-data-export-b modules for Test-2 (high load case).

Image Added

Kafka metrics

Image Added

DB CPU Utilization

DB CPU was 99% average with ERW: Exporting Receiving Information

Image Added

DB Connections

Max number of DB connections was 6000 in maximum fir Test 1, 7500 for Test 2 and 6000 for Test 3.

Image Added

DB load

Image Added

Top SQL-queries

Image Added

Resource utilization for

Test

№8

№4

The Baseline MCPT Environment configuration was applied,

the instance type was changed to x2gd.large, the number of instances was changed to 14, placement strategy was updated to "one task per host", and CPU=2 was set for all services.

Service CPU Utilization

Here we can see that okapi used 38% of the unit CPU power.

Service Memory Utilization

Here we can't see any sign of memory leaks on every module. Memory shows stable trend.

Kafka metrics

DB CPU Utilization

DB CPU was maximum 53%.

DB Connections

Max number of DB connections was 3842.

DB load

Top SQL-queries

Appendix

Infrastructure

PTF - Baseline MCPT environment configuration

...

1 database instance, writer

...

db.r6g.4xlarge

...

Data nodes
- Instance type - r6g.2xlarge.search
- Number of nodes - 4
- Version: OpenSearch_2_7_R20240502
Dedicated master nodes
- Instance type - r6g.large.search
- Number of nodes - 3

...

4 m5.2xlarge brokers in 2 zones
Apache Kafka version 2.8.0
EBS storage volume per broker 300 GiB
auto.create.topics.enable=true
log.retention.minutes=480
default.replication.factor=3

...

title	mcpt modules memory and CPU parameters

Cluster Resources - mcpt-pvt

...

and CPU=1 was set for all modules.

Instance CPU Utilization

Image Added

Service CPU Utilization

Here we can see that okapi used 100000% of the CPU power parameter CPU=1 for module.

Image Added

Service Memory Utilization

Here we can't see any sign of memory leaks on every module. Memory shows stable trend.

Image Added

Kafka metrics

Image Added

DB CPU Utilization

DB CPU was 99% average with ERW: Exporting Receiving Information

Image Added

DB Connections

Max number of DB connections was 5600.

Image Added

DB load

Image Added

Top SQL-queries

Image Added

Test №5

The Baseline MCPT Environment configuration was applied, and CPU=2 was set for all modules, Check In\Check Out with 20 users for one tenant on 30 minutes test was run.

Instance CPU Utilization

Image Added

Service CPU Utilization

Here we can see that mod-permissions used 18300% of the CPU power of the parametr CPU=2.

Image Added

Service Memory Utilization

Here we can't see any sign of memory leaks on every module. Memory shows stable trend.

Image Added

Kafka metrics

Image Added

DB CPU Utilization

DB CPU was 64% maximum.

Image Added

DB Connections

Max number of DB connections was 2112.

Image Added

DB load

Image Added

Top SQL-queries

Image Added

Test №6

The Baseline MCPT Environment configuration was applied, and CPU=2 was set for all modules, Data Import - Create with 25k and 100k records files tests were run.

Instance CPU Utilization

Image Added

Service CPU Utilization

Here we can see that mod-source-record-storage-b used 72000% of the CPU power of parameter CPU=2.

Image Added

Service Memory Utilization

Here we can't see any sign of memory leaks on every module. Memory shows stable trend.

Image Added

Kafka metrics

Image Added

DB CPU Utilization

DB CPU was 99%.

Image Added

DB Connections

Max number of DB connections was 2500.

Image Added

DB load

Image Added

Top SQL-queries

Image Added

Test №7

The Baseline QCP1 Environment configuration was applied, and CPU=2 was set for all modules, Check In\Check Out with 20 users for one tenant on 30 minutes test was run.

Instance CPU Utilization

Image Added

Service CPU Utilization

Here we can see that okapi used 20% of the absolute CPU power of the container instance.

Image Added

Service Memory Utilization

Here we can't see any sign of memory leaks on every module. Memory shows stable trend.

Image Added

Kafka metrics

Image Added

DB CPU Utilization

DB CPU was 57%.

Image Added

DB Connections

Max number of DB connections was 3650.

Image Added

Test №8

The Baseline QCP1 Environment configuration was applied, and CPU=2 was set for all modules, Data Import - Create with 25k and 100k records files tests were run.

Instance CPU Utilization

Image Added

Service CPU Utilization

Here we can see that okapi used 57000% CPU of unit power.

Image Added

Service Memory Utilization

Here we can't see any sign of memory leaks on every module. Memory shows stable trend.

Image Added

Kafka metrics

Image Added

DB CPU Utilization

DB CPU was 98%.

Image Added

DB Connections

Max number of DB connections was 971.

Image Added

Test №9

The Baseline QCP1 Environment configuration was applied, and CPU=0 was set for all modules, Check In\Check Out with 20 users for one tenant on 30 minutes test was run.

Instance CPU Utilization

Image Added

Service CPU Utilization

Here we can see that okapi used 15% of the absolute CPU power of the container instance.

Image Added

Service Memory Utilization

Here we can't see any sign of memory leaks on every module. Memory shows stable trend.

Image Added

Kafka metrics

Image Added

DB CPU Utilization

DB CPU was 98%.

Image Added

DB Connections

Max number of DB connections was 5150.

Image Added

Test №10

The Baseline QCP1 Environment configuration was applied, and CPU=0 was set for all modules, Data Import - Create with 25k and 100k records files tests were run.

Instance CPU Utilization

Image Added

Service CPU Utilization

Here we can see that okapi used 38% of the unit CPU power.

Image Added

Service Memory Utilization

Here we can't see any sign of memory leaks on every module. Memory shows stable trend.

Image Added

Kafka metrics

Image Added

DB CPU Utilization

DB CPU was maximum 92%.

Image Added

DB Connections

Max number of DB connections was 921.

Image Added

Appendix

Infrastructure

PTF - Baseline MCPT environment configuration

14 m6g.2xlarge EC2 instances located in US East (N. Virginia)us-east-1
1 database instance, writer

Name Memory GIB vCPUs
db.r6g.4xlarge
128 GiB 16 vCPUs
Open Search ptf-test
- Data nodes
  - Instance type - r6g.2xlarge.search
  - Number of nodes - 4
  - Version: OpenSearch_2_7_R20240502
- Dedicated master nodes
  - Instance type - r6g.large.search
  - Number of nodes - 3
MSK ptf-KRaft-mode
- 2 brokers, kafka.m7g.xlarge brokers in 2 zones
- Apache Kafka version 3.7.x
- EBS storage volume per broker 300 GiB
- auto.create.topics.enable=true
- log.retention.minutes=480
- default.replication.factor=3

Expand

title	mcpt modules memory and CPU parameters

Cluster Resources - mcpt-pvt

Module	Task Def. Revision	Module Version	Task Count	Mem Hard Limit	Mem Soft limit	CPU units	Xmx	MetaspaceSize	MaxMetaspaceSize
mod-remote-storage	18	mod-remote-storage:3.0.1	2	4920	4472	1024	3960	512	512
mod-ncip	12	mod-ncip:1.14.4	2	1024	896	128	768	88	128
mod-finance-storage	14	mod-finance-storage:8.5.0	4	1024	896	1024	700	88	128
mod-agreements	17	mod-agreements:6.0.2	0	1592	1488	128	0	0	0
mod-ebsconet	12	mod-ebsconet:2.1.1	2	1248	1024	128	700	128	256
edge-sip2	12	edge-sip2:3.1.1	2	1024	896	128	768	88	128
mod-consortia	18	mod-consortia:1.0.3	2	5136	4776	1024	4416	384	512
mod-organizations	12	mod-organizations:1.8.0	2	1024	896	128	768	88	128
mod-finance-storagesettings	1417	mod-finance-storagesettings:81.50.02	42	1024	896	1024200	700768	88	128
modedge-agreementsdematic	1715	modedge-agreementsdematic:62.01.210	1	15921024	1488896	128	07680	88	0128
mod-data-ebsconetimport	1215	mod-data-ebsconetimport:23.10.7	1	22048	12481844	1024256	1281292	700384	128512	256
edgemod-sip2search	1218	edgemod-sip2search:3.10.15	24	10242592	8962480	1282048	7681440	88512	1281024
mod-consortiatags	1812	mod-consortiatags:2.1.0.3	2	51361024	4776896	1024128	4416768	38488	512128
mod-organizationsauthtoken	1222	mod-organizationsauthtoken:12.814.012	6	10241440	8961152	1281024	768922	88	128
modedge-settingscourses	1712	modedge-settingscourses:1.3.0.2	2	1024	896	200128	768	88	128
edge-dematicinventory	156	edge-dematicinventory:21.14.101	2	1024	896	128	768	88	128
mod-datainventory-importupdate	1512	mod-datainventory-importupdate:3.02.711	2	20481024	1844896	256128	1292768	38488	512128
mod-searchnotify	1812	mod-searchnotify:3.1.0.5	42	25921024	2480896	2048128	1440768	51288	1024128
mod-tagsconfiguration	1213	mod-tagsconfiguration:25.19.022	4	1024	896	128	768	88	128
mod-authtokenorders-storage	2213	mod-authtokenorders-storage:213.146.106	4	14401024	1152896	1024512	922700	88	128
edge-coursescaiasoft	12	edge-coursescaiasoft:2.1.3.0	2	1024	896	128	768	88	128
edgemod-login-inventorysaml	615	edgemod-login-inventorysaml:12.47.02	2	1024	896	128	768	88	128
mod-inventoryerm-usage-updateharvester	127	mod-inventoryerm-usage-updateharvester:34.24.1	2	1024	896	128	768	88	128
mod-password-notifyvalidator	12	mod-notifypassword-validator:3.1.0	2	10241440	8961298	128	768	88384	128512
mod-configurationgobi	13	mod-configurationgobi:52.97.1	2	4	1024	896	128	768	88	128
mod-orders-storagelicenses	1315	mod-orders-storagelicenses:135.60.02	42	10242480	8962312	512128	7001792	88384	128512
edgemod-fqm-caiasoftmanager	12	edgemod-fqm-caiasoftmanager:2.1.0.3	2	1024	896	128	768	88	128
modedge-login-samldcb	159	modedge-login-samldcb:21.70.2	2	1024	896	128	768	88	128
mod-ermbulk-usage-harvesteroperations	718	mod-ermbulk-usage-harvesteroperations:41.41.19	2	10243072	8962600	1281024	7681536	88384	128512
mod-password-validatorgraphql	12	mod-password-validatorgraphql:31.112.0	20	14401024	1298896	128	768	38488	512128
mod-gobifinance	1312	mod-gobifinance:24.78.102	4	1024	896	128	768	88	128
mod-licenseserm-usage	157	mod-licenses:5erm-usage:4.6.0.2	2	24801024	2312896	128	1792768	38488	512128
mod-fqm-managerlists	1215	mod-fqm-managerlists:1.0.35	2	1024	896	128	768	88	128
edgemod-dcbcopycat	912	edgemod-dcbcopycat:1.5.0.2	2	1024	896	128	768	88	128
mod-bulk-operationspermissions	1826	mod-bulk-operationspermissions:16.14.90	4	21684	30721544	2600512	10241536	384	512
mod-entities-graphqllinks	1217	mod-graphqlentities-links:12.120.040	2	10242592	8962480	128400	7681440	880	1281024
modpub-financeedge	126	modpub-financeedge:42023.806.0144	2	1024	896	128	768	880	1280
mod-erm-usageorders	713	mod-erm-usageorders:412.67.01	24	10242048	8961440	1281024	7681024	88384	128512
modedge-listspatron	1514	modedge-listspatron:15.0.0.5158	2	1024	896	128256	768	88	128
modedge-copycatncip	12	modedge-copycatncip:1.59.02	2	1024	896	128	768	88	128
mod-permissionsusers-bl	2612	mod-users-permissionsbl:7.6.4.04	2	16841440	15441152	512	1024922	38488	512128
modedge-ea-entitiesdata-linksexport	177	modedge-ea-entitiesdata-linksexport:24.1.0.4	2	25921024	2480896	400128	1440768	088	1024128
pubmod-edgeinvoice	613	pubmod-edgeinvoice:20235.067.1422	4	10241440	8961152	128512	768922	0880	128
mod-ordersinventory-storage	1319	mod-inventory-ordersstorage:1227.70.14	4	20484096	14403690	102420481024	3076	384	512
edgemod-user-patronimport	1412	edgemod-user-patronimport:53.08.0.158	2	1024	896	256128	768	88	128
edgemod-ncipsender	12	edgemod-ncipsender:1.911.20	2	1024	896	128	768	88	128
modedge-usersoai-blpmh	1215	modedge-usersoai-blpmh:2.7.6.01	2	14401512	11521360	5121024	9221440	88384	128512
edgemod-ea-data-export-worker	715	edgemod-ea-data-export-worker:43.1.02	2	10243072	8962800	1281024	7682048	88384	128512
mod-invoicertac	1312	mod-invoicertac:3.5.7.0	2	41024	1440896	1152128	512	922768	88	128
mod-inventorytask-storagelist	197	mod-inventorytask-storagelist:271.09.424	2	40961024	3690896	2048128	3076768	38488	512128
mod-usercirculation-importstorage	1220	mod-usercirculation-importstorage:317.81.07	24	10242880	8962592	1281536	7681814	88384	128512
mod-sendercalendar	1213	mod-sendercalendar:12.115.0	2	1024	896	128	768	88	128	edge-oai-pmh	15	edge-oai-pmh:2.7.1	2	1512	1360	1024	1440	384	512
mod-datasource-exportrecord-workerstorage	1521	mod-datasource-exportrecord-workerstorage:35.17.25	2	30725600	28005000	102420482048	3500	384	512
mod-event-rtacconfig	12	mod-event-rtacconfig:32.56.0	2	1024	896	128	768	88	128
mod-task-listcourses	712	mod-task-listcourses:1.94.28	2	1024	896	128	768	88	128
mod-circulation-storageitem	209	mod-circulation-storageitem:17.1.7	4	2880	2592	1536	1814	384	512	mod-calendar	13	mod-calendar:2.51.0.0	2	1024	896	128	768	88	128
mod-source-record-storage21inventory	19	mod-source-record-storage:5.7.5	2	5600	5000	2048	3500inventory:20.1.6	4	2880	2592	1024	1814	384	512
mod-event-configemail	12	mod-event-configemail:21.616.0	2	1024	896	128	768	88	128
mod-coursescirculation	1220	mod-coursescirculation:124.0.11	4.8	2880	2592	1536	1814	384	512
mod-pubsub	16	mod-pubsub:2.11.3	2	10241536	8961440	1281024	768922	88384	128512
mod-circulationdi-converter-itemstorage	917	mod-circulationdi-converter-itemstorage:2.1.0.05	2	1024	896	128	768	88	128
modedge-inventoryrtac	1915	modedge-inventoryrtac:20.12.6.2	42	28801024	2592896	1024128	1814768	38488	512128
modedge-emailorders	1215	modedge-emailorders:12.169.01	2	1024	896	128	768	88	128
mod-template-circulationengine	2012	mod-circulationtemplate-engine:241.019.111	42	28801024	2592896	1536128	1814768	38488	512128
mod-pubsubusers	1617	mod-pubsubusers:19.2.11.322	4	15361024	1440896	1024128	922768	38488	512128
mod-di-converterpatron-storageblocks	1714	mod-dipatron-converter-storageblocks:21.19.50	2	1024	896	1281024	768	88	128
edgemod-rtacaudit	1513	edgemod-rtacaudit:2.68.20	2	1024	896	128	768	88	128
edge-ordersfqm	1513	edge-ordersfqm:21.90.1	2	1024	896	128	768	88	128
mod-source-templaterecord-enginemanager	1218	mod-template-engine:1.19.1-source-record-manager:3.7.7	2	5600	5000	2048	3500	384	512
nginx-edge	6	nginx-edge:2023.06.14	2	1024	896	128	7680	880	1280
mod-usersquick-marc	1716	mod-usersquick-marc:195.20.21	41	10242288	8962176	128	7681664	88384	128512
modnginx-patron-blocksokapi	149	modnginx-patron-blocksokapi:12023.906.014	2	1024	896	10241536	7680	880	1280
modokapi-auditb	138	mod-auditokapi:25.81.022	3	10241684	8961440	1281536	768922	88384	128512
edgemod-fqmfeesfines	1312	edgemod-fqmfeesfines:119.0.10	2	1024	896	128	768	88	128
mod-sourceinvoice-record-managerstorage	1813	mod-sourceinvoice-record-managerstorage:35.7.70	24	56001872	50001536	20481024	35001024	384	512
nginxedge-edgeusers	6	nginxedge-edgeusers:20231.062.140	2	1024	896	128	07680	88	0128
mod-quick-marcdcb	169	mod-quick-marcdcb:51.0.101	2	22881024	2176896	128	1664768	38488	512128
nginxmod-service-okapiinteraction	915	nginxmod-service-okapiinteraction:20233.060.142	2	10242048	8961844	1536256	012900	384	0512
okapimod-data-bexport	8	okapi:5.1.2	3	1684	1440	1536	922	384	512	mod-feesfines17	mod-data-export:4.8.7	1	1024	896	1024	768	88	128
mod-patron	12	mod-feesfinespatron:196.0.0	24	1024	896	128	768	88	128
mod-invoiceoai-storagepmh	1316	mod-invoiceoai-storagepmh:53.712.08	42	18724096	15363690	102420481024	3076	384	512
edge-usersconnexion	612	edge-usersconnexion:1.21.01	2	1024	896	128	768	88	128
mod-kb-ebsco-dcbjava	912	mod-kb-ebsco-dcbjava:14.0.0	2	1024	896	128	768	88	128
mod-service-interactionnotes	1513	mod-service-interaction:3notes:5.1.0.2	2	20481024	1844896	256128	1290952	384	512
mod-data-export-spring	1715	mod-data-export-spring:43.80.72	1	10242048	8961844	1024256	7681536	88384	128512
mod-patronlogin	12	mod-patronlogin:67.010.01	42	10241440	8961298	1281024	768	88384	128512
mod-oaiorganizations-pmhstorage	1613	mod-oaiorganizations-pmhstorage:34.126.80	24	40961024	3690896	2048128	3076768	38488	512128
edgepub-connexionokapi	126	edgepub-connexionokapi:12023.106.114	2	1024	896	128	768	880	1280
mod-kb-ebsco-java	12	mod-kb-ebsco-java:4edge-erm	7	edge-erm:1.0.0	2	1024	896	128	768	88	128
mod-eusage-notesreports	137	mod-eusage-notesreports:52.10.0	2	1024	896	128	952	384	512
mod-data-export-spring	15	mod-data-export-spring:3.0.2	1	2048	1844	256	1536	384	512
mod-login	12	mod-login:7.10.1	2	1440	1298	1024	768	384	512
mod-organizations-storage	13	mod-organizations-storage:4.6.0	4	1024	896	128	768	88	128
pub-okapi	6	pub-okapi:2023.06.14	2	1024	896	128	768	0	0
edge-erm	7	edge-erm:1.0.0	2	1024	896	128	768	88	128
mod-eusage-reports	7	mod-eusage-reports:2.0.0	2	1024	896	128	768	88	128

Methodology/Approach

MOBIUS Tests: scenarios were started by JMeter script from load generator. We had 100% error count for AIE_TC: Create Invoices, AIE_TC: Invoices Approve, AIE_TC: Paying Invoices, TC: Receiving-an-Order-Line, Unreceiving-a-Piece and Unreceiving-a-Piece Workflows because data was not regenerated.

Baseline MCPT Environment configuration according to tunning environment from previous report: task count: 4 for services: mod-permissions, mod-search, mod-patron, mod-inventory, mod-inventory-storage, mod-circulation, mod-circulation-storage, mod-order, mod-order-storage, mod-invoice, mod-invoice-storage, for mod-users and mod-authtoken task count 6. Parameter srs.marcIndexers.delete.interval.seconds=86400 for mod-source-record-storage. Instance type: m6g.2xlarge. Instances count: 14. Database r6g.4xlarge, Amazon OpenSearch Service ptf-test: r6g.2хlarge.search (4 nodes).

...

768

88

128

Methodology/Approach

MOBIUS Tests: scenarios were started by JMeter script from load generator. We had 100% error count for AIE_TC: Create Invoices, AIE_TC: Invoices Approve, AIE_TC: Paying Invoices, TC: Receiving-an-Order-Line, Unreceiving-a-Piece and Unreceiving-a-Piece Workflows because data was not regenerated.

Baseline MCPT Environment configuration according to tunning environment from previous report: task count: 4 for services: mod-permissions, mod-search, mod-patron, mod-inventory, mod-inventory-storage, mod-circulation, mod-circulation-storage, mod-order, mod-order-storage, mod-invoice, mod-invoice-storage, for mod-users and mod-authtoken task count 6. Parameter srs.marcIndexers.delete.interval.seconds=86400 for mod-source-record-storage. Instance type: m6g.2xlarge. Instances count: 14. Database r6g.4xlarge, Amazon OpenSearch Service ptf-test: r6g.2хlarge.search (4 nodes).

Test 1: The Baseline MCPT Environment configuration was applied, and CPU=0 was set for all modules, Fixed Load (average case) MOBIUS test was run.
Test 2: The Baseline MCPT Environment configuration was applied, and CPU=0 was set for all modules, Fixed Load (high load case) MOBIUS test was run.
Test 3: The Baseline MCPT Environment configuration was applied, and CPU=0 was set for all modules, Fixed Load (average case) MOBIUS test was run (rerun Test 1).
Test 4: The Baseline MCPT Environment configuration was applied, and CPU=1 was set for all modules, Fixed Load (average case) MOBIUS test was run.
Test 5: The Baseline MCPT Environment configuration was applied, and CPU=2 was set for all modules, Check In\Check Out with 20 users for one tenant on 30 minutes test was run.
Test 6: The Baseline MCPT Environment configuration was applied, and CPU=2 was set for all modules, Data Import - Create with 25k and 100k records files tests were run.
Test 7: The Baseline QCP1 Environment configuration was applied, and CPU=2 was set for all modules, Check In\Check Out with 20 users for one tenant on 30 minutes test was run.
Test 8: The Baseline QCP1 Environment configuration was applied, and CPU=02 was set for all modules, Fixed Load (average case) MOBIUS test was Data Import - Create with 25k and 100k records files tests were run.
Test 29: The Baseline MCPT QCP1 Environment configuration was applied, and CPU=0 was set for all modules, Fixed Load (high load case) MOBIUS Check In\Check Out with 20 users for one tenant on 30 minutes test was run.
Test 310: The The Baseline MCPT QCP1 Environment configuration was applied, and CPU=0 was set for all modules, Fixed Load (average case) MOBIUS test was run (rerun Test 1) Data Import - Create with 25k and 100k records files tests were run.

Version	Old Version 7	New Version Current
Changes made by	Stanislav Nehrii	Stanislav Nehrii
Saved on	Aug 15, 2024	Sep 03, 2024

Test #	Description	Status
Test 1	The Baseline MCPT Environment configuration was applied, and CPU=0 was set for all modules, Fixed Load (average case) MOBIUS test was run.	Completed
Test 2	The Baseline MCPT Environment configuration was applied, and CPU=0 was set for all modules, Fixed Load (high load case) MOBIUS test was run.	Completed
Test 3	The Baseline MCPT Environment configuration was applied, and CPU=0 was set for all modules, Fixed Load (average case)

Name	Memory GIB	vCPUs
db.r6g.4xlarge	128 GiB	16 vCPUs

Page Comparison

Versions Compared

Key

Overview

Summary

Test Runs

Comparison

Resource utilization for Test №1

Service CPU Utilization

Service Memory Utilization

Kafka metrics

DB CPU Utilization

DB Connections

Resource utilization for Test №2

Service CPU Utilization

Service Memory Utilization

Kafka metrics

DB CPU Utilization

DB Connections

DB load

Top SQL-queries

Resource utilization for Test №3

Service CPU Utilization

Service Memory Utilization

Kafka metrics

DB CPU Utilization

DB Connections

Resource utilization for Test №4

Service CPU Utilization

Service Memory Utilization

Kafka metrics

DB CPU Utilization

DB Connections

DB load

Top SQL-queries

Resource utilization for Test №5

Service CPU Utilization

Service Memory Utilization

Kafka metrics

DB CPU Utilization

DB Connections

DB load

Top SQL-queries

Resource utilization for Test №6

Service CPU Utilization

Service Memory Utilization

Inctanse CPU Utilization

Kafka metrics

DB CPU Utilization

DB Connections

DB load

Top SQL-queries

Resource utilization for Test №7

Service CPU Utilization

Service Memory Utilization

Inctanse CPU Utilization

Kafka metrics

DB CPU Utilization

DB Connections

DB load

Test Results

Comparison

Test №1-2-3

Instance CPU Utilization

Service CPU Utilization

Service Memory Utilization

Kafka metrics

DB CPU Utilization

DB Connections

DB load

Top SQL-queries

Test

№4

Service CPU Utilization

Service Memory Utilization

Kafka metrics

DB CPU Utilization

DB Connections

DB load

Top SQL-queries