/
PTF - Performance testing of CPU=0 for Services (mcpt)

PTF - Performance testing of CPU=0 for Services (mcpt)

Owned by Stanislav Nehrii
Last updated: Sep 03, 2024

Overview

  • In this report, PTF investigates the impact of setting CPU allocations to 0 units across all tasks within an AWS ECS cluster. The purpose of this study is to determine whether removing CPU constraints reveals the actual CPU usage of the tasks and to assess how this adjustment affects overall performance. By comparing key workflows across different environments, we aim to identify any potential changes in efficiency, throughput, or resource utilization that may result from setting CPU = 0. The findings from these tests will help inform best practices for resource allocation and performance optimization within ECS clusters.

PERF-959 - Getting issue details... STATUS  

Summary

  • During the tests Parameter CPU=0 improved performance for a several workflows, like MSF_TC: mod search by auth query, MSF_TC: mod search by contributors, MSF_TC: mod search by title query and ULR_TC: Users loan Renewal Transaction.

  • No memory leaks, memory consumption was stable during all of the tests.

  • Tests had 100% errors count for AIE_TC: Create Invoices, AIE_TC: Invoices Approve, AIE_TC: Paying Invoices, TC: Receiving-an-Order-Line, Unreceiving-a-Piece and Unreceiving-a-Piece Workflows because data was not regenerated.  

Test Runs

Test #DescriptionStatus
Test 1The Baseline MCPT Environment configuration was applied, and CPU=0 was set for all modules,  Fixed Load (average case) MOBIUS test was run.Completed
Test 2The Baseline MCPT Environment configuration was applied, and CPU=0 was set for all modules,  Fixed Load (high load case) MOBIUS test was run.Completed
Test 3The Baseline MCPT Environment configuration was applied, and CPU=0 was set for all modules,  Fixed Load (average case) MOBIUS test was run (rerun Test 1).Completed
Test 4The Baseline MCPT Environment configuration was applied, and CPU=1 was set for all modules,  Fixed Load (average case) MOBIUS test was run.Completed
Test 5The Baseline MCPT Environment configuration was applied, and CPU=2 was set for all modules,  Check In\Check Out with 20 users for one tenant on 30 minutes test was run.Completed
Test 6The Baseline MCPT Environment configuration was applied, and CPU=2 was set for all modules,  Data Import - Create with 25k and 100k records files tests were run.Completed
Test 7The Baseline QCP1 Environment configuration was applied, and CPU=2 was set for all modules,  Check In\Check Out with 20 users for one tenant on 30 minutes test was run.Completed
Test 8The Baseline QCP1 Environment configuration was applied, and CPU=2 was set for all modules,  Data Import - Create with 25k and 100k records files tests were run.Completed
Test 9The Baseline QCP1 Environment configuration was applied, and CPU=0 was set for all modules,  Check In\Check Out with 20 users for one tenant on 30 minutes test was run.Completed
Test 10The Baseline QCP1 Environment configuration was applied, and CPU=0 was set for all modules,  Data Import - Create with 25k and 100k records files tests were run.Completed

Test Results

This table contains durations for all Workflows. 

Workflow

Test 1
(average case)

CPU=0

Test 2
(high load)

CPU=0

Test 3
(average case) 

CPU=0

Test 4
(average case) 

CPU=1


Average
(milliseconds)		ErrorsAverage
(milliseconds)		ErrorsAverage
(milliseconds)		ErrorsAverage
(milliseconds)		Errors
AIE_TC: Create Invoices8350100%13458100%7421100%8251100%
AIE_TC: Invoices Approve3180100%5251100%2960100%3088100%
AIE_TC: Paying Invoices3289100%4919100%3060100%3285100%
CICO_TC_Check-In Controller23230%33227%22280%23130%
CICO_TC_Check-Out Controller41540%570010%38960%39850%
CSI_TC:Share local instance1300819%1478114%1296920%1302219%
DE_Exporting MARC Bib records custom workflow529830%37991295%787590%1007000%
DE_Exporting MARC Bib records workflow427850%40202698%739570%1097940%
EVA_TC: View Account8033%9623%7001%8323%
ILR_TC: Create ILR15270%22154%14510%15180%
MSF_TC: mod search by auth query6720%18307%30964%10230%
MSF_TC: mod search by boolean query1650%4852%7371%2120%
MSF_TC: mod search by contributors3980%10633%16642%6040%
MSF_TC: mod search by filter query2860%7132%10702%4170%
MSF_TC: mod search by keyword query2840%6581%10061%4220%
MSF_TC: mod search by subject query4070%11121%15302%6230%
MSF_TC: mod search by title query10310%24491%29071%17580%
OPIH_/oai/records60420%4649100%35870%54480%
POO_TC: Add Order Lines553340%9907619%544400%552240%
POO_TC: Approve Order425670%7902212%421910%429070%
POO_TC Create Order319330%5646813%314060%321660%
RTAC_TC: edge-rtac41500%47260%40840%40990%
SDIC_Single Record Import (Create)1377719%2047513%1377320%1387319%
SDIU_Single Record Import (Update)195490%3663916%195821%195680%
TC: Receiving-an-Order-Line45895100%83735100%45888100%46890100%
Serials-Receiving-Workflow48584100%86692100%47338100%48019100%
Unreceiving-a-Piece8080100%13729100%8013100%8297100%
ULR_TC: Users loan Renewal Transaction31890%48186%29700%31250%


This table contains durations for Check In\Check Out tests. 

RequestsExecutions Response Times (ms)   

#SamplesError %Average90th pct95th pct99th pct
Check-In Controller (MCPT)
CPU=2		24800%4905796661576

Check-Out Controller (MCPT)

CPU=2

24070%1018113912772246

Check-In Controller (QCP1)

CPU=2

22140.63%583659702880

Check-Out Controller (QCP1)

CPU=2

29750.94%953107711661448

Check-In Controller (QCP1)

CPU=0

21910.68%612672719887

Check-Out Controller (QCP1)

CPU=0

29370.89%984110411921442


This table contains durations for Data Import tests. 

 Job ProfileFile

Duration

(h:mm:ss)

Status

PTF Create 2 (MCPT)

CPU=2

25k.mrc

0:15:21COMMITTED

PTF Create 2 (MCPT)

CPU=2

100k.mrc

1:01:25COMMITTED

PTF - Create 2 (QCP1)

CPU=2

25k.mrc

0:14:52COMMITTED

PTF - Create 2 (QCP1)

CPU=2

100k.mrc


0:50:36COMMITTED

PTF - Create 2 (QCP1)

CPU=0

25k.mrc0:14:47COMMITTED

PTF - Create 2 (QCP1)

CPU=0

100k.mrc0:52:34COMMITTED