Overview

Test goal is to assess performance impact of Check-out lock feature on Check-out functionality.

Ticket: PERF-627 - Getting issue details... STATUS

Summary

• Load tests showed that there is no significant influence of check-out lock on performance of the application.
• Average response time of check-out request is about 0.5s (level of load - 6.6 requests/minute).

Recomendations and Jiras

It would be usefull to implement some additional monitoring for the check-out lock feature. It can include such metrics as number of locks accross the users, locking history and rate of locking to closely monitor it for performance implications.

Test runs

*Check-out lock feature was enabled with the following parameters:

• "noOfRetryAttempts" - 30
• "retryInterval" - 250
• "lockTtl" - 2000

**Tests were repeated with higher level of load (66 requests/minute) to ensure that there are enough data points for statistics and results are accurate.

Test results

Check-out lock enabled

Check-out lock disabled

Check-out lock enabled

Check-out lock disabled

Service CPU Utilization

Memory Utilization

DB CPU Utilization

DB Connections

DB Load

Top SQL

Check-out lock enabled

Check-out lock disabled

Additional Information

In order to make sure that check-out lock work correctly, next requests were found in logs during load tests with check-out lock enabled:

At the screenshot we can see that lock is being created in check-out-lock-storage and then deleted almost at the same moment. Deletion of lock happens when check-out for item is done.

Appendix

Infrastructure

PTF -environment pcp1

• 10 m6i.2xlarge EC2 instances located in US East (N. Virginia)us-east-1

• 1 database  instances, writer

  Name	API Name	Memory GIB	vCPUs	max_connections
  R6G Extra Large	db.r6g.xlarge	32 GiB	4 vCPUs	2731

• MSK ptf-kakfa-3
  
  • 4 m5.2xlarge brokers in 2 zones

  • Apache Kafka version 2.8.0

  • EBS storage volume per broker 300 GiB

  • auto.create.topics.enable=true
  • log.retention.minutes=480
  • default.replication.factor=3

Methodology

Conduct baseline tests with check-out lock disabled.

Conduct verification tests with check-out lock enabled. It can be done directly in the database (table [tenant]_mod_settings.settings).

Value used for the testing:

{  "value": "{\"checkOutLockFeatureEnabled\":true,\"noOfRetryAttempts\":30,\"retryInterval\":250,\"lockTtl\":2000}" }