Versions Compared

Old Version 15

changes.mady.by.user Hanna Tuhuzbayeva

Saved on

compared with

New Version Current

changes.mady.by.user Hanna Tuhuzbayeva

Saved on

Key

  • This line was added.
  • This line was removed.
  • Formatting was changed.

Table of Contents
Overview

This page is created to investigate Aurora serverless performance by comparing DB xlarge, 8xlarge and Aurora serverless instance types under load running Data Import (DI) with Check-in Check-out (CICO) running as background. 

Ticket:

Jira Legacy
serverSystem Jira
serverId01505d01-b853-3c2e-90f1-ee9b165564fc
keyPERF-611

...

Table of Contents
Overview

This page is created to investigate Aurora serverless performance by comparing DB xlarge, 8xlarge and Aurora serverless instance types under load running Data Import (DI) with Check-in Check-out (CICO) running as background. 

Ticket:

Jira Legacy
serverSystem JIRA
serverId01505d01-b853-3c2e-90f1-ee9b165564fc
keyPERF-611

Summary

  • Tests showed that there is slight performance improvement for DI itself and DI with CICO comparing xlarge and 0.5-128 ACU. Show more details...
  • For CICO itself and CICO together with DI there is no significant influence of using serverless configuration. Show more details...
  • Serverless DB configuration consumes more CPU resources and less DB resources than RDS  configuration comparing Serverless (0.5 - 128 ACUs) and RDS db.r6g.xlarge configurations. Show more details...
  • Serverless DB configuration consumes less CPU and DB resources than RDS  configuration comparing Serverless (32 - 128 ACUs) and RDS db.r6g.8xlarge configurations. Show more details...

As a conclusion, there is no much degradation with usage of serverless DB. According to the results, it could be an adequate replacement for regular RDS.

Tests

Job profiles - PTF - Create 2, PTF - Updates Success - 1

ScenarioData quantity
DI Create, DI Update (coherently)25K
DI Create, DI Update (coherently) with CICO  in parallel

DI - 25K

CICO - 20 users

CICO20 users

...

DB typeDB configuration
RDS

db.r6g.xlarge

db.r6g.8xlarge

Serverless

Serverless v2 (0.5 - 128 ACUs)

Serverless v2 (32 - 128 ACUs)

Results

At the table below next results are compared: RDS xlarge agains against 0.5-128 ACU Serverless and RDS 8xlarge agains against 32-128 ACU Serverless (marked by color).

Process/requestRDSServerless
xlarge8xlarge0.5-128 ACU32-128 ACU
DI Create18 min9 min11 min (-7 min)​14 min (+5 min)​
DI Update17 min10 min16 min (-1 min)​11 min  (+1 min)​
DI Create (with CICO in parallel)22 min9 min20 min (-2 min)​11 min   (+2 min)​
DI Update (with CICO in parallel)22 min, error*12 min25 min, error** (+3 min)​12 min (0 min)​
CI, 95%0.680.7180.806 (+0.126 sec)0.794 (+0.076 sec)​
CO, 95%1.2471.2511.393 (+0.146 min, error** (+3 min)​12 min (0 min)​
CI, 95%0.680.7180.806 (+0.126 sec)0.794 (+0.076 sec)​
CO, 95%1.2471.2511.393 (+0.146 sec)​1.312 (+0.061 sec)​
CI (with DI Update in parallel), 95%1.361.1931.277 (-0.083 sec)​1.11 (-0.083 sec)​
CO (with DI Update in parallel), 95%2.3352.0622.321 (-0.014 sec)​1.312 963 (+-0.061 099 sec)​
CI (with DI Update in parallel), 95%1.361.1931.277 (-0.083 sec)​1.11 (-0.083 sec)​
CO (with DI Update in parallel), 95%2.3352.0622.321 (-0.014 sec)​1.963 (-0.099 sec)​

*some records were discarded

**Error - io.vertx.core.impl.NoStackTraceThrowable

...

*some records were discarded

**Error - io.vertx.core.impl.NoStackTraceThrowable

Response time

db.r6g.xlarge DI with CICO

There are more response time spikes during the test with xlarge configuration comparing to the tests with 8xlarge.

Image Added

db.r6g.8xlarge DI with CICO

Image Added

Serverless v2 (0.5 - 128 ACUs) DI with CICO

There is a spike at the beginning of DI Update job. This can be connected to mod-audit CPU increase at this moment.

Image Added

Serverless v2 (32 - 128 ACUs) DI with CICO

Image Added

Service CPU Utilization

db.r6g.xlarge DI without CICO

Maximum CPU utilization - 80% DI Create, 120% DI Update

Image Added

db.r6g.xlarge DI with CICO

There are more response time spikes during the test with xlarge configuration comparing to the tests with 8xlarge.

Image RemovedMaximum CPU utilization - 75% DI Create, 100% DI Update

Image Added

db.r6g.8xlarge DI

...

Image Removed

Serverless v2 (0.5 - 128 ACUs) DI with CICO

There is a spike at the begining of DI Update job. This can be connected ot mod-audit CPU increase at this moment.

Image Removed

...

without CICO

Maximum CPU utilization - 135% DI Create, 160% DI Update

Image Added

db.r6g.8xlarge DI with CICO

Maximum CPU utilization - 150% DI Create, 200% DI Update

Image Added

Serverless v2 (0.5 - 128 ACUs) DI

...

without CICO

Image Removed

Service CPU Utilization

db.r6g.xlarge DI without CICO

Image Removed

db.r6g.xlarge DI with CICO

Image Removed

db.r6g.8xlarge DI without CICO

Image Removed

db.r6g.8xlarge DI with CICO

Image RemovedMaximum CPU utilization - 110% DI Create, 125% DI Update

Serverless v2 (0.5 - 128 ACUs) DB configuration consumes more CPU resources than RDS db.r6g.xlarge configuration (110%-125% compared to 80%-120%).

Image Added

Serverless v2 (0.5 - 128 ACUs) DI

...

with CICO

Maximum CPU utilization - 100% DI Create, 105% DI Update

Image Added

Serverless v2 (

...

32 - 128 ACUs) DI

...

without CICO

Image RemovedMaximum CPU utilization - 100% DI Create, 150% DI Update

Serverless v2 (32 - 128 ACUs

...

Image Removed) DB configuration consumes less CPU resources than RDS db.r6g.8xlarge configuration (100%-150% compared to 135%-160%).

Image Added

Serverless v2 (32 - 128 ACUs) DI with CICO

Image RemovedMaximum CPU utilization - 100% DI Create, 160% DI Update

Image Added

Memory Utilization

db.r6g.xlarge without DI CICO

Image RemovedImage Added

db.r6g.xlarge DI with CICO

Image RemovedImage Added

db.r6g.8xlarge DI without CICO

Image RemovedImage Added

db.r6g.8xlarge DI with CICO

Image RemovedImage Added

Serverless v2 (0.5 - 128 ACUs) DI without CICO

Image RemovedImage Added

Serverless v2 (0.5 - 128 ACUs) DI with CICO

Image RemovedImage Added

Serverless v2 (32 - 128 ACUs) DI without CICO

Image RemovedImage Added

Serverless v2 (32 - 128 ACUs) DI with CICO

Image RemovedImage Added

DB CPU Utilization

db.r6g.xlarge DI without CICO

Maximum DB CPU utilization - 90%

Image Added

db.r6g.xlarge DI with CICO

Maximum DB CPU utilization - 90%

Image Added

db.r6g.8xlarge DI without CICO

Image RemovedMaximum DB CPU utilization - 37%

Image Added

db.r6g.

...

8xlarge DI with CICO

...

Image Removed

...

Maximum DB CPU utilization - 37%

Image Added

Serverless v2 (0.5 - 128 ACUs) DI without CICO

Image Removed

Maximum DB CPU utilization - 30%

Serverless v2 (0.5 - 128 ACUs) DB configuration consumes less DB CPU resources than RDS db.r6g

...

Image Removed.xlarge configuration (30% compared to 90%).

Image Added

Serverless v2 (0.5 - 128 ACUs) DI

...

with CICO

Maximum DB CPU utilization - 25%

Image Added

Serverless v2 (

...

32 - 128 ACUs) DI

...

without CICO

Image RemovedMaximum DB CPU utilization - 25%

Serverless v2 (32 - 128 ACUs

...

Image Removed) DB configuration consumes slightly less DB CPU resources than RDS db.r6g.8xlarge configuration (25% compared to 37%).

Image Added

Serverless v2 (32 - 128 ACUs) DI with CICO

Image RemovedMaximum DB CPU utilization - 25%

Image Added

DB Connections

db.r6g.xlarge DI without CICO

Image RemovedImage Added

db.r6g.xlarge DI with CICO

Image RemovedImage Added

db.r6g.8xlarge DI without CICO

Image RemovedImage Added

db.r6g.8xlarge DI with CICO

Image RemovedImage Added


Serverless v2 (0.5 - 128 ACUs) DI without CICO

Image RemovedImage Added

Serverless v2 (0.5 - 128 ACUs) DI with CICO

Image RemovedImage Added

Serverless v2 (32 - 128 ACUs) DI without CICO

Image RemovedImage Added

Serverless v2 (32 - 128 ACUs) DI with CICO

Image RemovedImage Added

Database load

db.r6g.8xlarge DI without CICO

Image RemovedImage Added

db.r6g.8xlarge DI with CICO

Image RemovedImage Added

Serverless v2 (32 - 128 ACUs) DI without CICO

Image RemovedImage Added

Serverless v2 (32 - 128 ACUs) DI with CICO

Image RemovedImage Added

Appendix

Grafana dashboard

db.r6g.xlarge DI with CICO

http://carrier-io.int.folio.ebsco.com/grafana/d/SqzWB26nk/jmeter-performance-check-in-check-out?orgId=1&from=1690186885745&to=1690192449004&var-percentile=95&var-test_type=baseline&var-test=circulation_checkInCheckOut_Orchid_cptf2_4&var-env=int&var-grouping=1s&var-low_limit=250&var-high_limit=750&var-db_name=jmeter&var-sampler_type=All

db.r6g.8xlarge DI with CICO

http://carrier-io.int.folio.ebsco.com/grafana/d/SqzWB26nk/jmeter-performance-check-in-check-out?orgId=1&from=1690206826647&to=1690209403208&var-percentile=95&var-test_type=baseline&var-test=circulation_checkInCheckOut_Orchid_cptf2_4&var-env=int&var-grouping=1s&var-low_limit=250&var-high_limit=750&var-db_name=jmeter&var-sampler_type=All

Serverless v2 (0.5 - 128 ACUs) DI with CICO

http://carrier-io.int.folio.ebsco.com/grafana/d/SqzWB26nk/jmeter-performance-check-in-check-out?orgId=1&from=1689860038636&to=1689864442762&var-percentile=95&var-test_type=baseline&var-test=circulation_checkInCheckOut_Orchid_cptf2_4&var-env=int&var-grouping=1s&var-low_limit=250&var-high_limit=750&var-db_name=jmeter&var-sampler_type=All

Serverless v2 (32 - 128 ACUs) DI with CICO

http://carrier-io.int.folio.ebsco.com/grafana/d/SqzWB26nk/jmeter-performance-check-in-check-out?orgId=1&from=1689934923312&to=1689937778313&var-percentile=95&var-test_type=baseline&var-test=circulation_checkInCheckOut_Orchid_cptf2_4&var-env=int&var-grouping=1s&var-low_limit=250&var-high_limit=750&var-db_name=jmeter&var-sampler_type=All

...