Overview

This page is created to investigate Aurora serverless performance by comparing DB xlarge, 8xlarge and Aurora serverless instance types under load running Data Import (DI) with Check-in Check-out (CICO) running as background.

Ticket: PERF-611 - Getting issue details... STATUS

Summary

Tests showed that there is slight performance improvement for DI itself and DI with CICO comparing xlarge and 0.5-128 ACU. Show more details...
For CICO itself and CICO together with DI there is no significant influence of using serverless configuration. Show more details...
Serverless DB configuration consumes more CPU resources and less DB resources than RDS configuration comparing Serverless (0.5 - 128 ACUs) and RDS db.r6g.xlarge configurations. Show more details...
Serverless DB configuration consumes less CPU and DB resources than RDS configuration comparing Serverless (32 - 128 ACUs) and RDS db.r6g.8xlarge configurations. Show more details...

As a conclusion, there is no much degradation with usage of serverless DB. According to the results, it could be an adequate replacement for regular RDS.

Tests

Job profiles - PTF - Create 2, PTF - Updates Success - 1

Scenario	Data quantity
DI Create, DI Update (coherently)	25K
DI Create, DI Update (coherently) with CICO in parallel	DI - 25K CICO - 20 users
CICO	20 users

Each scenario was tested on next configurations:

DB type	DB configuration
RDS	db.r6g.xlarge
RDS	db.r6g.8xlarge
Serverless	Serverless v2 (0.5 - 128 ACUs)
Serverless	Serverless v2 (32 - 128 ACUs)

Results

At the table below next results are compared: RDS xlarge against 0.5-128 ACU Serverless and RDS 8xlarge against 32-128 ACU Serverless (marked by color).

Process/request	RDS		Serverless
Process/request	xlarge	8xlarge	0.5-128 ACU	32-128 ACU
DI Create	18 min	9 min	11 min (-7 min)	14 min (+5 min)
DI Update	17 min	10 min	16 min (-1 min)	11 min (+1 min)
DI Create (with CICO in parallel)	22 min	9 min	20 min (-2 min)	11 min (+2 min)
DI Update (with CICO in parallel)	22 min, error*	12 min	25 min, error** (+3 min)	12 min (0 min)
CI, 95%	0.68	0.718	0.806 (+0.126 sec)	0.794 (+0.076 sec)
CO, 95%	1.247	1.251	1.393 (+0.146 sec)	1.312 (+0.061 sec)
CI (with DI Update in parallel), 95%	1.36	1.193	1.277 (-0.083 sec)	1.11 (-0.083 sec)
CO (with DI Update in parallel), 95%	2.335	2.062	2.321 (-0.014 sec)	1.963 (-0.099 sec)

*some records were discarded

**Error - io.vertx.core.impl.NoStackTraceThrowable

Response time

db.r6g.xlarge DI with CICO

There are more response time spikes during the test with xlarge configuration comparing to the tests with 8xlarge.

db.r6g.8xlarge DI with CICO

Serverless v2 (0.5 - 128 ACUs) DI with CICO

There is a spike at the beginning of DI Update job. This can be connected to mod-audit CPU increase at this moment.

Serverless v2 (32 - 128 ACUs) DI with CICO

Service CPU Utilization

db.r6g.xlarge DI without CICO

Maximum CPU utilization - 80% DI Create, 120% DI Update

db.r6g.xlarge DI with CICO

Maximum CPU utilization - 75% DI Create, 100% DI Update

db.r6g.8xlarge DI without CICO

Maximum CPU utilization - 135% DI Create, 160% DI Update

db.r6g.8xlarge DI with CICO

Maximum CPU utilization - 150% DI Create, 200% DI Update

Serverless v2 (0.5 - 128 ACUs) DI without CICO

Maximum CPU utilization - 110% DI Create, 125% DI Update

Serverless v2 (0.5 - 128 ACUs) DB configuration consumes more CPU resources than RDS db.r6g.xlarge configuration (110%-125% compared to 80%-120%).

Serverless v2 (0.5 - 128 ACUs) DI with CICO

Maximum CPU utilization - 100% DI Create, 105% DI Update

Serverless v2 (32 - 128 ACUs) DI without CICO

Maximum CPU utilization - 100% DI Create, 150% DI Update

Serverless v2 (32 - 128 ACUs) DB configuration consumes less CPU resources than RDS db.r6g.8xlarge configuration (100%-150% compared to 135%-160%).

Serverless v2 (32 - 128 ACUs) DI with CICO

Maximum CPU utilization - 100% DI Create, 160% DI Update

Memory Utilization

db.r6g.xlarge without DI CICO

db.r6g.xlarge DI with CICO

db.r6g.8xlarge DI without CICO

db.r6g.8xlarge DI with CICO

Serverless v2 (0.5 - 128 ACUs) DI without CICO

Serverless v2 (0.5 - 128 ACUs) DI with CICO

Serverless v2 (32 - 128 ACUs) DI without CICO

Serverless v2 (32 - 128 ACUs) DI with CICO

DB CPU Utilization

db.r6g.xlarge DI without CICO

Maximum DB CPU utilization - 90%

db.r6g.xlarge DI with CICO

Maximum DB CPU utilization - 90%

db.r6g.8xlarge DI without CICO

Maximum DB CPU utilization - 37%

db.r6g.8xlarge DI with CICO

Maximum DB CPU utilization - 37%

Serverless v2 (0.5 - 128 ACUs) DI without CICO

Maximum DB CPU utilization - 30%

Serverless v2 (0.5 - 128 ACUs) DB configuration consumes less DB CPU resources than RDS db.r6g.xlarge configuration (30% compared to 90%).

Serverless v2 (0.5 - 128 ACUs) DI with CICO

Maximum DB CPU utilization - 25%

Serverless v2 (32 - 128 ACUs) DI without CICO

Maximum DB CPU utilization - 25%

Serverless v2 (32 - 128 ACUs) DB configuration consumes slightly less DB CPU resources than RDS db.r6g.8xlarge configuration (25% compared to 37%).

Serverless v2 (32 - 128 ACUs) DI with CICO

Maximum DB CPU utilization - 25%

DB Connections

db.r6g.xlarge DI without CICO

db.r6g.xlarge DI with CICO

db.r6g.8xlarge DI without CICO

db.r6g.8xlarge DI with CICO

Serverless v2 (0.5 - 128 ACUs) DI without CICO

Serverless v2 (0.5 - 128 ACUs) DI with CICO

Serverless v2 (32 - 128 ACUs) DI without CICO

Serverless v2 (32 - 128 ACUs) DI with CICO

Database load

db.r6g.8xlarge DI without CICO

db.r6g.8xlarge DI with CICO

Serverless v2 (32 - 128 ACUs) DI without CICO

Serverless v2 (32 - 128 ACUs) DI with CICO

Appendix

Links

Grafana dashboard

db.r6g.xlarge DI with CICO

http://carrier-io.int.folio.ebsco.com/grafana/d/SqzWB26nk/jmeter-performance-check-in-check-out?orgId=1&from=1690186885745&to=1690192449004&var-percentile=95&var-test_type=baseline&var-test=circulation_checkInCheckOut_Orchid_cptf2_4&var-env=int&var-grouping=1s&var-low_limit=250&var-high_limit=750&var-db_name=jmeter&var-sampler_type=All

db.r6g.8xlarge DI with CICO

http://carrier-io.int.folio.ebsco.com/grafana/d/SqzWB26nk/jmeter-performance-check-in-check-out?orgId=1&from=1690206826647&to=1690209403208&var-percentile=95&var-test_type=baseline&var-test=circulation_checkInCheckOut_Orchid_cptf2_4&var-env=int&var-grouping=1s&var-low_limit=250&var-high_limit=750&var-db_name=jmeter&var-sampler_type=All

Serverless v2 (0.5 - 128 ACUs) DI with CICO

http://carrier-io.int.folio.ebsco.com/grafana/d/SqzWB26nk/jmeter-performance-check-in-check-out?orgId=1&from=1689860038636&to=1689864442762&var-percentile=95&var-test_type=baseline&var-test=circulation_checkInCheckOut_Orchid_cptf2_4&var-env=int&var-grouping=1s&var-low_limit=250&var-high_limit=750&var-db_name=jmeter&var-sampler_type=All

Serverless v2 (32 - 128 ACUs) DI with CICO

http://carrier-io.int.folio.ebsco.com/grafana/d/SqzWB26nk/jmeter-performance-check-in-check-out?orgId=1&from=1689934923312&to=1689937778313&var-percentile=95&var-test_type=baseline&var-test=circulation_checkInCheckOut_Orchid_cptf2_4&var-env=int&var-grouping=1s&var-low_limit=250&var-high_limit=750&var-db_name=jmeter&var-sampler_type=All

Configuration

Version of modules:
Source Record Manager Module (mod-source-record-manager-3.5.6)
Source Record Storage Module (mod-source-record-storage-5.5.2)
Inventory Module (mod-inventory-19.0.2)
Inventory Storage Module (mod-inventory-storage-25.0.3)
Inventory Update Module (mod-inventory-update-2.3.1)
Data Import Module (mod-data-import-2.6.2)
quickMARC (mod-quick-marc-2.5.0)

CICO

Version of modules:
Okapi (okapi-4.14.10)
users (mod-users-19.0.0)
Remote storage API module (mod-remote-storage-1.7.2)
Pubsub (mod-pubsub-2.7.0)
Patron Blocks Module (mod-patron-blocks-1.7.1)
Inventory Storage Module (mod-inventory-storage-25.0.3)
Inventory Module (mod-inventory-19.0.2)
feesfines (mod-feesfines-18.1.1)
Configuration (mod-configuration-5.9.0)
Circulation Storage Module (mod-circulation-storage-15.0.2)
Circulation Module (mod-circulation-23.3.4)
authtoken (mod-authtoken-2.12.0)

Environment

UI endpoint: https://cornell-ptf.int.aws.folio.org/
Okapi endpoint: https://okapi-cornell-ptf.int.aws.folio.org/
Environment is configured to use shared MSK and ES

Created in INT account us-east-1 region, cluster name cptf2, created with snapshot of Cornell Test environment.

Task count: HA – okapi x3, mod-data-import, mod-data-export, mod-quick-marc, mod-data-export-spring x1, all other modules x2
OpenSearch: fse - shared domain (6 r6g.large.search datanodes)
MSK: dedicated cluster - total 4 brokers (kafka.m5.large)

RDS Configuration 1: db.r6g.8xlarge instance, Aurora PostgreSQL 13.9
RDS Configuration 2: db.r6g.xlarge instance, Aurora PostgreSQL 13.9
RDS Configuration 3: Aurora Serverless, min ACU: 0.5, max ACU: 128
RDS Configuration 4: Aurora Serverless, min ACU: 32, max ACU: 128

Browser not supported