Versions Compared

Old Version 5

changes.mady.by.user Hanna Tuhuzbayeva

Saved on

compared with

New Version 6

changes.mady.by.user Hanna Tuhuzbayeva

Saved on

Key

  • This line was added.
  • This line was removed.
  • Formatting was changed.

Overview

This page is created to investigate Aurora serverless performance by comparing DB xlarge, 8xlarge and Aurora serverless instance types under load running Data Import (DI) with Check-in Check-out (CICO) running as background. 

Summary


Tests

Job profiles - PTF - Create 2, PTF - Updates Success - 1

Scenario
DI Create 25K, DI Update 25K
DI Create 25K, DI Update 25K with CICO 20 users in parallel
CICO 20 users

...

Each scenario was tested on next configurations:

DB configuration
RDS

db.r6g.xlarge

db.r6g.8xlarge

Serverless

Serverless v2 (0.5 - 128 ACUs)

Serverless v2 (32 - 128 ACUs)


Results


Process/requestRDSServerless
xlarge8xlarge0.5-128 ACU32-128 ACU
DI Create18 min9 min11 min (-7 min)​14 min (+5 min)​
DI Update17 min10 min16 min (-1 min)​11 min  (+1 min)​
DI Create (with CICO in parallel)22 min9 min20 min (-2 min)​11 min   (+2 min)​
DI Update (with CICO in parallel)22 min, error12 min25 min, error* (+3 min)​12 min (0 min)​
CI, 95%0.680.7180.806 (+0.126 sec)0.794 (+0.076 sec)​
CO, 95%1.2471.2511.393 (+0.146 sec)​1.312 (+0.061 sec)​
CI (with DI Update in parallel), 95%1.361.1931.277 (-0.083 sec)​1.11 (-0.083 sec)​
CO (with DI Update in parallel), 95%2.3352.0622.321 (-0.014 sec)​1.963 (-0.099 sec)​

*Error - io.vertx.core.impl.NoStackTraceThrowable


Response time

db.r6g.xlarge DI with CICO

db.r6g.8xlarge DI with CICO

Image Modified

Serverless v2 (0.5 - 128 ACUs) DI without CICO

Image Modified

Serverless v2 (32 - 128 ACUs) DI without CICO

Service CPU Utilization

db.r6g.xlarge DI without CICO

Image Modified

db.r6g.xlarge DI with CICO

Image Modified

db.r6g.8xlarge DI without CICO

Image Modified

db.r6g.8xlarge DI with CICO

Image Modified

Serverless v2 (0.5 - 128 ACUs) DI without CICO

Serverless v2 (0.5 - 128 ACUs) DI with CICO

Serverless v2 (32 - 128 ACUs) DI without CICO

Serverless v2 (32 - 128 ACUs) DI with CICO

Memory Utilization

db.r6g.xlarge without DI CICO

Image Modified

db.r6g.xlarge DI with CICO

Image Modified

db.r6g.8xlarge DI without CICO

Image Modified

db.r6g.8xlarge DI with CICO

Image Modified

Serverless v2 (0.5 - 128 ACUs) DI without CICO

Image Modified

Serverless v2 (0.5 - 128 ACUs) DI with CICO

Image Modified

Serverless v2 (32 - 128 ACUs) DI without CICO

Image Modified

Serverless v2 (32 - 128 ACUs) DI with CICO

Image Modified

DB CPU Utilization

db.r6g.xlarge DI without CICO

Image Modified

db.r6g.xlarge DI with CICO

Image Modified

db.r6g.8xlarge DI without CICO

Image Modified

db.r6g.8xlarge DI with CICO

Image Modified

Serverless v2 (0.5 - 128 ACUs) DI without CICO

Image Modified

Serverless v2 (0.5 - 128 ACUs) DI with CICO

Image Modified

Serverless v2 (32 - 128 ACUs) DI without CICO

Image Modified

Serverless v2 (32 - 128 ACUs) DI with CICO

DB Connections

db.r6g.xlarge DI without CICO

Image Modified

db.r6g.xlarge DI with CICO

Image Modified

db.r6g.8xlarge DI without CICO

Image Modified

db.r6g.8xlarge DI with CICO

Image Modified


Serverless v2 (0.5 - 128 ACUs) DI without CICO

Image Modified

Serverless v2 (0.5 - 128 ACUs) DI with CICO

Image Modified

Serverless v2 (32 - 128 ACUs) DI without CICO

Image Modified

Serverless v2 (32 - 128 ACUs) DI with CICO

Image Modified

Appendix

Grafana dashboard

db.r6g.xlarge DI with CICO

http://carrier-io.int.folio.ebsco.com/grafana/d/SqzWB26nk/jmeter-performance-check-in-check-out?orgId=1&from=1690186885745&to=1690192449004&var-percentile=95&var-test_type=baseline&var-test=circulation_checkInCheckOut_Orchid_cptf2_4&var-env=int&var-grouping=1s&var-low_limit=250&var-high_limit=750&var-db_name=jmeter&var-sampler_type=All

db.r6g.8xlarge DI with CICO

http://carrier-io.int.folio.ebsco.com/grafana/d/SqzWB26nk/jmeter-performance-check-in-check-out?orgId=1&from=1690206826647&to=1690209403208&var-percentile=95&var-test_type=baseline&var-test=circulation_checkInCheckOut_Orchid_cptf2_4&var-env=int&var-grouping=1s&var-low_limit=250&var-high_limit=750&var-db_name=jmeter&var-sampler_type=All

Serverless v2 (0.5 - 128 ACUs) DI with CICO

http://carrier-io.int.folio.ebsco.com/grafana/d/SqzWB26nk/jmeter-performance-check-in-check-out?orgId=1&from=1689860038636&to=1689864442762&var-percentile=95&var-test_type=baseline&var-test=circulation_checkInCheckOut_Orchid_cptf2_4&var-env=int&var-grouping=1s&var-low_limit=250&var-high_limit=750&var-db_name=jmeter&var-sampler_type=All

Serverless v2 (32 - 128 ACUs) DI with CICO

http://carrier-io.int.folio.ebsco.com/grafana/d/SqzWB26nk/jmeter-performance-check-in-check-out?orgId=1&from=1689934923312&to=1689937778313&var-percentile=95&var-test_type=baseline&var-test=circulation_checkInCheckOut_Orchid_cptf2_4&var-env=int&var-grouping=1s&var-low_limit=250&var-high_limit=750&var-db_name=jmeter&var-sampler_type=All

Configuration

DI

Version of modules:

Source Record Manager Module (mod-source-record-manager-3.6.2)
Source Record Storage Module (mod-source-record-storage-5.6.5)
Inventory Module (mod-inventory-20.0.4)
Inventory Storage Module (mod-inventory-storage-26.0.0)
Inventory Update Module (mod-inventory-update-3.0.1)
Data Import Module (mod-data-import-2.7.1)
quickMARC (mod-quick-marc-3.0.0)

CICO

Version of modules:

Okapi (okapi-5.0.1)

users (mod-users-19.1.1)

Remote storage API module (mod-remote-storage-2.0.2)

Pubsub (mod-pubsub-2.9.1)

Patron Blocks Module (mod-patron-blocks-1.8.0)

Inventory Storage Module (mod-inventory-storage-26.0.0)

Inventory Module (mod-inventory-20.0.4)

feesfines (mod-feesfines-18.2.1)

Configuration (mod-configuration-5.9.1)

Circulation Storage Module (mod-circulation-storage-16.0.0)

Circulation Module (mod-circulation-23.5.4)

authtoken (mod-authtoken-2.13.0)

Environment

  • UI endpoint: https://cornell-ptf.int.aws.folio.org/
  • Okapi endpoint: https://okapi-cornell-ptf.int.aws.folio.org/
  • Environment is configured to use shared MSK and ES

  • Created in INT account us-east-1 region, cluster name cptf2, created with snapshot of Cornell Test environment.

    Task count: HA – okapi x3, mod-data-import, mod-data-export, mod-quick-marc, mod-data-export-spring x1, all other modules x2
    OpenSearch: fse - shared domain (6 r6g.large.search datanodes)
    MSK: dedicated cluster - total 4 brokers (kafka.m5.large)
    RDS Configuration 1: db.r6g.8xlarge instance, Aurora PostgreSQL 13.9
    RDS Configuration 2: db.r6g.xlarge instance, Aurora PostgreSQL 13.9 
    RDS Configuration 3: Aurora Serverless, min ACU: 0.5, max ACU: 128 
    RD