Versions Compared

Key

  • This line was added.
  • This line was removed.
  • Formatting was changed.

Table of Contents

...

  • RDS CPU utilization average
    • For both 45 minute and longevity tests RDS used max 867 connections. Without test it was 812 connections.
  • CPU (User) usage by broker
    • Common CPU utilization by broker during longevity was 18, 18, 16, 14% accordingly to broker distribution. Resource utilization decreased significantly after 6 hours 20 minutes (17:40 UTC) to 14, 12, 12, 11%

Notes:

  • The reason of degradation, which occurs after 6 hours of testing, has not yet been found.

...

During the test RDS grew from 32% to 40% in average. Spikes observed after 10 hours

RDS Database Connections

For both 45 minute and longevity tests RDS used max 867 connections. Without test it was 812 connections.

45 minute tests

Longevity test

CPU (User) usage by broker

Common CPU utilization by broker during all tests was 18, 15, 15, 13 % accordingly to broker distribution in 45 min tests

Common CPU utilization by broker during longevity was 18, 18, 16, 14% accordingly to broker distribution. Resource utilization decreased significantly after 6 hours 20 minutes (17:40 UTC) to 14, 12, 12, 11%

45 minute tests

Longevity test

Database load

45 minute tests

Longevity

Target Group 5XX Count

Longevity test 24 hours

Appendix

Infrastructure

PTF environment qecp1
  • 11 m6i.2xlarge EC2 instances located in US East (N. Virginia)us-east-1 
  • 1 instance of db.r6.xlarge database instance: Writer instance
  • MSK tenant [ kafka configurations]
    • 4 kafka.m5.2xlarge brokers in 2 zones
    • Apache Kafka version 2.8.0

    • EBS storage volume per broker 300 GiB

    • auto.create.topics.enable=true
    • log.retention.minutes=480
    • default.replication.factor=3

...

Expand
titleAll modules


ModuleTask Definition RevisionModule VersionTask CountMem Hard LimitMem Soft LimitCPU UnitsXmxMetaspace SizeMax Metaspace Size
mod-remote-storage3mod-remote-storage:3.2.024920447203960512512
mod-remote-storage - Sidecar 1N/Afolio-module-sidecar:1.0.0.312N/A665512012800
mod-ncip3mod-ncip:1.14.521024896076888128
mod-ncip - Sidecar 1N/Afolio-module-sidecar:1.0.0.312N/A665512012800
mod-finance-storage3mod-finance-storage:8.6.121024896070088128
mod-finance-storage - Sidecar 1N/Afolio-module-sidecar:1.0.0.312N/A665512012800
mod-agreements3mod-agreements:7.0.62159214880000
mod-agreements - Sidecar 1N/Afolio-module-sidecar:1.0.0.312N/A665512012800
mod-ebsconet3mod-ebsconet:2.2.02124810240700128256
mod-ebsconet - Sidecar 1N/Afolio-module-sidecar:1.0.0.312N/A665512012800
mod-organizations3mod-organizations:1.9.221024896076888128
mod-organizations - Sidecar 1N/Afolio-module-sidecar:1.0.0.312N/A665512012800
edge-sip22edge-sip2:3.2.521024896076888128
mod-settings3mod-settings:1.0.321024896076888128
mod-settings - Sidecar 1N/Afolio-module-sidecar:1.0.0.312N/A665512012800
mod-serials-management3mod-serials-management:1.0.322480231201792384512
mod-serials-management - Sidecar 1N/Afolio-module-sidecar:1.0.0.312N/A665512012800
mod-search3mod-search:3.2.6225922480014405121024
mod-search - Sidecar 1N/Afolio-module-sidecar:1.0.0.312N/A665512012800
mod-data-import3mod-data-import:3.1.112048184401292384512
mod-data-import - Sidecar 1N/Afolio-module-sidecar:1.0.0.312N/A665512012800
edge-dematic2edge-dematic:2.2.311024896076888128
mod-tags3mod-tags:2.2.021024896076888128
mod-tags - Sidecar 1N/Afolio-module-sidecar:1.0.0.312N/A665512012800
edge-courses2edge-courses:1.4.321024896076888128
mod-notify3mod-notify:3.2.021024896076888128
mod-notify - Sidecar 1N/Afolio-module-sidecar:1.0.0.312N/A665512012800
mod-inventory-update3mod-inventory-update:3.3.121024896076888128
mod-inventory-update - Sidecar 1N/Afolio-module-sidecar:1.0.0.312N/A665512012800
mod-configuration3mod-configuration:5.10.021024896076888128
mod-configuration - Sidecar 1N/Afolio-module-sidecar:1.0.0.312N/A665512012800
mod-orders-storage3mod-orders-storage:13.7.421024896070088128
mod-orders-storage - Sidecar 1N/Afolio-module-sidecar:1.0.0.312N/A665512012800
folio-keycloak3folio-keycloak:25.0.31240020480204800
edge-caiasoft2edge-caiasoft:2.2.421024896076888128
mod-licenses3mod-licenses:6.0.322480231201792384512
mod-licenses - Sidecar 1N/Afolio-module-sidecar:1.0.0.312N/A665512012800
mod-gobi3mod-gobi:2.8.121024896076888128
mod-gobi - Sidecar 1N/Afolio-module-sidecar:1.0.0.312N/A665512012800
mod-password-validator3mod-password-validator:3.2.12144012980768384512
mod-password-validator - Sidecar 1N/Afolio-module-sidecar:1.0.0.312N/A665512012800
mod-fqm-manager3mod-fqm-manager:2.0.501024896076888128
mod-fqm-manager - Sidecar 1N/Afolio-module-sidecar:1.0.0.312N/A665512012800
mod-bulk-operations3mod-bulk-operations:2.0.223072260001536384512
mod-bulk-operations - Sidecar 1N/Afolio-module-sidecar:1.0.0.312N/A665512012800
mod-graphql3mod-graphql:1.12.121024896076888128
mod-graphql - Sidecar 1N/Afolio-module-sidecar:1.0.0.312N/A665512012800
mod-finance3mod-finance:4.9.021024896076888128
mod-finance - Sidecar 1N/Afolio-module-sidecar:1.0.0.312N/A665512012800
mod-roles-keycloak3mod-roles-keycloak:1.4.521024896051288256
mod-roles-keycloak - Sidecar 1N/Afolio-module-sidecar:1.0.0.312N/A665512012800
mod-batch-print3mod-batch-print:1.1.021024896076888128
mod-batch-print - Sidecar 1N/Afolio-module-sidecar:1.0.0.312N/A665512012800
mod-copycat3mod-copycat:1.6.021024896076888128
mod-copycat - Sidecar 1N/Afolio-module-sidecar:1.0.0.312N/A665512012800
mod-lists4mod-lists:2.0.621024896076888128
mod-lists - Sidecar 1N/Afolio-module-sidecar:1.0.0.312N/A665512012800
mod-entities-links3mod-entities-links:3.0.22259224800144001024
mod-entities-links - Sidecar 1N/Afolio-module-sidecar:1.0.0.312N/A665512012800
mod-permissions6mod-permissions:6.5.02168415445121024384512
mod-permissions - Sidecar 1N/Afolio-module-sidecar:1.0.0.312N/A665512012800
pub-edge1pub-edge:2023.06.1421024896076800
mod-orders3mod-orders:12.8.822048174001024384512
mod-orders - Sidecar 1N/Afolio-module-sidecar:1.0.0.312N/A665512012800
kong9kong:3.7.1161446144204876888128
edge-patron2edge-patron:5.1.22102489625676888128
mod-users-keycloak3mod-users-keycloak:1.5.321024896051288256
mod-users-keycloak - Sidecar 1N/Afolio-module-sidecar:1.0.0.312N/A665512012800
edge-ncip2edge-ncip:1.10.121024896076888128
mgr-applications2mgr-applications:1.3.31102489651251288256
mod-users-bl3mod-users-bl:7.8.0214401152092288128
mod-users-bl - Sidecar 1N/Afolio-module-sidecar:1.0.0.312N/A665512012800
mgr-tenants2mgr-tenants:1.3.31102489651251288256
mod-invoice3mod-invoice:5.8.2214401152092288128
mod-invoice - Sidecar 1N/Afolio-module-sidecar:1.0.0.312N/A665512012800
mod-inventory-storage4mod-inventory-storage:27.1.424096369003076384512
mod-inventory-storage - Sidecar 1N/Afolio-module-sidecar:1.0.0.312N/A665512025600
mod-user-import3mod-user-import:3.8.021024896076888128
mod-user-import - Sidecar 1N/Afolio-module-sidecar:1.0.0.312N/A665512012800
mod-sender3mod-sender:1.12.021024896076888128
mod-sender - Sidecar 1N/Afolio-module-sidecar:1.0.0.312N/A665512012800
edge-oai-pmh2edge-oai-pmh:2.9.221512136010241440384512
mod-login-keycloak8mod-login-keycloak:1.4.421024896051288256
mod-login-keycloak - Sidecar 1N/Afolio-module-sidecar:1.0.0.312N/A665512012800
mgr-tenant-entitlements2mgr-tenant-entitlements:1.2.41102489651251288256
mod-data-export-worker3mod-data-export-worker:3.2.423072280002048384512
mod-data-export-worker - Sidecar 1N/Afolio-module-sidecar:1.0.0.312N/A665512012800
mod-rtac4mod-rtac:3.6.021024896076888128
mod-rtac - Sidecar 1N/Afolio-module-sidecar:1.0.0.312N/A665512025600
mod-circulation-storage4mod-circulation-storage:17.2.322880259201814384512
mod-circulation-storage - Sidecar 1N/Afolio-module-sidecar:1.0.0.312N/A665512025600
mod-source-record-storage4mod-source-record-storage:5.8.625600500003500384512
mod-source-record-storage - Sidecar 1N/Afolio-module-sidecar:1.0.0.312N/A665512012800
mod-scheduler3mod-scheduler:1.2.321024896051288256
mod-scheduler - Sidecar 1N/Afolio-module-sidecar:1.0.0.312N/A665512012800
mod-calendar4mod-calendar:3.1.0220481536076888128
mod-calendar - Sidecar 1N/Afolio-module-sidecar:1.0.0.317N/A665512012800
mod-event-config3mod-event-config:2.7.121024896076888128
mod-event-config - Sidecar 1N/Afolio-module-sidecar:1.0.0.312N/A665512012800
mod-courses3mod-courses:1.4.1021024896076888128
mod-courses - Sidecar 1N/Afolio-module-sidecar:1.0.0.312N/A665512012800
mod-inventory5mod-inventory:20.2.722881259201814384512
mod-inventory - Sidecar 1N/Afolio-module-sidecar:1.0.0.312N/A665512012800
mod-email3mod-email:1.17.022800255001800384512
mod-email - Sidecar 1N/Afolio-module-sidecar:1.0.0.312N/A665512012800
mod-di-converter-storage3mod-di-converter-storage:2.2.221024896076888128
mod-di-converter-storage - Sidecar 1N/Afolio-module-sidecar:1.0.0.312N/A665512012800
mod-pubsub3mod-pubsub:2.14.02153614400922384512
mod-pubsub - Sidecar 1N/Afolio-module-sidecar:1.0.0.312N/A665512012800
mod-circulation4mod-circulation:24.2.522880259201814384512
mod-circulation - Sidecar 1N/Afolio-module-sidecar:1.0.0.312N/A665512025600
edge-orders2edge-orders:3.0.321024896076888128
edge-rtac2edge-rtac:2.7.321024896076888128
mod-template-engine3mod-template-engine:1.20.021024896076888128
mod-template-engine - Sidecar 1N/Afolio-module-sidecar:1.0.0.312N/A665512012800
mod-users3mod-users:19.3.121024896076888128
mod-users - Sidecar 1N/Afolio-module-sidecar:1.0.0.312N/A665512012800
mod-patron-blocks3mod-patron-blocks:1.10.021024896076888128
mod-patron-blocks - Sidecar 1N/Afolio-module-sidecar:1.0.0.312N/A665512012800
mod-audit3mod-audit:2.9.021024896076888128
mod-audit - Sidecar 1N/Afolio-module-sidecar:1.0.0.312N/A665512012800
edge-fqm2edge-fqm:2.0.221024896076888128
mod-source-record-manager3mod-source-record-manager:3.8.625600500003500384512
mod-source-record-manager - Sidecar 1N/Afolio-module-sidecar:1.0.0.312N/A665512012800
nginx-edge1nginx-edge:2023.06.14110248960000
mod-quick-marc3mod-quick-marc:5.1.112288217601664384512
mod-quick-marc - Sidecar 1N/Afolio-module-sidecar:1.0.0.312N/A665512012800
mod-feesfines3mod-feesfines:19.1.021024896076888128
mod-feesfines - Sidecar 1N/Afolio-module-sidecar:1.0.0.312N/A665512012800
mod-invoice-storage3mod-invoice-storage:5.8.221872153601024384512
mod-invoice-storage - Sidecar 1N/Afolio-module-sidecar:1.0.0.312N/A665512012800
mod-service-interaction3mod-service-interaction:4.0.222048184401290384512
mod-service-interaction - Sidecar 1N/Afolio-module-sidecar:1.0.0.312N/A665512012800
mod-data-export11mod-data-export:5.0.412592248001440881024
mod-data-export - Sidecar 1N/Afolio-module-sidecar:1.0.0.312N/A665512012800
mod-patron3mod-patron:6.1.021024896076888128
mod-patron - Sidecar 1N/Afolio-module-sidecar:1.0.0.312N/A665512012800
mod-oai-pmh3mod-oai-pmh:3.13.224096369003076384512
mod-oai-pmh - Sidecar 1N/Afolio-module-sidecar:1.0.0.312N/A665512012800
edge-connexion2edge-connexion:1.3.121024896076888128
mod-kb-ebsco-java3mod-kb-ebsco-java:4.0.021024896076888128
mod-kb-ebsco-java - Sidecar 1N/Afolio-module-sidecar:1.0.0.312N/A665512012800
mod-notes3mod-notes:5.2.0210248960952384512
mod-notes - Sidecar 1N/Afolio-module-sidecar:1.0.0.312N/A665512012800
mod-organizations-storage3mod-organizations-storage:4.7.021024896076888128
mod-organizations-storage - Sidecar 1N/Afolio-module-sidecar:1.0.0.312N/A665512012800
mod-data-export-spring3mod-data-export-spring:3.2.212048184401536384512
mod-data-export-spring - Sidecar 1N/Afolio-module-sidecar:1.0.0.312N/A665512012800
edge-erm2edge-erm:1.2.121024896076888128



Methodology/Approach

  • Steps to avoid possible session or token expiration:
    •  modify tenant/realm-settings/tokens in keycloaks UI to 24 hours (1 day)
    •  modify tenant/realm-settings/sessions in keycloaks UI to 24 hours (1 day) Default - 10 hours
  • Run data preparation script before each CI/CO test
  • Update .jmx file script for Quesnelia release
  • Create  artefact and upload to carrier-io
  • Use Jenkins job to change parameters and run tests
  • Test CI/CO with 8, 20, 30, 75 concurrent users for 45 minutes each. 
  • Test CI/CO with 30 users for 24 hours to detect any trends in memory. 

...