Skip to end of banner
Go to start of banner

Check-Out by users with 2000 loans each (Nolana)

Skip to end of metadata
Go to start of metadata

You are viewing an old version of this page. View the current version.

Compare with Current View Page History

« Previous Version 4 Current »

Overview

The test goal is to assess the performance of circulation Check-Out functionality for users with 2000 loans each.

Ticket: PERF-243 - Getting issue details... STATUS


Summary

Load tests showed that there is performance degradation for 1 - 8 concurrent users tests users with 2000 loans each compared to those without loans.

For 20 concurrent users time of CO is almost the same for both but the degradation of API response time GET_circulation/loans (Submit_barcode_checkout) and GET_circulation/loans (Submit_patron_barcode) is about 5.6%.

Test Runs 

Test #

Test Conditions

Duration 

Load generator size (recommended)Load generator Memory(GiB) (recommended)

Notes


1.

Checkout with 1, 5, 8, and 20 users

30 mint3.medium3

ncp3 environment


Results

Response Times

Baseline (users without loans)


Verification (users with 2000 loans)

At the end of 5 users' test, the response time of CO was 20 s

Checkout response time comparison


Users quantity

Response time 95prc, sec


Degradation, sec


Degradation, %

Response time Avg, sec


Degradation, sec


Degradation, %

Baseline

(users without loans)

Verification

(users with 2000 loans each)

Baseline

(users without loans)

Verification

(users with 2000 loans each)

1 user1.1461.2690.123 s10.73%0.9501.0150.065 s6.84%
5 users1.0041.0710.067 s6.67%0.8520.9080.056 s6.57%
8 users0.9460.9950.049 s5.18%0.8370.8840.047 s5.62%
20 users0.9320.928-0.004 s-0.43%0.8500.85000.00%

API with degraded response time 95prc, all tests degradation


Degradation, %
API1 user5 users8 users20 users
GET_circulation/loans (Submit_barcode_checkout)5.61%18.62%8.70%5.67%
GET_circulation/loans (Submit_patron_barcode)10.16%18.00%9.35%5.67%

Instance CPU Utilization

Baseline (users without loans)


Verification (users with 2000 loans)


Service CPU Utilization

Baseline (users without loans)


Verification (users with 2000 loans)

Memory Utilization

Baseline (users without loans)

mod-circulation memory utilization increases from 71% to 73%


Verification (users with 2000 loans)

mod-circulation and mod-circulation-storage were restarted before the tests as you can see at the lowest memory utilization percent and then grow reaching baseline numbers.

RDS CPU Utilization

Baseline (users without loans)



Verification (users with 2000 loans)

RDS DB connections

Baseline (users without loans)


Verification (users with 2000 loans)


Database Load

Baseline (users without loans)


Verification (users with 2000 loans)


Appendix

Infrastructure

PTF -environment ncp3


  • 10 m6i.2xlarge EC2 instances located in US East (N. Virginia)us-east-1
  • 2 instances of db.r6.xlarge database instances: Writer & reader instances
  • MSK ptf-kakfa-3 [ kafka configurations]
    • 4 kafka.m5.2xlarge brokers in 2 zones
    • Apache Kafka version 2.8.0

    • EBS storage volume per broker 300 GiB

    • auto.create.topics.enable=true
    • log.retention.minutes=480
    • default.replication.factor=3


Modules memory and CPU parameters:

Modules

Version

Task Definition

Running Tasks 

CPU

Memory (Soft/Hard limits)

MaxMetaspaceSize

Xmx

okapi4.14.7131024

1440/1684

512922
mod-feesfines18.1.132128896/1024128768
mod-patron-blocks1.7.1421024896/1024128768
mod-pubsub2.7.0421024

1440/1536

512922
mod-authtoken2.12.032

512

1152/1440

128

922

mod-circulation-storage15.0.2321024

1440/1536

512896
mod-circulation23.3.2321024896/1024128768
mod-configuration5.9.032128896/1024128768
mod-users19.0.042128896/1024128768
mod-remote-storage1.7.1321281692/18725121178


Methodology/Approach

  1. Conduct necessary commands to return the database to the initial state. Do this before each test run. Wait several minutes before the test start.
  2. Conduct check-out load tests with different numbers of users.
  3. Set a delay of all circulation_ and circulation-storage_  okapi timers to 0 (to avoid background jobs starting). Repeat tests with the same approach but before each test also generates 2000 loans for each user by running the command ./high_circ-data-load.sh psql_[environment name].conf [tenant].
  4. Compare test results.

Grafana dashboard

Baseline (users without loans): http://carrier-io.int.folio.ebsco.com/grafana/d/elIt9zCnz/jmeter-performance-test-copy?orgId=1&from=1675932840092&to=1675949459000&var-percentile=95&var-test_type=baseline&var-test=circulation_CheckOut_nolana&var-env=int&var-grouping=1s&var-low_limit=250&var-high_limit=750&var-db_name=jmeter&var-sampler_type=All

Verification (users with 2000 loans): http://carrier-io.int.folio.ebsco.com/grafana/d/elIt9zCnz/jmeter-performance-test-copy?orgId=1&from=1675857341711&to=1675878170222&var-percentile=95&var-test_type=baseline&var-test=circulation_CheckOut_nolana&var-env=int&var-grouping=1s&var-low_limit=250&var-high_limit=750&var-db_name=jmeter&var-sampler_type=All

API response comparison: https://docs.google.com/spreadsheets/d/1560SZ7vb9aNip0sBsUesXbaKhn0l5HzDd7_Ts8TxJGQ/edit#gid=1126827422

Please note that dashboards will expire in 6 weeks after test runs.



  • No labels