Skip to end of banner
Go to start of banner

[Poppy] List App with single tenant and R/W split enabled

Skip to end of metadata
Go to start of metadata

You are viewing an old version of this page. View the current version.

Compare with Current View Page History

« Previous Version 7 Next »

Overview

This document contains the results of testing List App refreshing of 200k records on single tenant with R/W split enabled (Poppy release). The goal of testing is to assess the performance of mod-lists with load spread across multiple tenants.

Ticket: PERF-718 - Getting issue details... STATUS

Summary

  • Load tests showed that single tenant ListApp refreshing duration is:
    • 2.6 min for 1 user test;
    • 3.1 min for 3 user test;
    • 4.2 min for 5 user test;
    • 6.5 min for 10 user test.
    • Duration is below the maximum accepted value of 10 minutes.
  • Load tests showed that ListApp refreshing duration increases 2-3 times when conducted with the same user number but on 1 tenant instead of 3 tenants. For example, during 3 users test duration increased from 1.5 min to 3.1 min, for 10 users - from 2.3 min to 6.5 min.
  • During the 10 users test CPU utilization reached 89% for mod-fqm-manager and 101% for mod-lists. In comparison with multiple tenants testing, CPU utilization for mod-fqm-manager is about 2 times lower for single tenant test.
  • Maximum DB CPU utilization reached 63% (writer instance) and 99% (reader instance) during 10 user test. In comparison with multiple tenants testing, RDS CPU utilization for writer instance is 20% lower for single tenant test.
  • No memory leak is suspected for all the modules.

Test runs

Query used in lists - "Item status != Available". List refresh result is about 200K records.

Scenario

Data quantity

List App refresh

single tenant


1 user
3 users
5 users
10 users

Results

ListApp refresh, avg1 user3 users5 users10 users
Poppy, single tenant, R/W split enabled2.6 min3.1 min4.2 min6.5 min
Poppy, multiple tenants, R/W split enabled*
1.5 min
2.3 min

*Results are taken from previous test report: [Poppy] List App with multiple tenants and R/W split enabled

Service CPU Utilization

During the 10 users test CPU utilization reached 89% for mod-fqm-manager and 101% for mod-lists.

In comparison with multiple tenants testing, CPU utilization for mod-fqm-manager is about 2 times lower for single tenant test.

Memory Utilization

No memory leak is suspected for all the modules.

DB CPU Utilization

Maximum DB CPU utilization reached 63% (writer instance) and 99% (reader instance) during 10 user test. 

In comparison with multiple tenants testing, RDS CPU utilization for writer instance is 20% lower for single tenant test.

DB Connections

DB Load

TOP SQL

Writer DB node

Reader DB node


Long-running queries: 

select id from [tenant]_mod_fqm_manager.drv_item_callnumber_location where lower(cast(item_status as varchar)) <> lower($1)
parameters: $1 = 'Available'


delete from list_contents where list_id=$1 and refresh_id=$2

Appendix

Infrastructure

PTF -environment pcp1

  • 10 m6i.2xlarge EC2 instances located in US East (N. Virginia)us-east-1

  • 1 database  instance, writer

    NameAPI NameMemory GIBvCPUsmax_connections
    R6G Extra Largedb.r6g.xlarge32 GiB4 vCPUs2731
  • MSK tenant
    • 4 m5.2xlarge brokers in 2 zones

    • Apache Kafka version 2.8.0

    • EBS storage volume per broker 300 GiB

    • auto.create.topics.enable=true
    • log.retention.minutes=480
    • default.replication.factor=3

Module
pcp1-pvt
Fri Oct 27 08:26:47 UTC 2023

Task Def. Revision

Task Count

Mem Hard Limit

Mem Soft limit

CPU units

Xmx

MetaspaceSize

MaxMetaspaceSize

R/W split enabled

mod-inventory-storage:27.0.01024096369020483076384512false
mod-users:19.2.0192102489612876888128false
nginx-okapi:2023.06.14821024896128000false
mod-circulation-storage:17.1.01022880259215361814384512false
okapi:5.1.193168414401024922384512false
mod-inventory:20.1.0922880259210241814384512false
mod-circulation:24.0.01022880259215361814384512false
pub-okapi:2023.06.1482102489612876800false
mod-fqm-manager:1.0.052102489612876888128true
mod-lists:1.0.052300026001282048384512false

Methodology

  1. Enable R/W split for mod-fqm-manager.
  2. Create 10 lists with the query "Item status != Available" on each of three tenants to be able to run a test for up to 30 concurrent users.
  3. Prepare 200K item records for the query to return.
  4. Conduct tests with JMeter script (single tenant).
  • No labels