Skip to end of banner
Go to start of banner

MaxRAMPercentage vs Xmx with Data Import and Check-ins Check-outs (Poppy)

Skip to end of metadata
Go to start of metadata

You are viewing an old version of this page. View the current version.

Compare with Current View Page History

Version 1 Next »

Overview

This document contains the results of testing different configurations of modules  with changes in JAVA_OPTS replacing XMX with MaxRAMPercentage. Memory utilization, Check-in/Check-out response times and Data Import durations for MARC Bibliographic records should be compared.

Ticket: PERF-893 - Getting issue details... STATUS

Summary



Test Runs 

Test #

Scenario

Load levelJAVA_OPTS
120 vUsers longevity CI/CO + DI MARC Bib Create12 Data Import tests with 100K sequentially (with 5 min pause)
-XMX
220 vUsers longevity CI/CO + DI MARC Bib Create12 Data Import tests with 100K sequentially (with 5 min pause)
-XX:MaxRAMPercentage, Default
320 vUsers longevity CI/CO + DI MARC Bib Create12 Data Import tests with 100K sequentially (with 5 min pause)-XX:MaxRAMPercentage, Recommended - 66%

Test Results

Comparison

The following table compares test results 

 BASELINE, XMX   STEP #1 default MaxPercentage 

STEP #2 recommended MaxPercentage, 66%
Baseline 16 hours CICO
hh:mm:ss
24 hours CICO
hh:mm:ssdelta, default - xmx x hours CICO
hh:mm:ssdelta, 66% - xmx

















DI 100k Create #1101:39:094 jobs stucked
DI 100k Create #11






DI 100k Create #1001:03:19

DI 100k Create #10






DI 100k Create #900:59:40

DI 100k Create #9


DI 100k Create #800:55:01

DI 100k Create #800:58:4700:03:46
DI 100k Create #800:55:2300:00:22

DI 100k Create #700:54:43

DI 100k Create #700:58:0800:03:26
DI 100k Create #702:46:273 jobs stucked

DI 100k Create #600:55:25

DI 100k Create #600:56:0900:00:44
DI 100k Create #601:03:0700:07:42

DI 100k Create #500:54:45

DI 100k Create #500:58:1900:03:33
DI 100k Create #501:03:3700:08:52

DI 100k Create #400:55:38

DI 100k Create #400:54:2700:01:11
DI 100k Create #400:59:1100:03:33

DI 100k Create #300:53:30

DI 100k Create #300:54:3600:01:06
DI 100k Create #300:58:3400:05:04

DI 100k Create #200:53:16

DI 100k Create #200:53:0300:00:13
DI 100k Create #200:57:5900:04:43

DI 100k Create #100:52:51

DI 100k Create #101:05:1000:12:18
DI 100k Create #101:05:4200:12:50













DI +DI -

DI +DI -


DI +DI -
Check-Ins1162538
Check-Ins985467

Check-Ins943469
Check-Outs1840978
Check-Outs1476809

Check-Outs1408834

Detailed CICO response time for CICO with DI

Request*Response time (avg, sec)
Pure CICOCICO + 100K MARC BIB Create




































*Top-10 requests were taken for analysis.

Response time

DI MARC BIB Create + CICO



Service CPU Utilization


DI MARC BIB Update+ CICO


Service Memory Utilization


DI MARC BIB Create + CICO



DB CPU Utilization

DI MARC BIB Create + CICO


DB Connections

DI MARC BIB Create + CICO



DB load

DI MARC BIB Create + CICO



Top SQL-queries:


Appendix

Infrastructure

PTF -environment pcp1

  • 10 m6i.2xlarge EC2 instances located in US East (N. Virginia)us-east-1

  • 2 database  instances, writer/reader

    NameMemory GIBvCPUsmax_connections

    db.r6g.xlarge

    		32 GiB4 vCPUs2731
  • MSK tenant
    • 4 m5.2xlarge brokers in 2 zones

    • Apache Kafka version 2.8.0

    • EBS storage volume per broker 300 GiB

    • auto.create.topics.enable=true
    • log.retention.minutes=480
    • default.replication.factor=3
ModuleTask Def. RevisionTask Def. Revision with MaxRAMPercentage = by defaultTask Def. Revision with MaxRAMPercentage = 66.0Module VersionTask CountMem Hard LimitMem Soft limitCPU unitsXmxMetaspaceSizeMaxMetaspaceSizeR/W split enabledMaxRAMPercentageMaxRAMPercentage = recommended (66%)
pcp1-pvt













Thu Apr 04 09:40:28 UTC 2024













mod-data-import202324mod-data-import:3.0.71204818442561292384512FALSE70.166.0
mod-authtoken161718mod-authtoken:2.14.121440115251292288128FALSE80.066.0
mod-configuration101213mod-configuration:5.9.22102489612876888128FALSE85.766.0
mod-users-bl101112mod-users-bl:7.6.021440115251292288128FALSE80.066.0
mod-inventory-storage151718mod-inventory-storage:27.0.424096369020483076384512FALSE83.466.0
mod-circulation-storage141516mod-circulation-storage:17.1.722880259215361814384512FALSE70.066.0
mod-source-record-storage181920mod-source-record-storage:5.7.525600500020483500384512FALSE70.066.0
mod-inventory151617mod-inventory:20.1.822880259210241814384512FALSE70.066.0
mod-di-converter-storage181920mod-di-converter-storage:2.1.52102489612876888128FALSE85.766.0
mod-circulation141516mod-circulation:24.0.1122880259215361814384512FALSE70.066.0
mod-pubsub131415mod-pubsub:2.11.32153614401024922384512FALSE64.066.0
mod-users343536mod-users:19.3.0-SNAPSHOT.6772102489612876888128FALSE85.766.0
mod-patron-blocks101112mod-patron-blocks:1.9.021024896102476888128FALSE85.766.0
mod-source-record-manager171920mod-source-record-manager:3.7.825600500020483500384512FALSE70.066.0
nginx-okapi9

nginx-okapi:2023.06.1421024896128000FALSE0.0
okapi-b111213okapi:5.1.23168414401024922384512FALSE64.066.0
mod-feesfines111213mod-feesfines:19.0.02102489612876888128FALSE85.766.0

Methodology/Approach

  1. Run baseline for Check-in / Check-out longevity test with Data import Create job with 100k file.
  2. Change revisions of modules with needed configuration of JAVA_OPTS - replaced XMX with MaxRAMPercentage in aa
  3. Run CI/CO
  4. Run DI script after ramp-up period for CI/CO or at least 10-15 minutes


  • No labels