Versions Compared

Version Old Version 15 New Version Current
Changes made by

Stanislav Nehrii

Stanislav Nehrii

Saved on

Key

  • This line was added.
  • This line was removed.
  • Formatting was changed.

Table of Contents
outlinetrue

Overview

  • This document contains the results of testing  Data testing Check-in/Check-out and Data Import for MARC Bibliographic records with an update job  on the Quesnelia [ECS] release on qcon environment. 

Jira Legacy
serverSystem Jira
columnIdsissuekey,summary,issuetype,created,updated,duedate,assignee,reporter,priority,status,resolution
columnskey,summary,type,created,updated,due,assignee,reporter,priority,status,resolution
serverId01505d01-b853-3c2e-90f1-ee9b165564fc
keyPERF-846
 

Summary

  • Data Import with Check In/Check Out tests finished successfully on qcon environment using the PTF - Create 2 and PTF - Updates Success - 2 profile and file with 25k records profiles with files 5K, 10K, 25K, 50K, 100K records.
  • Comparing with previous testing results Poppy and Quesnelia releases
    • Data Import processed all jobs including test on 3 tenants concurrently without errors durations for create jobs has performance degradation in Average up to 10% for Quesnelia releases.
    • Data Import durations stayed in the same time range in Average for Quesnelia releases but it works stable and without errorsdurations for update jobs has performance degradation in Average up to 40% for Quesnelia releases.
    • Check In/Check Out Response time has slight improvement in Average up to 10% for Quesnelia releases.
  • During testing, we noticed that mod-permission did not have any spikes and used 12% data-import module used maximum 57% CPU for Quesnelia releases. For Poppy releases we had errorwe mod-data-import module used 130% CPU and 320% in spike.
  • No memory leaks are observed.

Test Runs 

Test №ScenarioTest ConditionsResults

1		DI MARC Bib Create5K, 10K, 25K, 50K, 100K consequentially (with 5 min pause)
Completed
CICO 8 users

2		DI MARC Bib Update5K, 10K, 25K, 50K, 100K consequentially (with 5 min pause)


Completed

CICO8 users

...

ProfileMARC File

DI Duration

Quesnelia (hh:mm:ss)

Check In, Check Out Response time (8 users)

Quesnelia

CI Average secCO Average sec

DI MARC Bib Create

(PTF - Create 2)

5K.mrc0:03:210.8311.357
10K.mrc0:06:510.8451.410
25K.mrc0:12:410.7191.333
50K.mrc0:23:190.6911.327
100K.mrc0:51:240.6641.335

DI MARC Bib Create

(PTF - Updates Success - 2)

5K.mrc0:04:120.7641.458
10K.mrc0:08:150.7791.377
25K.mrc0:20:380.7551.401
50K.mrc0:43:060.7501.444
100K.mrc1:29:090.7301.458


Check-in/Check-out without DI


ScenarioLoad levelRequestResponse time, sec
Quesnelia
95 percaverage
Circulation Check-in/Check-out
(without Data import)		8 usersCheck-in0.6350.493
Check-out1.2431.078


Comparison

This table contains DI durations with CICO comparison between Poppy and Quesnelia releases.


ProfileMARC FileDI Durationwith CI/CODI Delta  
Poppy/Quesnelia
(hh:mm:ss / %)		Check In, Check Out
Response time
(8 users)		Check In, Check Out
Response time
(8 users)		Delta, %

PoppyQuesneliaPoppy/QuesneliaPoppy/Quesnelia
PoppyQuesneliaCI Average secCO Average secCI Average secCO Average secCICO
DI MARC Bib Create
(PTF - Create 2)		5K.mrc00:02:530:03:21+0:00:28
10K.
/ +16.18%0.9011.3750.8311.357-7.77%-1.31%
10K.mrc00:04:320:06:51+0:02:19/ +51.10%0.9021.470.8451.410-6.32%-4.08%
25K.mrc00:11:140:12:41+0:01:27
50K.mrc00:21:550:23:19+0:01:24100K
/ +12.91%11.5710.7191.333-28.1%-15.15%
50K.mrc00:
47
21:
02
550:
51
23:
24
19+0:
04:22

DI MARC Bib Create

(PTF - Updates Success - 2)

5K
01:24 / +6.39%0.9811.460.6911.327-29.57%-9.11%
100K.mrc00:
03
47:
19
020:
04
51:
12
24+0:
00:5310K.mrc00:06:200:08:15+0:01:5525K.mrc00:14:040:20:38+0:06:3450K
04:22 / +9.28%1.0181.4910.6641.335-34.78%-10.47%

DI MARC Bib Create

(PTF - U1.571pdates Success - 2)

5K.mrc00:
29
03:
59
190:
43
04:
06
12+0:
13:07100K
00:53 /+26.63%0.7551.1690.7641.458+1.19%+24.73%
10K.mrc
01
00:
03
06:
03
20
1
0:
29
08:
09
15+0:
26:06

Detailed CICO response time comparison without DI

ScenarioLoad levelRequestResponse time, sec
PoppyResponse time, sec
Quesnelia95 percaverage95 percaverageCirculation Check-in/Check-out (without Data import)8 usersCheck-inCheck-out

Detailed CICO response time comparison for CICO with DI in Poppy

Request*Response time (avg, sec)Pure CICOCICO + 100K MARC BIB CreateCICO + 100K MARC BIB UpdateCheck-Out Controller0.8281.4911.51Check-In Controller0.4311.0180.908POST_circulation/check-out-by-barcode (Submit_barcode_checkout)0.2660.6470.718POST_circulation/check-in-by-barcode (Submit_barcode_checkin)0.1870.570.477GET_circulation/loans (Submit_barcode_checkout)0.1280.2330.215GET_inventory/items (Submit_barcode_checkin)0.0480.1260.118GET_inventory/items (Submit_barcode_checkout)0.0460.1250.117GET_note-links (Submit_barcode_checkout)0.0460.0240.024GET_users (Submit_patron_barcode)0.0370.0410.037GET_circulation/loans (Submit_patron_barcode)0.0280.030.049GET_automated-patron-blocks (Submit_patron_barcode)0.0240.0260.024GET_users (Get_check_in_page)0.0230.0540.051*Top-10 requests were taken for analysis.
01:55 / +30.26%0.751.3070.7791.377+3.87%+5.36%
25K.mrc00:14:040:20:38+0:06:34 / +46.68%0.8221.4030.7551.401-8.15%-0.14%
50K.mrc00:29:590:43:06+0:13:07 / +43.74%0.8931.4240.7501.444-16.01%+1.40%
100K.mrc01:03:031:29:09+0:26:06 / +41.40%0.9081.510.7301.458-19.60%-3.44%


Detailed CICO response time comparison without DI


ScenarioLoad levelRequestResponse time, sec
Poppy		Response time, sec
Quesnelia
95 percaverage95 percaverage
Circulation Check-in/Check-out (without Data import)8 usersCheck-in0.4890.4310.6350.493
Check-out0.9690.8281.2431.078



Resource utilization for Test №1

...

Expand
titleResource utilization table


CPURAMCPU
nginx-okapi67%mod-data-import-b72%
mod-data-import-b50%mod-inventory-b66%
okapi-b42%mod-source-record-manager-b52%
mod-di-converter-storage-b41%mod-source-record-storage-b45%
mod-source-record-storage-b21%mod-pubsub-b35%
mod-inventory-storage-b20%okapi-b35%
mod-source-record-manager-b9%mod-di-converter-storage-b34%
mod-quick-marc-b9%mod-feesfines-b33%
mod-feesfines-b9%mod-patron-blocks-b32%
mod-authtoken-b9%mod-quick-marc-b31%
mod-pubsub-b7%mod-circulation-storage-b30%
mod-configuration-b6%mod-configuration-b29%
pub-okapi4%mod-users-bl-b28%
mod-circulation-storage-b2%mod-circulation-b28%
mod-remote-storage-b2%mod-inventory-storage-b22%
mod-inventory-update-b2%mod-authtoken-b21%
mod-circulation-b1%mod-remote-storage-b18%
mod-patron-blocks-b1%nginx-okapi4%
mod-users-bl-b0.8%pub-okapi4%


...

PTF - environment Quesnelia (qcon)

  • 11 m6i10 m6i.2xlarge EC2 instances located in US East (N. Virginia)us-east-1 [Number of ECS instances, instance type, location region]

  • 1 instance of db.r6.xlarge database instance: Writer instance

  • OpenSearch

    • domain: fse

    • Number of nodes: 96

    • Version: OpenSearch_2_7_R20240502

  • MSK - tenat

    • 4 kafka.m5.2xlarge brokers in 2 zones

    • Apache Kafka version 2.8.0

    • EBS storage volume per broker 300 GiB

    • auto.create.topics.enable=true

    • log.retention.minutes=480

    • default.replication.factor=3

    • Kafka consolidated topics enabled

...

Expand
titleQuesnelia modules memory and CPU parameters


ModuleTask Def. RevisionModule VersionTask CountMem Hard LimitMem Soft limitCPU unitsXmxMetaspaceSizeMaxMetaspaceSize
qcon-pvt








Thu May 23 10:47:25 UTC 2024








mod-remote-storage1579891902283.dkr.ecr.us-east-1.amazonaws.com/folio/-remote-storage1mod-remote-storage:3.2.024920447210243960512512
mod-finance-storage1579891902283.dkr.ecr.us-east-1.amazonaws.com/folio/mod-finance-storage:8.6.021024896102470088128
mod-ncip1579891902283.dkr.ecr.us-east-1.amazonaws.com/folio/mod-ncip:1.14.42102489612876888128
mod-agreements1579891902283.dkr.ecr.us-east-1.amazonaws.com/folio/mod-agreements:7.0.0215921488128000
mod-ebsconet1579891902283.dkr.ecr.us-east-1.amazonaws.com/folio/mod-ebsconet:2.2.0212481024128700128256
mod-consortia1579891902283.dkr.ecr.us-east-1.amazonaws.com/folio/mod-consortia:1.1.023072204812820485121024
mod-organizations1579891902283.dkr.ecr.us-east-1.amazonaws.com/folio/mod-organizations:1.9.02102489612870088128
mod-serials-management1579891902283.dkr.ecr.us-east-1.amazonaws.com/folio/mod-serials-management:1.0.02248023121281792384512
mod-settings1579891902283.dkr.ecr.us-east-1.amazonaws.com/folio/mod-settings:1.0.32102489620076888128
mod-search9579891902283.dkr.ecr.us-east-1.amazonaws.com/folio/-search9mod-search:3.3.0-SNAPSHOT.224225922480204814405121024
edge-dematic1579891902283.dkr.ecr.us-east-1.amazonaws.com/folio/edge-dematic:2.2.01102489612876888128
mod-data-import1579891902283.dkr.ecr.us-east-1.amazonaws.com/folio/mod-data-import:3.1.01204818442561292384512
mod-tags1579891902283.dkr.ecr.us-east-1.amazonaws.com/folio/mod-tags:2.2.02102489612876888128
mod-authtoken3579891902283.dkr.ecr.us-east-1.amazonaws.com/folio/mod-authtoken:2.15.121440115251292288128
edge-courses1579891902283.dkr.ecr.us-east-1.amazonaws.com/folio/edge-courses:1.4.02102489612876888128
mod-inventory-update1579891902283.dkr.ecr.us-east-1.amazonaws.com/folio/mod-inventory-update:3.3.02102489612876888128
mod-notify1579891902283.dkr.ecr.us-east-1.amazonaws.com/folio/mod-notify:3.2.02102489612876888128
mod-configuration1579891902283.dkr.ecr.us-east-1.amazonaws.com/folio/mod-configuration:5.10.02102489612876888128
mod-orders-storage1579891902283.dkr.ecr.us-east-1.amazonaws.com/folio/-orders-storage1mod-orders-storage:13.7.02102489651270088128
edge-caiasoft1579891902283.dkr.ecr.us-east-1.amazonaws.com/folio/edge-caiasoft:2.2.02102489612876888128
mod-login-saml1579891902283.dkr.ecr.us-east-1.amazonaws.com/folio/mod-login-saml:2.8.02102489612876888128
mod-erm-usage-harvester1579891902283.dkr.ecr.us-east-1.amazonaws.com/folio/mod-erm-usage-harvester:4.5.02102489612876888128
mod-password-validator1579891902283.dkr.ecr.us-east-1.amazonaws.com/folio/mod-password-validator:3.2.0214401298128768384512
mod-licenses1579891902283.dkr.ecr.us-east-1.amazonaws.com/folio/mod-licenses:6.0.02248023121281792384512
mod-gobi1579891902283.dkr.ecr.us-east-1.amazonaws.com/folio/mod-gobi:2.8.02102489612870088128
mod-bulk-operations1579891902283.dkr.ecr.us-east-1.amazonaws.com/folio/mod-bulk-operations:2.0.023072260010241536384512
mod-fqm-manager1579891902283.dkr.ecr.us-east-1.amazonaws.com/folio/mod-fqm-manager:2.0.12300026001282048384512
edge-dcb1579891902283.dkr.ecr.us-east-1.amazonaws.com/folio/edge-dcb1edge-dcb:1.1.02102489612876888128
mod-graphql1579891902283.dkr.ecr.us-east-1.amazonaws.com/folio/mod-graphql:1.12.12102489612876888128
mod-finance1579891902283.dkr.ecr.us-east-1.amazonaws.com/folio/mod-finance:4.9.02102489612870088128
mod-erm-usage1579891902283.dkr.ecr.us-east-1.amazonaws.com/folio/mod-erm-usage:4.7.02102489612876888128
mod-batch-print1579891902283.dkr.ecr.us-east-1.amazonaws.com/folio/mod-batch-print:1.1.02102489612876888128
mod-copycat1579891902283.dkr.ecr.us-east-1.amazonaws.com/folio/mod-copycat:1.6.02102451212876888128
mod-lists1579891902283.dkr.ecr.us-east-1.amazonaws.com/folio/mod-lists:2.0.02300026001282048384512
mod-entities-links1579891902283.dkr.ecr.us-east-1.amazonaws.com/folio/mod-entities-links:3.0.0225922480400144001024
mod-permissions2579891902283.dkr.ecr.us-east-1.amazonaws.com/folio/mod-permissions:6.5.02168415445121024384512
pub-edge1579891902283.dkr.ecr.us-east-1.amazonaws.com/folio/pub-edge1pub-edge:2023.06.142102489612876800
mod-orders1579891902283.dkr.ecr.us-east-1.amazonaws.com/folio/mod-orders:12.8.022048144010241024384512
edge-patron1579891902283.dkr.ecr.us-east-1.amazonaws.com/folio/edge-patron:5.1.02102489625676888128
edge-ncip1579891902283.dkr.ecr.us-east-1.amazonaws.com/folio/edge-ncip:1.9.22102489612876888128
mod-users-bl1579891902283.dkr.ecr.us-east-1.amazonaws.com/folio/mod-users-bl:7.7.021440115251292288128
mod-invoice1579891902283.dkr.ecr.us-east-1.amazonaws.com/folio/mod-invoice:5.8.021440115251292288128
mod-inventory-storage2579891902283.dkr.ecr.us-east-1.amazonaws.com/folio/mod-inventory-storage:27.2.0-SNAPSHOT.73824096369020483076384512
mod-user-import1579891902283.dkr.ecr.us-east-1.amazonaws.com/folio/mod-user-import:3.8.02102489612876888128
mod-sender1579891902283.dkr.ecr.us-east-1.amazonaws.com/folio/mod-sender:1.12.02102489612876888128
edge-oai-pmh1579891902283.dkr.ecr.us-east-1.amazonaws.com/folio/-oai-pmh1edge-oai-pmh:2.9.021512136010241440384512
mod-data-export-worker1579891902283.dkr.ecr.us-east-1.amazonaws.com/folio/mod-data-export-worker:3.2.123072204810242048384512
mod-rtac1579891902283.dkr.ecr.us-east-1.amazonaws.com/folio/mod-rtac:3.6.02102489612876888128
mod-circulation-storage1579891902283.dkr.ecr.us-east-1.amazonaws.com/folio/mod-circulation-storage:17.2.022880259215361814384512
mod-source-record-storage1579891902283.dkr.ecr.us-east-1.amazonaws.com/folio/mod-source-record-storage:5.8.025600500020483500384512
mod-calendar1579891902283.dkr.ecr.us-east-1.amazonaws.com/folio/mod-calendar:3.1.02102489612876888128
mod-event-config1579891902283.dkr.ecr.us-east-1.amazonaws.com/folio/mod-event-config:2.7.02102489612876888128
mod-courses1579891902283.dkr.ecr.us-east-1.amazonaws.com/folio/mod-courses:1.4.102102489612876888128
mod-circulation-item1579891902283.dkr.ecr.us-east-1.amazonaws.com/folio/mod-circulation-item:1.0.021024896128000
mod-inventory3579891902283.dkr.ecr.us-east-1.amazonaws.com/folio/-inventory3mod-inventory:20.2.022880259210241814384512
mod-email1579891902283.dkr.ecr.us-east-1.amazonaws.com/folio/mod-email:1.17.02102489612876888128
mod-circulation2579891902283.dkr.ecr.us-east-1.amazonaws.com/folio/mod-circulation:24.2.122880259215361814384512
mod-pubsub1579891902283.dkr.ecr.us-east-1.amazonaws.com/folio/mod-pubsub:2.13.02153614401024922384512
mod-di-converter-storage2579891902283.dkr.ecr.us-east-1.amazonaws.com/folio/mod-di-converter-storage:2.2.22102489612876888128
edge-orders1579891902283.dkr.ecr.us-east-1.amazonaws.com/folio/edge-orders:3.0.02102489612876888128
edge-rtac1579891902283.dkr.ecr.us-east-1.amazonaws.com/folio/edge-rtac:2.7.12102489612876888128
mod-users1579891902283.dkr.ecr.us-east-1.amazonaws.com/folio/mod-users:19.3.12102489612876888128
mod-template-engine1579891902283.dkr.ecr.us-east-1.amazonaws.com/folio/mod-template-engine:1.20.02102489612876888128
mod-patron-blocks1579891902283.dkr.ecr.us-east-1.amazonaws.com/folio/-patron-blocks1mod-patron-blocks:1.10.021024896102476888128
mod-audit1579891902283.dkr.ecr.us-east-1.amazonaws.com/folio/mod-audit:2.9.02102489612876888128
edge-fqm1579891902283.dkr.ecr.us-east-1.amazonaws.com/folio/edge-fqm:2.0.02102489612876888128
mod-source-record-manager1579891902283.dkr.ecr.us-east-1.amazonaws.com/folio/mod-source-record-manager:3.8.025600500020483500384512
nginx-edge1579891902283.dkr.ecr.us-east-1.amazonaws.com/folio/nginx-edge:2023.06.1421024896128000
mod-quick-marc1579891902283.dkr.ecr.us-east-1.amazonaws.com/folio/mod-quick-marc:5.1.01228821761281664384512
nginx-okapi1579891902283.dkr.ecr.us-east-1.amazonaws.com/folio/nginx-okapi:2023.06.1421024896128000
okapi-b1579891902283.dkr.ecr.us-east-1.amazonaws.com/folio/okapi:5.3.03168414401024922384512
mod-feesfines1579891902283.dkr.ecr.us-east-1.amazonaws.com/folio/mod-feesfines:19.1.02102489612876888128
mod-invoice-storage1579891902283.dkr.ecr.us-east-1.amazonaws.com/folio/-invoice-storage1mod-invoice-storage:5.8.021872153610241024384512
mod-service-interaction1579891902283.dkr.ecr.us-east-1.amazonaws.com/folio/mod-service-interaction:4.0.12204818442561290384512
mod-dcb1579891902283.dkr.ecr.us-east-1.amazonaws.com/folio/mod-dcb:1.1.02102489612876888128
mod-patron1579891902283.dkr.ecr.us-east-1.amazonaws.com/folio/mod-patron:6.1.02102489612876888128
mod-data-export1579891902283.dkr.ecr.us-east-1.amazonaws.com/folio/mod-data-export:5.0.01204815241024000
mod-oai-pmh1579891902283.dkr.ecr.us-east-1.amazonaws.com/folio/mod-oai-pmh:3.13.024096369020483076384512
edge-connexion1579891902283.dkr.ecr.us-east-1.amazonaws.com/folio/edge-connexion:1.2.02102489612876888128
mod-notes1579891902283.dkr.ecr.us-east-1.amazonaws.com/folio/mod-notes:5.2.021024896128952384512
mod-kb-ebsco-java1579891902283.dkr.ecr.us-east-1.amazonaws.com/folio/mod-kb-ebsco-java:4.0.02102489612876888128
mod-login1579891902283.dkr.ecr.us-east-1.amazonaws.com/folio/mod-login1mod-login:7.11.02144012981024768384512
mod-organizations-storage1579891902283.dkr.ecr.us-east-1.amazonaws.com/folio/mod-organizations-storage:4.7.02102489612870088128
mod-data-export-spring1579891902283.dkr.ecr.us-east-1.amazonaws.com/folio/mod-data-export-spring:3.2.01204818442561536384512
pub-okapi1579891902283.dkr.ecr.us-east-1.amazonaws.com/folio/pub-okapi:2023.06.142102489612876800
mod-eusage-reports1579891902283.dkr.ecr.us-east-1.amazonaws.com/folio/mod-eusage-reports:2.1.12102489612876888128


...

  • Test 1: Manually tested 5K, 10K, 25K, 50K, 100K consequentially (with 5 min pause) records files, DI (DI MARC Bib Create ) started on College tenant(cs00000int_0001) only, and CICO with 8 users on background.
  • Test 2: Manually tested 5K, 10K, 25K, 50K, 100K consequentially (with 5 min pause) records files, DI (DI MARC Bib Update) started on College tenant(cs00000int_0001) only, and CICO with 8 users on background.

At the time of the test run, Grafana was not available. As a result, response times for Check-In/Check-Out were parsed manually from a .jtl files, using the start and finish dates of the data import tests. These results were visualized in JMeter using a Listener (Response Times Over Time).