Skip to end of banner
Go to start of banner

[Ramsons] List App test report for Contains any, Contains all, Not contains any, Not contains all Operators

Skip to end of metadata
Go to start of metadata

You are viewing an old version of this page. View the current version.

Compare with Current View Page History

« Previous Version 7 Next »


Overview

This document contains the results of testing List App refreshing of records in the Ramsons release for Contains any, Contains all, Not contains any, Not contains all Operators. PERF-892 - Getting issue details... STATUS

Summary


  • Memory utilization increasing for mod-fgm-manager up to 80% during lists refresh then goes down in few minutes - OOM issue for mod-fqm-manager MODFQMMGR-501 - Getting issue details... STATUS but only for creating new lists while the test query is running.
  • For mod-lists, CPU utilization was up to 237% for the test with 10 concurrent users and R/W split enabled. Average CPU usage did not exceed 14 % for all other modules.
  • Approximately DB CPU usage is up to 98.5%.

Recommendations and Jiras

To have the possibility of refreshing lists for a large number of records, we need to increase QUERY RETENTION DURATION (mod-fqm-manager) and mod-lists.general.refresh-query-timeout-minutes to at least 10 hours.

MODFQMMGR-506 - Getting issue details... STATUS

MODLISTS-149 - Getting issue details... STATUS

MODLISTS-150 - Getting issue details... STATUS

MODFQMMGR-522 - Getting issue details... STATUS

MODFQMMGR-501 - Getting issue details... STATUS

MODFQMMGR-505 - Getting issue details... STATUS

MODFQMMGR-510 - Getting issue details... STATUS

MODLISTS-147 - Getting issue details... STATUS

MODFQMMGR-518 - Getting issue details... STATUS

Results

List name Search from the number of recordsRecords found

 1 user

4 users

8 users

10 usersquery
Holdings all statistical code28,016,666500,00000:05:4000:17:4100:18:0400:24:32"holdings.statistical_code_names contains all [PTF: PTF1 - PTF1, University of Chicago: ebooks - Books, electronic (ebooks)]"
Holdings any statistical code28,016,6665,347,86602:03:17

04:56:23

05:01:3409:04:10holdings.statistical_code_names contains any [PTF: PTF1 - PTF1, University of Chicago: ebooks - Books, electronic (ebooks)]
Holdings not all statistical code28,016,6665,453,772

02:24:23 if success

  • [404 Not Found] during [GET] to [http://query/[id]/sortedIds?offset=3978000&limit=1000] [QueryClient#getSortedIds(UUID,Integer,Integer)]: [{"message":"Entity type with ID [id] not found","type":null,"code":null,"parameters":[{"key":"entityTypeId","value":"[id]"}]}]

    in 03:10:28, 03:22:46
05:08:4805:08:2404:31:33holdings.statistical_code_names does not contain all [PTF: PTF1 - PTF1, University of Chicago: ebooks - Books, electronic (ebooks)]
Holdings not any statistical code28,016,666105,90800:03:2100:18:3600:05:2900:06:38holdings.statistical_code_names does not contain any [PTF: PTF1 - PTF1, University of Chicago: ebooks - Books, electronic (ebooks)]
Items all statistical code27,957,839100,00000:06:0200:09:5500:07:0300:12:37items.statistical_code_names contains all [University of Chicago: books - Books, print (books), University of Chicago: mfilm - Microfilm (mfilm)]
Items any statistical code27,957,8394,874,98202:36:3203:22:25

FAILED

SQL [select cast(array["item.item".id, 'tenant'] as varchar[]) from...

FAILED in 03:42:33
[500 Internal Server Error] during [GET] to [http://query/73497f47-34fd-4eef-93db-4b858b20c6af/sortedIds?offset=2004000&limit=1000] [QueryClient#getSortedIds(UUID,Integer,Integer)]: [{"timestamp":"2024-10-09T13:55:52.355+00:00","status":500,"error":"Internal Server Error","path":"/query/73497f47-34fd-4eef-93db-4b858b20c6af/sortedIds"}]		items.statistical_code_names contains any [University of Chicago: books - Books, print (books), University of Chicago: mfilm - Microfilm (mfilm)]
Items not all statistical code27,957,8395,184,43302:01:1502:16:1204:58:38FAILED in 03:43:02
[500 Internal Server Error] during [GET] to [http://query/22bdc696-558e-47f5-9661-f940369359e2/sortedIds?offset=3989000&limit=1000] [QueryClient#getSortedIds(UUID,Integer,Integer)]: [{"timestamp":"2024-10-09T13:56:23.683+00:00","status":500,"error":"Internal Server Error","path":"/query/22bdc696-558e-47f5-9661-f940369359e2/sortedIds"}]		items.statistical_code_names does not contain all [University of Chicago: books - Books, print (books), University of Chicago: mfilm - Microfilm (mfilm)]
Items not any statistical code27,957,839409,45100:08:3900:17:3300:19:3200:21:39items.statistical_code_names does not contain any [University of Chicago: mfilm - Microfilm (mfilm), University of Chicago: books - Books, print (books)]
Users all departments250,0857,68600:00:1300:00:1200:00:1200:00:13users.departments contains all [9af255eb-b87c-4045-9407-d9c324bf0e8c, b1318aeb-0273-42b3-ba57-f1ee5d5cc59e]
Users any departments250,08529,21600:00:1800:00:1900:00:2200:00:22users.departments contains any [b1318aeb-0273-42b3-ba57-f1ee5d5cc59e, fa6b405f-311e-493c-be6f-41610552742c, 9af255eb-b87c-4045-9407-d9c324bf0e8c]
Users not all departments250,08536,44500:00:2400:00:2400:00:2500:00:27users.departments does not contain all [b1318aeb-0273-42b3-ba57-f1ee5d5cc59e, 2eceb91f-6230-4fdb-9d22-8b34216ab800, 9af255eb-b87c-4045-9407-d9c324bf0e8c]
Users not any departments250,0857,22900:00:1200:00:1200:00:1200:00:13users.departments does not contain any [b1318aeb-0273-42b3-ba57-f1ee5d5cc59e, fa6b405f-311e-493c-be6f-41610552742c, 9af255eb-b87c-4045-9407-d9c324bf0e8c]
POL all Acquisition unitsany0not able to find POL MODFQMMGR-506 - Getting issue details... STATUS ---vendor_organization.accounts[*]->acq_unit_ids contains all [d7515bef-206f-4f23-be80-563f9bca866b, 0ebb1f7d-983f-3026-8a4c-5318e0ebc041]
POL any Acquisition unitsany0not able to find POL MODFQMMGR-506 - Getting issue details... STATUS ---vendor_organization.accounts[*]->acq_unit_ids contains any [d7515bef-206f-4f23-be80-563f9bca866b, 0ebb1f7d-983f-3026-8a4c-5318e0ebc041]
POL not all Acquisition unitsany0not able to find POL MODFQMMGR-506 - Getting issue details... STATUS ---vendor_organization.accounts[*]->acq_unit_ids does not contain all [d7515bef-206f-4f23-be80-563f9bca866b, 0ebb1f7d-983f-3026-8a4c-5318e0ebc041]
POL not any Acquisition unitsany0not able to find POL MODFQMMGR-506 - Getting issue details... STATUS ---vendor_organization.accounts[*]->acq_unit_ids does not contain any [d7515bef-206f-4f23-be80-563f9bca866b, 0ebb1f7d-983f-3026-8a4c-5318e0ebc041]
Instance all statistical code29,105,407

20,000 or "FAILED"

Condition with org.folio.list.services.refresh.ListRefreshService was not fulfilled within 1 hours 30 minutes. MODLISTS-147 - Getting issue details... STATUS FAILED in 02:35:45
[400 Bad Request] during [GET] to [http://query/b793e898-9a17-4b28-bcd5-38fd89f55ba1] [QueryClient#getQuery(UUID)]: []		FAILED
SQL [select cast(array["instance.inst".id, 'tenant'] as varchar[]) from...		03:56:59instance.statistical_code_names contains all [University of Chicago: books - Books, print (books), RECM (Record management): XOCLC - Do not share with OCLC]
Instance any statistical code29,105,407"FAILED" MODFQMMGR-505 - Getting issue details... STATUS

Not able to create the list

MODFQMMGR-501 - Getting issue details... STATUS

--instance.statistical_code_names contains any [PTF: PTF1 - PTF1, PTF: PTF3 - PTF3, University of Chicago: books - Books, print (books), RECM (Record management): XOCLC - Do not share with OCLC]
Instance not all statistical code29,105,407"FAILED" MODFQMMGR-505 - Getting issue details... STATUS Not able to create the list MODFQMMGR-501 - Getting issue details... STATUS --instance.statistical_code_names not contain all [PTF: PTF1 - PTF1, PTF: PTF3 - PTF3, University of Chicago: books - Books, print (books), RECM (Record management): XOCLC - Do not share with OCLC]
Instance not any statistical code29,105,407

534,975 or "FAILED"

02:51:2902:47:14FAILED
SQL [select cast(array["instance.inst".id, 'tenant'] as varchar[]) from...		04:17:27instance.statistical_code_names not contain any [PTF: PTF1 - PTF1, PTF: PTF3 - PTF3, University of Chicago: books - Books, print (books), RECM (Record management): XOCLC - Do not share with OCLC]


Instance CPU Utilization

Service Memory Utilization

mod-lists & mod-fqm-manager

Service CPU Utilization 


mod-fqm-manager

mod-lists



RDS CPU Utilization 



RDS Database Connections


Long-running queries: 

select cast(array["instance.inst".id, ?] as varchar[]) from [tenant]_mod_fqm_manager.src_inventory_instance "instance.inst" left join [tenant]_mod_fqm_manager.src_inventory_mode_of_issuance "instance.mode_of_issuance" ON "instance.mode_of_issuance".id = "instance.inst".modeOfIssuanceId left join [tenant]_mod_fqm_manager.src_inventory_instance_type "instance.instance_type" ON "instance.instance_type".id = "instance.inst".instanceTypeId left join [tenant]_mod_fqm_manager.src_inventory_inst

Appendix

Infrastructure

Records count :

  • mod_inventory_storage.instance = 29105407
  • mod_inventory_storage.item = 27957839
  • mod_inventory_storage.holdings_record = 28016666
  • mod_users.users = 

PTF -environment ncp5 

  • 12 m6i.2xlarge EC2 instances located in US East (N. Virginia)us-east-1

  • 2 database  instances, one reader, and one writer

    NameAPI NameMemory GIBvCPUsmax_connections
    R6G Extra Largedb.r6g.xlarge32 GiB4 vCPUs2731
  • MSK ptf-kakfa-3
    • 4 m5.2xlarge brokers in 2 zones

    • Apache Kafka version 2.8.0

    • EBS storage volume per broker 300 GiB

    • auto.create.topics.enable=true
    • log.retention.minutes=480
    • default.replication.factor=3


Modules memory and CPU parameters

Module
ncp5-pvt
Wed Aug 23 10:38:27 UTC 2023		Task Def. RevisionModule VersionTask CountMem Hard LimitMem Soft limitCPU unitsXmxMetaspaceSizeMaxMetaspaceSizeR/W split enabled
mod-remote-storage14mod-remote-storage:2.0.324920447210243960512512true
mod-codex-mux3mod-codex-mux:2.13.0-SNAPSHOT.1082102489612876888128false
mod-finance-storage9mod-finance-storage:8.4.221024896102470088128false
mod-ncip9mod-ncip:1.13.12102489612876888128false
mod-agreements6mod-agreements:5.5.2215921488128968384512false
mod-ebsconet6mod-ebsconet:2.0.0212481024128700128256false
mod-organizations6mod-organizations:1.7.02102489612870088128false
edge-sip24edge-sip2:3.0.02102489612876888128false
mod-settings6mod-settings:1.0.021024896200000false
mod-data-import9mod-data-import:2.7.11204818442561292384512true
edge-dematic14edge-dematic:2.0.111024896128000false
mod-search8mod-search:2.0.1225922480204814405121024false
mod-inn-reach1mod-inn-reach:3.1.0-SNAPSHOT.4922304204851214405121024false
mod-tags6mod-tags:2.0.12102489612876888128false
mod-authtoken7mod-authtoken:2.13.021440115251292288128true
mod-notify7mod-notify:3.0.02102489612876888128false
mod-inventory-update6mod-inventory-update:3.0.12102489612876888128false
mod-configuration9mod-configuration:5.9.12102489612876888128true
mod-orders-storage6mod-orders-storage:13.5.02102489651270088128false
edge-caiasoft6edge-caiasoft:2.0.021024896128000false
mod-login-saml6mod-login-saml:2.6.22102489612876888128false
mod-erm-usage-harvester7mod-erm-usage-harvester:4.3.12102489612876888128false
mod-password-validator6mod-password-validator:3.0.0214401298128768384512false
mod-licenses6mod-licenses:4.3.12248023121281792384512false
mod-gobi6mod-gobi:2.6.02102489612870088128false
mod-bulk-operations17mod-bulk-operations:1.0.623072260010241536384512false
mod-fqm-manager19mod-fqm-manager:1.0.0-SNAPSHOT.8411300026001282048384512true
mod-graphql6mod-graphql:1.11.02102489612876888128false
mod-finance6mod-finance:4.7.12102489612870088128false
mod-erm-usage7mod-erm-usage:4.5.32102489612876888128false
mod-copycat6mod-copycat:1.4.02102489612876888128false
mod-lists3-4-5mod-lists:1.0.0-SNAPSHOT.1130130002600128/5122048384512false/true
mod-permissions20mod-permissions:6.3.22168415445121024384512true
mod-entities-links7mod-entities-links:1.0.2225922480400144001024false
pub-edge7pub-edge:2023.06.142102489612876800false
mod-codex-inventory3mod-codex-inventory:2.4.0-SNAPSHOT.1392102489612876888128true
mod-orders11mod-orders:12.6.822048144010241024384512false
edge-patron6edge-patron:4.11.02102489625676888128false
edge-ncip6edge-ncip:1.8.12102489612876888128false
mod-users-bl8mod-users-bl:7.5.021440115251292288128false
mod-inventory-storage11mod-inventory-storage:26.0.024096369020483076384512true
mod-invoice6mod-invoice:5.6.521440115251292288128false
mod-user-import6mod-user-import:3.7.22102489612876888128false
mod-sender7mod-sender:1.10.02102489612876888128false
edge-oai-pmh9edge-oai-pmh:2.6.121512136010241440384512false
mod-data-export-worker10mod-data-export-worker:3.0.1323072280010242048384512false
mod-rtac8mod-rtac:3.5.02102489612876888128false
mod-task-list3mod-task-list:1.7.12102489612876888128false
mod-circulation-storage13mod-circulation-storage:16.0.0.1622880259215361814384512false
mod-calendar8mod-calendar:2.4.22102489612876888128false
mod-source-record-storage26mod-source-record-storage:5.6.725600500020483500384512true
mod-event-config6mod-event-config:2.5.02102489612876888128false
mod-courses6mod-courses:1.4.72102489612876888128false
mod-inventory1mod-inventory:20.0.402880259210241814384512false
mod-email6mod-email:1.15.32102489612876888128false
mod-circulation7mod-circulation:23.5.6228802592153676888128false
mod-di-converter-storage6mod-di-converter-storage:2.0.52102489612876888128false
mod-pubsub10mod-pubsub:2.9.12153614401024922384512true
edge-orders6edge-orders:2.8.12102489612876888128false
edge-rtac6edge-rtac:2.6.02102489612876888128false
mod-template-engine6mod-template-engine:1.18.02102489612876888128false
mod-users8mod-users:19.1.12102489612876888128true
mod-patron-blocks11mod-patron-blocks:1.8.021024896102476888128true
mod-audit9mod-audit:2.7.02102489612876888128false
mod-source-record-manager18mod-source-record-manager:3.6.425600500020483500384512false
nginx-edge8nginx-edge:2023.06.1421024896128000false
mod-quick-marc6mod-quick-marc:3.0.01228821761281664384512false
nginx-okapi7nginx-okapi:2023.06.1421024896128000false
okapi-b8okapi:5.0.13168414401024922384512false
mod-feesfines9mod-feesfines:18.2.12102489612876888128true
mod-invoice-storage6mod-invoice-storage:5.6.021872153610241024384512false
mod-codex-ekb3mod-codex-ekb:1.11.0-SNAPSHOT.1412102489612876888128false
mod-service-interaction6mod-service-interaction:2.2.22204818442561290384512false
mod-data-export6mod-data-export:4.7.111024896102476888128false
mod-patron7mod-patron:5.5.22102489612876888128false
mod-oai-pmh10mod-oai-pmh:3.11.324096369020483076384512true
edge-connexion6edge-connexion:1.0.62102489612876888128false
mod-kb-ebsco-java7mod-kb-ebsco-java:3.13.02102489612876888128false
mod-notes6mod-notes:5.0.121024896128952384512false
mod-data-export-spring8mod-data-export-spring:2.0.21204818442561536384512false
mod-organizations-storage7mod-organizations-storage:4.5.12102489612870088128false
mod-login6mod-login:7.9.02144012981024768384512false
pub-okapi7pub-okapi:2023.06.142102489612876800false
mod-data-import-converter-storage4mod-data-import-converter-storage:1.16.0-SNAPSHOT.1322102489612876888128false
mod-eusage-reports7mod-eusage-reports:1.3.02102489612876888128false

Methodology/Approach

To test Baseline for List refresh JMeter scripts were used.

Test preparation: 

  • Background circulation jobs were disabled.
  • 200k items were checked out
  • 10 lists were created with the query: (item_status == "Checked out") to be able to run a test for 10 concurrent users.

Refresh list for 1, 2, 5, and 10 concurrent users tested

  • No labels