Versions Compared

Key

  • This line was added.
  • This line was removed.
  • Formatting was changed.

Table of Contents

Overview

The purpose of the concurrent OAI-PMH, data import and CI/CO tests is to determine the areas which may be affected by increasing of harvests frequency. 

Summary


Test Runs & Results


Data import duration and CI/CO response times with DI & OAI-PMH results

Test #

CI/CO

Scenario

Job profile

Duration

CI average 

CO average 

Load level

Comments

Scenario 1

OAI-PMH incremental

5 hours

DI MARC Bib Create

PTF - Create 2

00:00:48

0.961

1.398

For scenario 1

1K (with pause ~5 min)


DI MARC Bib Update

PTF - Updates Success - 1

00:00:56

0.706

1.125


DI MARC Bib Create

PTF - Create 2

00:00:43

0.843

1.402


DI MARC Bib Update

PTF - Updates Success - 1

00:00:44

0.848

1.335


Scenario 2

OAI-PMH full mode

DI MARC Bib Create

PTF - Create 2

00:53:30

1.0781.545

For scenario 2

100K (with pause ~5 min)


DI MARC Bib Update

PTF - Updates Success - 1

01:04:38

0.7251.231

DI MARC Bib Update

PTF - Updates Success - 1

01:05:48

0.691.249

5 hours

DI MARC Bib Update

PTF - Updates Success - 1

01:17:580.9031.333

DI MARC Bib Update

PTF - Updates Success - 1

01:18:080.7371.221

DI MARC Bib Update

PTF - Updates Success - 1

01:21:210.621.106Last 30 minutes without OAI-PMH

Scenario 1

This table contains CI/CO response times without DI & OAI-PMH

Requests

50th pct

75th pct

95th pct

Average

Check-Out Controller

0.862

0.935

1.133

0.904

Check-In Controller

0.581

0.633

0.827

0.629

Response time


Service CPU Utilization

TBD

Service Memory Utilization

TBD

RDS CPU Utilization

TBD

Image Modified

Scenario 2

Response time


Service CPU Utilization

TBD

Service Memory Utilization

TBD

RDS CPU Utilization

TBD

Image Modified


Appendix

Methodology/Approach

Circulation rules should be modified before CI/CO test in Circulation rules editor to run it without issues from POST_circulation/check-out-by-barcode (Submit_barcode_checkout) side.

Partitions number should be equal to 2 in all DI related topics.

Before running OAI-PMH with full harvest, following database commands to optimize the tables should be executed (from https://wiki.folio.org/display/FOLIOtips/OAI-PMH+Best+Practices#OAIPMHBestPractices-SlowPerformance):

REINDEX index <tenant>_mod_inventory_storage.audit_item_pmh_createddate_idx ;
REINDEX index <tenant>_mod_inventory_storage.audit_holdings_record_pmh_createddate_idx;
REINDEX index <tenant>_mod_inventory_storage.holdings_record_pmh_metadata_updateddate_idx;
REINDEX index <tenant>_mod_inventory_storage.item_pmh_metadata_updateddate_idx;
REINDEX index <tenant>_mod_inventory_storage.instance_pmh_metadata_updateddate_idx;
analyze verbose <tenant>_mod_inventory_storage.instance;
analyze verbose <tenant>_mod_inventory_storage.item;
analyze verbose <tenant>_mod_inventory_storage.holdings_record;

  1. Execute the following query in a related database for removing existed 'instances' created by previous harvesting request and a request itself:

TRUNCATE TABLE fs09000000_mod_oai_pmh.request_metadata_lb cascade

Modules

ModuleTask Def. RevisionModule VersionTask CountMem Hard LimitMem Soft limitCPU unitsXmxMetaspaceSizeMaxMetaspaceSizeR/W split enabled
ocp2-pvt
Mon Jul 03 14:54:13 UTC 2023
mod-inventory-storage4579891902283.dkr.ecr.us-east-1.amazonaws.com/folio/mod-inventory-storage:26.0.022208195210241440384512FALSE
mod-inventory3579891902283.dkr.ecr.us-east-1.amazonaws.com/folio/mod-inventory:20.0.0-SNAPSHOT.39222880259210241814384512FALSE
mod-source-record-storage5579891902283.dkr.ecr.us-east-1.amazonaws.com/folio/mod-source-record-storage:5.6.525600500020483600384512FALSE
mod-source-record-manager3579891902283.dkr.ecr.us-east-1.amazonaws.com/folio/mod-source-record-manager:3.6.0-SNAPSHOT.19724096368810242048384512FALSE
mod-data-import3579891902283.dkr.ecr.us-east-1.amazonaws.com/folio/mod-data-import:2.7.0-SNAPSHOT.1011204818442561292384512FALSE
mod-di-converter-storage1579891902283.dkr.ecr.us-east-1.amazonaws.com/folio/mod-di-converter-storage:2.1.0-SNAPSHOT.322102489612876888128FALSE
mod-data-import-converter-storage3579891902283.dkr.ecr.us-east-1.amazonaws.com/folio/mod-data-import-converter-storage:1.16.0-SNAPSHOT.1322102489612876888128FALSE
mod-remote-storage3579891902283.dkr.ecr.us-east-1.amazonaws.com/folio/mod-remote-storage:2.0.0-SNAPSHOT.8324920447210243960512512FALSE
mod-users3579891902283.dkr.ecr.us-east-1.amazonaws.com/folio/mod-users:19.2.0-SNAPSHOT.5842102489612876888128FALSE
mod-configuration3579891902283.dkr.ecr.us-east-1.amazonaws.com/folio/mod-configuration:5.9.2-SNAPSHOT.2912102489612876888128FALSE
mod-circulation-storage3579891902283.dkr.ecr.us-east-1.amazonaws.com/folio/mod-circulation-storage:16.1.0-SNAPSHOT.3052153614401024896384512FALSE
mod-circulation3579891902283.dkr.ecr.us-east-1.amazonaws.com/folio/mod-circulation:23.5.0-SNAPSHOT.55621024896102476888128FALSE
mod-authtoken3579891902283.dkr.ecr.us-east-1.amazonaws.com/folio/mod-authtoken:2.14.0-SNAPSHOT.23821440115251292288128FALSE
mod-pubsub3579891902283.dkr.ecr.us-east-1.amazonaws.com/folio/mod-pubsub:2.10.0-SNAPSHOT.1242153614401024922384512FALSE
pub-okapi2579891902283.dkr.ecr.us-east-1.amazonaws.com/folio/pub-okapi:2022.03.022102489612876800FALSE
okapi-b2579891902283.dkr.ecr.us-east-1.amazonaws.com/folio/okapi:5.1.0-SNAPSHOT.13523168414401024922384512FALSE

Partitions