Overview

• The purpose of the concurrent OAI-PMH, data import and CI/CO tests is to determine how these workflows affect each other. The report contain results for the PERF-492 - Getting issue details... STATUS

Summary

• The OAI-PMH has an influence on CI/CO response times - it worsen results up to 13%. DI worsens CI/CO results mostly with create job profile up to 55% with 1.000 records and up to 71% with 100.000 records.  DI with update profile worsens CI/CO results less than create profile - up to 25% with 1.000 records and 36% with 100.000 records. OAI-PMH (incremental) duration was in range from 04:20 min till 05:20 min for all tests of DI with 1.000 records and without it.  OAI-PMH duration calculation is described in Methodology/Approach section.
• For major services memory usage didn't exceed the level of 60%. The highest level was registered for mod-source-record-manager 107% and mod-inventory-b 98%. After tests for Scenario 1 it achieved its stable level and didn't change.
• Running OAI-PMH, DI and CI/CO simultaneously it has been shown that the environment can handle such load.
• CI/CO response times during DI and OAI-PMH has degradation depending on the profile that was used. It was a number of consequent DI operations (create and update job profiles).
• After  90 minutes of full harvest the growth of CPU utilization up to 188 % was observed for mod-oai-pmh-b. This increased CPU utilization lasted during 10 minutes. After it got back to steady state ( 5-7 % ). 
• Service CPU Utilization at the beginning of DI mostly used by mod-di-converter-storage-b ( 253 % ), mod-inventory-b ( 172 % ), mod-quick-marc-b ( 108 % ). For the rest of modules it was under 70%. At the highest level it was mod-di-converter-storage-b ( 453 % ), mod-inventory-b ( 190 % ), mod-quick-marc-b ( 121 % ).
• RDS CPU Utilization during incremental harvesting didn't exceed 60 % for all DI job profiles (1.000 records). Data export took 40%  But for full harvesting with DI Create job profile (100.000 records) it became instantly 96 %  and stayed on this level major part of process. DI Update used up to 90%. 
• All oai-pmh tests were executed by EBSCO Harvester in the AWS ptf-windows instance.
• During full harvesting (504) Gateway Timeout issue happened after all DI create and update were done so it didn't affect the results. It happened during all two Full harvesting runs with returned instances count ( during first OAI-PMH full - 1764989 records, second - 1166089 out of total 10433728 ).

Recommendations & Jiras

• Allocate more CPU resources to mod-di-convertor-storage and mod-inventory-b

Test Runs & Results

Data import duration and CI/CO response times with DI & OAI-PMH results

Comparisons

Comparison table for CI/CO response times

Scenario 1

Response time

This table shows s40 minutes of CI/CO

Service CPU Utilization

Service Memory Utilization

RDS CPU Utilization