Poppy - OAI-PMH Performance testing

Owned by Magda Zacharska
Last updated: Sep 21, 2023
1 min read

Modules needed

  • edge-oai-pmh
  • mod-oai-pmh
  • mod-inventory-storage
  • mod-source-record-storage
  • okapi

Testing expectations 

  • Tenant level full harvest completes within less than 12 hours.
  • Multiple concurrent harvests complete successfully 
  • At least one full and two incremental harvests can run concurrently.
  • Incremental harvests run every four hours complete successfully
  • Incremental harvests do not affect CI/CO and Data Import operations.


Data volumes

  • Full harvest of eight million or records
  • Incremental harvest of: 
    • 100 000 records
    • 5 000 records
    • 20 000 records
    • 40 000 records
  • Data import -2 000 instances (to be confirmed)
  • CI/CO - TBD

Consortia requirements

  • Each harvest contains shared and local instances
  • Harvests started at the central tenant harvest across all tenant consecutively until all tenant are harvested
  • Harvest started at the member tenant level, harvest only this tenant's inventory

User records testing scenarios

  1. Full harvest:
    1. Record source SRS
    2. Record source= SRS and Inventory
  2. Incremental harvests (includes updates and additions for period specified by providing from" and "until" parameters)





Concurrent and consecutive tests:

Scenario 1.  Run simultaneously full harvest and incremental harvests

Scenario 2. Run incremental harvests every  four hours and start data import at the same time

Scenario 3.  Run incremental harvests every four hours, start data import, test CO/CI

Failover tests:

  1. Determine likelihood of failover
  2. Does tasks restart?
  3. Does other task pick up the jobs that were in progress?