Hardening of OAI-PMH
Stakeholders |
|---|
PO - @Magda Zacharska SA - @Mikhail Fokanov |
FOLIO's implementation of OAI-PMH was developed with a requirement that the full inventory harvest is a rare event, done once or twice a year. The full harvest should be supported by incremental harvests that can be scheduled on a daily basis to make sure all the inventory changes are included in the library's catalogs. However, in production we see that the full harvests are triggered multiple times during a day, sometimes concurrently, sometimes inadvertently. The purpose of this page is to list the proposed initiatives and stories that will allow to harden OAI-PMH implementation and make it more robust.
Table
Date | Type | Details | Plan / Action items | Jira | Status/Timeline | |
|---|---|---|---|---|---|---|
| 1 | Mar 24, 2022 | Story | Noticed that two processes to fill instances table of mod OAI PMH module were created | Verify if the issue can be recreated or if it was related to the client requesting the harvest | Closed/ no longer occurs | |
| 2 | Mar 11, 2022 | Initiative | Investigate possible ways to limit impact of the full harvest on inventory and SRS storage | Review available options, create required stories and prioritize the work |
|
|
| 3 | Mar 29, 2022 | Story | Prevent users from inadvertently triggering multiple full harvests | Limit of max initial concurrent oai-pmh requests for tenant |
| Sprint 138 |
| 4 | Mar 30, 2022 | initiative | Determine how many full harvests the system can support before it crashes | Review available options, create required stories and prioritize the work |
| PTF team: Sprint 137, Sprint 138 |
| 5 | Dec 15, 2021 | Initiative | Perform analysis for the case when handling client waits while oai-pmh searching for instances with underlying records | Review available options, create required stories and prioritize the work |
| |
| 6 | Feb 21, 2022 | Initiative | Investigate handling invalid XML characters in the library data | Prioritize the work described in MODIPMH-402 |
| |
| 7 | Mar 21, 2022 | Story | Implement handling invalid XML characters in the library data | Prioritize the work |
|
|
| 8 | Mar 24, 2022 | Story | Performance testing for each release | Performance testing for the Lotus release | PTF team, Sprint 137, Sprint 138 | |
| 9 | Apr 4, 2022 | Initiative | Issues still occur. Need to carefully analyze the logs to have data for RCA | Perform analysis on the logs |
| Sprint 137 |
| 10 | Apr 5, 2022 | bug |
|
|
| Sprint 138 |
| 11 | Apr 5, 2022 | bug |
|
|
| Sprint 138 |
| 12 | Apr 11, 2022 | Story | Collect mod-oai-pmh instances statistics |
|
| |
| 13 | Apr 18, 2022 | Story | Build API for harvesting statistics |
| Sprint 138 |
