OAI-PMH Support (UXPROD-993)

[UXPROD-4006] Harvesting bad data Created: 01/Feb/23  Updated: 08/Feb/24  Resolved: 19/May/23

Status: Closed
Project: UX Product
Components: None
Affects versions: None
Fix versions: Poppy (R2 2023)
Parent: OAI-PMH Support

Type: New Feature Priority: P2
Reporter: Magda Zacharska Assignee: Magda Zacharska
Resolution: Done Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original estimate: Not Specified

Issue links:
Continues
is continued by UXPROD-4094 Reports on bad data encountered durin... Closed
Defines
is defined by MODINVSTOR-895 bad data in item.statisticalCodeIds s... Closed
is defined by MODINVSTOR-897 Enchance inventory-hierarchy SQL to o... Closed
is defined by MODOAIPMH-402 Verify if harvest can handle control ... Closed
is defined by MODOAIPMH-442 bad data in call number type field re... Closed
is defined by MODOAIPMH-444 bad data in instance.natureOfContentT... Closed
is defined by MODOAIPMH-454 Spike: Make records with bad data ava... Closed
Release: Poppy (R2 2023)
Epic Link: OAI-PMH Support
Back End Estimate: XL < 15 days
Back End Estimator: Viachaslau Khandramai (Inactive)
Back-End Confidence factor: 60%
Development Team: Firebird
PO Rank: 0

 Description   

Current situation or problem:
OAI-PMH implementation is not robust enough to handle bad data that often causes the harvest to crash.

In scope

  1. Define the process for handling bad data in case of:
    1. invalid references
    2. special characters
  2. Implement solution where:
    1. bad data does not stop the harvest and when the problem with the data is encountered then:
      1. the problem is not attempted to be resolved but
      2. the affected record is omitted from the harvest and the harvest moves to the next record
      3. the record with bad data is logged so that the libraries could address the issue:

Out of scope - the functionality below will be handled in UXPROD-4094 Closed

  1. bad data to be saved so that the libraries could address the issue:
  2. an endpoint to retrieve the bad data
  3. UI to make the data easily available to users

Generated at Fri Feb 09 00:36:32 UTC 2024 using Jira 1001.0.0-SNAPSHOT#100246-sha1:7a5c50119eb0633d306e14180817ddef5e80c75d.