API for querying for MARC records stored in SRS (UXPROD-3086)

[UXPROD-3078] SRS MARC Query API part 3 Created: 19/May/21  Updated: 29/Dec/22  Resolved: 29/Dec/22

Status: Closed
Project: UX Product
Components: None
Affects versions: None
Fix versions: None
Parent: API for querying for MARC records stored in SRS

Type: New Feature Priority: P2
Reporter: Jenn Colt Assignee: Jenn Colt
Resolution: Won't Do Votes: 0
Labels: r1-highlight, split
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original estimate: Not Specified

Issue links:
Cloners
clones UXPROD-2940 SRS MARC Query API part 2 Closed
Defines
is defined by MODSOURCE-215 MARC search functionality - tech part Closed
is defined by MODSOURCE-264 Documentation for SRS Query API Closed
is defined by MODSOURCE-281 Provide a filter for different types ... Closed
is defined by MODSOURCE-284 Document performance impact of SRS qu... Closed
is defined by MODSOURCE-224 Support MARC search by internal Id an... Draft
Epic Link: API for querying for MARC records stored in SRS
Estimation Notes and Assumptions: More advanced search functionality split from the R1 issue. Edge module functionality has been split to UXPROD-2916.
Development Team: Spitfire
Cap Plan Fix Version (DO NOT CHANGE): R2 2021
Rank: Cornell (Full Sum 2021): R1
Rank: Duke (Full Sum 2021): R1
Rank: 5Colleges (Full Jul 2021): R2
Rank: MO State (MVP June 2020): R1
Rank: TAMU (MVP Jan 2021): R1
Rank: U of AL (MVP Oct 2020): R2
Score: 8
Showstopper for Summer 2021 Implementers?: Yes
Showstopper Comments from Summer 2021 Implementers: [Jenn from Cornell: This is critical for us because without it we cannot query the 10 million bibliographic records that describe our collection and drive discovery. These queries are used for selecting records to export to services like OCLC, identifying records as part of import processes, and identifying records that need cleanup.
Potential workarounds:
* Query LDP - MARC functionality is also not present there, in addition the data is only loaded every 24 hours. More than that, these functions are part of important record operations and we want the maintenance of the API providing that functionality to be part of FOLIO proper. In this same category would be a nightly dump of MARC records that we then examine with local scripting, requiring a lot of local, not shareable development.
* Direct database access - this might be an alternative but it is not available to us and would likely be more difficult than using the API, and is generally contrary to how FOLIO works.
* Don't do the work - Not performing the cleanup, enhancement, and export jobs facilitated by these queries would have a significantly negative impact on discovery and patron experience.]

Showstopper December 11 Meeting Summary: The data is expected to be in the LDP, but it is not there yet. What if it isn't there on time? Or isn't refreshed enough? Also, why should a library have to implement the LDP in order to do this work? Jenn is the PO for this feature, and is from Cornell, so this feature wouldn't need to be completed too far in advance of Cornell's go-live date. If the date ends up not being in the LDP on time, other implementers will need this as well.
Showstopper Capacity Planning Team Recommendation: The Capacity Planning Team recommended to the FOLIO Product Council that the Iris release date be extended to May 3 (from March 1) so that all of the at-risk 'showstopper' features could be completed and released before the July implementations. (Slide deck presented to PC: https://docs.google.com/presentation/d/12s_fs3vqjm4hAGIfZ_HX1jm--v8QOEFber5uvo0VTGw/edit?usp=sharing)
Showstopper FOLIO Product Council Decision: FOLIO Product Council compromised and allowed the Iris release to be delayed by one month, to April 5. Due to the size of the t-shirt estimate for this feature, will NOT be completed as part of the Iris release. The Capacity Planning Team will be meeting to discuss how/when this feature will be delivered to the libraries needing it before implementation.

 Description   

This feature holds the remainder of the API stories, particularly covering interaction with data import/quick marc and with the new types of records being added to SRS.

 

Current situation or problem:
Given FOLIO doesn't yet support searching of MARC records in source record storage (SRS), this feature would provide an API that FOLIO implementers could use to query MARC. The API could be used for a variety of different purposes, such as identifying records for export, updates, and clean up. It could also, eventually, be used to support UI(s) for searching MARC within FOLIO, but that UI is out of scope for this feature. This feature will NOT use elastic search, implementation details to come.

In scope

Allow for search based on the presence or absence of a field, subfield, indicator, or fixed field position
Allow for date range search based on dates present in a subfield
Allow consumers to search new and/or local MARC tags and have them indexed appropriately

Out of scope
A search UI for MARC data
Searching inventory data

Use case(s)
Searching MARC data in order to identify records for data export
Searching MARC data in order to match records to be updated with data import
Searching MARC data as part of ETL clean up processes that libraries run to main data quality
Searching MARC data to identify records in cases where only examination of the MARC allows accurate selection of records


Generated at Fri Feb 09 00:29:10 UTC 2024 using Jira 1001.0.0-SNAPSHOT#100246-sha1:7a5c50119eb0633d306e14180817ddef5e80c75d.