API for querying for MARC records stored in SRS (UXPROD-3086)

[UXPROD-2940] SRS MARC Query API part 2 Created: 08/Mar/21  Updated: 08/Oct/21  Resolved: 19/May/21

Status: Closed
Project: UX Product
Components: None
Affects versions: None
Fix versions: R2 2021
Parent: API for querying for MARC records stored in SRS

Type: New Feature Priority: P2
Reporter: Jenn Colt Assignee: Jenn Colt
Resolution: Done Votes: 0
Labels: r1-2021-at-risk, r1-highlight, split
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original estimate: Not Specified

Issue links:
Blocks
is blocked by MODSOURCE-276 Add existing records to the SRS Query... Closed
Cloners
clones UXPROD-2791 SRS MARC Query API Closed
is cloned by UXPROD-3078 SRS MARC Query API part 3 Closed
Defines
is defined by MODSOURCE-265 Implement presence or absence searches Closed
is defined by MODSOURCE-269 Implement presence or absence searche... Closed
is defined by MODSOURMAN-396 SRS MARC queries Closed
Relates
relates to MODSOURCE-254 SPIKE: Review MARC Query work Open
Requires
is required by UXPROD-2942 quickMARC | Search MARC Authority Rec... Closed
Epic Link: API for querying for MARC records stored in SRS
Estimation Notes and Assumptions: More advanced search functionality split from the R1 issue. Edge module functionality has been split to UXPROD-2916.
Development Team: Spitfire
Cap Plan Fix Version (DO NOT CHANGE): R2 2021
Rank: Cornell (Full Sum 2021): R1
Rank: Duke (Full Sum 2021): R1
Rank: 5Colleges (Full Jul 2021): R2
Rank: TAMU (MVP Jan 2021): R1
Rank: U of AL (MVP Oct 2020): R2
Score: 8
Showstopper for Summer 2021 Implementers?: Yes
Showstopper Comments from Summer 2021 Implementers: [Jenn from Cornell: This is critical for us because without it we cannot query the 10 million bibliographic records that describe our collection and drive discovery. These queries are used for selecting records to export to services like OCLC, identifying records as part of import processes, and identifying records that need cleanup.
Potential workarounds:
* Query LDP - MARC functionality is also not present there, in addition the data is only loaded every 24 hours. More than that, these functions are part of important record operations and we want the maintenance of the API providing that functionality to be part of FOLIO proper. In this same category would be a nightly dump of MARC records that we then examine with local scripting, requiring a lot of local, not shareable development.
* Direct database access - this might be an alternative but it is not available to us and would likely be more difficult than using the API, and is generally contrary to how FOLIO works.
* Don't do the work - Not performing the cleanup, enhancement, and export jobs facilitated by these queries would have a significantly negative impact on discovery and patron experience.]

Showstopper December 11 Meeting Summary: The data is expected to be in the LDP, but it is not there yet. What if it isn't there on time? Or isn't refreshed enough? Also, why should a library have to implement the LDP in order to do this work? Jenn is the PO for this feature, and is from Cornell, so this feature wouldn't need to be completed too far in advance of Cornell's go-live date. If the date ends up not being in the LDP on time, other implementers will need this as well.
Showstopper Capacity Planning Team Recommendation: The Capacity Planning Team recommended to the FOLIO Product Council that the Iris release date be extended to May 3 (from March 1) so that all of the at-risk 'showstopper' features could be completed and released before the July implementations. (Slide deck presented to PC: https://docs.google.com/presentation/d/12s_fs3vqjm4hAGIfZ_HX1jm--v8QOEFber5uvo0VTGw/edit?usp=sharing)
Showstopper FOLIO Product Council Decision: FOLIO Product Council compromised and allowed the Iris release to be delayed by one month, to April 5. Due to the size of the t-shirt estimate for this feature, will NOT be completed as part of the Iris release. The Capacity Planning Team will be meeting to discuss how/when this feature will be delivered to the libraries needing it before implementation.

 Description   

This feature holds the story for more advanced MARC queries split into R2 or R3.

 

Current situation or problem:
Given FOLIO doesn't yet support searching of MARC records in source record storage (SRS), this feature would provide an API that FOLIO implementers could use to query MARC. The API could be used for a variety of different purposes, such as identifying records for export, updates, and clean up. It could also, eventually, be used to support UI(s) for searching MARC within FOLIO, but that UI is out of scope for this feature. This feature will NOT use elastic search, implementation details to come.

In scope

Allow for search based on the presence or absence of a field, subfield, indicator, or fixed field position
Allow for date range search based on dates present in a subfield
Allow consumers to search new and/or local MARC tags and have them indexed appropriately

Out of scope
A search UI for MARC data
Searching inventory data

Use case(s)
Searching MARC data in order to identify records for data export
Searching MARC data in order to match records to be updated with data import
Searching MARC data as part of ETL clean up processes that libraries run to main data quality
Searching MARC data to identify records in cases where only examination of the MARC allows accurate selection of records



 Comments   
Comment by Khalilah Gambrell [ 12/May/21 ]

Jenn Colt, is the labeling of this feature as At-Risk tied to the unknowns regarding MARC Authority records?

Comment by Jenn Colt [ 12/May/21 ]

Hi Khalilah Gambrell no, I was thinking of the migration script. Maybe I'm being overly anxious about it. If the meeting we've had to postpone a couple of times has a good result tomorrow, I can remove it.

Generated at Fri Feb 09 00:28:05 UTC 2024 using Jira 1001.0.0-SNAPSHOT#100246-sha1:7a5c50119eb0633d306e14180817ddef5e80c75d.