quickMARC: MARC Validation (UXPROD-3985)

[UXPROD-3940] MARC21 record validation support - Phase 1 Created: 29/Dec/22  Updated: 06/Feb/24

Status: Draft
Project: UX Product
Components: None
Affects versions: None
Fix versions: Quesnelia (R1 2024)
Parent: quickMARC: MARC Validation

Type: New Feature Priority: P3
Reporter: Khalilah Gambrell Assignee: Kalibek Turgumbayev
Resolution: Unresolved Votes: 1
Labels: LC-priority2, SolutionArchitecture, cataloging, galileo, loc, metadatamanagement
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original estimate: Not Specified

Issue links:
Blocks
is blocked by UIQM-600 FE - MARC validation\Bib Rec.\Only on... Open
is blocked by UXPROD-4575 Create\Edit\Derive quickMARC Bibliogr... Open
is blocked by UXPROD-4388 MARC\Holding\validation support Open
is blocked by UXPROD-4402 MARC\Authority\validation support Open
is blocked by UXPROD-4403 MARC validation\Bib Rec. Open
is blocked by UIQM-469 Create\Edit\Derive quickMARC Bibliogr... Closed
is blocked by UIQM-398 Create\Edit quickMARC Authority: Add ... Draft
is blocked by MODQM-388 BE - MARC validation\Bib Rec.\Only on... Blocked
is blocked by UXPROD-4375 MARC21 record validation support\Gene... In Refinement
Cloners
is cloned by UXPROD-4549 MARC21 record validation support - Ph... Open
Relates
relates to UIQM-599 Remove additional 005 and/or 999 fiel... Open
relates to UIQM-382 quickMARC Bib Validation: Make 010 a ... Closed
relates to UIQM-123 Characters in block type "a", " t " a... Closed
relates to MODQM-378 Tenant level configuration - Make 010... Draft
relates to MODQM-394 MARC BIB Validation: Create a global ... Draft
relates to UIQM-495 Add more tests for MARC records valid... Draft
relates to MODQM-398 MARC BIB Record validation | Tenant c... In Refinement
relates to MODQM-399 MARC BIB Record validation | Permissi... In Refinement
relates to MODQM-400 MARC AUTHORITY Record validation | Te... In Refinement
relates to MODQM-401 MARC Authority Record validation | Pe... In Refinement
relates to MODQM-256 User can enter {dollar} or {$} to han... Open
relates to UXPROD-4286 Feature: MARC validation rules | BF C... Open
relates to UIQM-526 Jest tests for quickMARC validation r... Closed
relates to UIQM-103 Edit/DeriveMARC bib record and View S... Closed
relates to UIQM-131 Add "guidance" to LDR field Closed
relates to MODELINKS-131 Authority: Restrict authority to have... Draft
relates to MODQM-235 POC: quickMARC: Extend MARC validation Draft
relates to MODQM-328 Formalization of error messages Draft
relates to MODQM-380 Inaccuracies in the definition of the... Draft
relates to UIQM-461 Display positions (subfields) numbers... Draft
Requires
requires UIQM-601 Add error when user enters an unsuppo... Closed
requires UIQM-612 MARC create/edit/derive: LDR - add dr... Closed
requires UIQM-613 MARC authority create/edit: Split LDR... Closed
requires UIQM-614 MARC authority create/edit: Add hover... Closed
requires UIQM-615 MARC authority create/edit: LDR - add... Closed
requires UIQM-616 MARC holdings create/edit: Split LDR ... Closed
requires UIQM-617 MARC holdings create/edit: Add hover-... Closed
requires UIQM-618 MARC holdings create/edit: LDR - add ... Closed
requires UIQM-610 MARC record - create/edit/derive: Spl... In Refinement
requires UIQM-611 MARC create/edit/derive: Add hover-ov... In Refinement
Epic Link: quickMARC: MARC Validation
Front End Estimate: XL < 15 days
Front-End Confidence factor: 100%
Back End Estimate: Medium < 5 days
Back-End Confidence factor: 80%
Development Team: Spitfire
PO Rank: 0

 Description   

Goal: When a user creates/updates a record then more robust MARC validation should happen. Feature intent is to define default validation rules and allow the library (tenant level setting) to set custom rules. "Q" release focus will be on defining technical design approach.

To be considered:

  • The tech design should not be MARC centric meaning that it should be flexible enough to assign a set of validation rules per record type + format. 
  • The tech design should also allow for any flow that creates/updates MARC to opt-in to using these validation rules 
  • This feature will however focus on MARC. Additional features must be created to support other formats. 
  • Technical design will focus on MARC bibliographic and MARC authority records. 

Requirements/Scope:

  • Implement standard MARC21 validation rules
    • Repeatability 
    • Valid values (fields, indicators, subfields)
    • Mandatory/optional
    • See Bibliographic rules MODQM-390 Draft and MODQM-392 Draft
    • See Authority rules UXPROD-4361  and MODQM-393 Draft
  • Ability to revise default validation rules at tenant level and global level 
  • Ability to add new, local rules
    • Repeatability
    • Valid values (fields, indicators, subfields)
    • Mandatory/optional
  • Ability to bypass validation:
    • Default (MARC21)
    • ISBN/ISSN Validation
  • Ability to set whether validation rules should be a warning (warn, but permit save) OR error that prevents user from saving record
  • Ability to report catalog errors
  • Need to consider migration workflows where these rules can be ignored/bypassed. Same for data import workflows. 
  • Need to consider when default validation rules change due to change in MARC standards 
  • Consider ECS requirements


 Comments   
Comment by Michelangelo Viana [ 13/Jul/23 ]

Hi friends. Besides MARC21 (BIB/AUT/HOL) fields/indicators/subfields valid codes (according to https://www.loc.gov/marc/marcdocz.html), validation could include (a) mandatory fields control and (b) dependency between fields control, for example:

A) MANDATORY FIELDS:

040## MUST be present
245## MUST be present
902## MUST be present (local MARC field adopted by the library to every record)

B) DEPENDENCY BETWEEN FIELDS:

IF 245 0# is present, 1## ## CANNOT be present
IF 245 1# is present, 1## ## MUST be present
IF 593 ## is present, 013 ## MUST be present

And, at the suggestion of a librarian using the FOLIO: the MARC validation setup on the FOLIO could adopt a "visual" rules editor similar to the one adopted for Circulation Rules: https://github.com/folio-org/mod-circulation/blob/master/doc/circulationrules.md thus allowing adjustments to be flexible and easy to implement by the librarian (and not through hardcoded rules), or maybe by a dedicated visual editor inside Settings App (like Calendars, for example).

Comment by Khalilah Gambrell [ 14/Aug/23 ]
  • only allow certain unicodes (no emojis) – possibly >>> whitelist support
  • Initial bibliographic control based on 9XX 
Comment by Natalia Zaitseva [ 03/Nov/23 ]

as per the Q Release Planning session, we agreed on the following:

  • in Q release Kalibek Turgumbayev will work on the approach
  • the implementation part will be considered in the R Release

Estimation provided:
Back-End: M (5 days) for review (assumed that review from back-end will not be needed in Q release)
Front-End: XL (15 days) for the implementation part in Ramsons release

Generated at Fri Feb 09 00:36:01 UTC 2024 using Jira 1001.0.0-SNAPSHOT#100246-sha1:7a5c50119eb0633d306e14180817ddef5e80c75d.