quickMARC: MARC Validation (UXPROD-3985)

[UXPROD-4201] MARC Definition Created: 14/Apr/23  Updated: 08/Dec/23

Status: Open
Project: UX Product
Components: None
Affects versions: None
Fix versions: None
Parent: quickMARC: MARC Validation

Type: New Feature Priority: P2
Reporter: Jacek Gajkiewicz Assignee: Unassigned
Resolution: Unresolved Votes: 1
Labels: authority_control, bibframe, bibliographic_record, metadata-definition, metadatamanagement, mol, validation
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original estimate: Not Specified

Attachments: Microsoft PowerPoint MARC definition.pptx    
Issue links:
Relates
relates to UXPROD-4286 Feature: MARC validation rules | BF C... Open
Epic Link: quickMARC: MARC Validation
Development Team: Spitfire
PO Rank: 0

 Description   

Please refer to the attached pptx which describes basics of this idea.

As a user I want to set my own rules which describe how MARC records work in my FOLIO tenant. That includes, but should not be limited to: validation rules, authority control rules, local thesauri, default values, contextual help.

To do that, an extensive definition of all types of MARC records on tenant level should be built. User should be able to describe MARC record via GUI, giving detailed information. An example of a variable field definition: filed tag, description, status: repeatable/ non repeatable, obligatory/ optional, indicators: list of proper values, default values, subfields: list of proper subfields, their order, repeatability; information if any of the subfields should be controlled by authority or thesaurus, in what manner, etc.

The definition itself is not intended to make any direct changes of logic. Thus it can be built independently of any other FOLIO components and would not affect the behavior of FOLIO. Once it is built, it can serve as a source of information for all modules managing MARC records. This can be done step by step, for example at first applying information from the definition for validation purposes, then for authority control rules, and so on.

The benefits of that approach: definitions and rules that are currently hard-coded or set in other forms in different parts of FOLIO would be gathered together, easy to manage by users on tenant level. This would allow users to tailor FOLIO to their needs without any development effort.

Questions

However such an approach might be difficult to apply, as there are some MARC functionality already done and it needs to be rewritten again. On the other hand, there are still a lot of functionality to be done and that could utilize the new approach. Taking into account the existing FOLIO logic, there are many questions to be considered, for example:

  1. The most important question is whether the definition should or could affect the structure of inventory instance mapping rules or even the inventory instance structure and search rules? The examples of user cases to be considered here: user makes new MARC field/subfield and wants that information to be searchable and displayed in FOLIO, user deletes a MARC field which is already mapped into instance, etc.
  1. Do we need to review and rewrite all already existing cataloging rules? For example: https://folio-org.atlassian.net/wiki/display/FOLIJET/Authority+linking-mapping+rules
  2. Do we need definition for MARC holdings records? I need to mention that actual construction of Folio does not allow multiply $852, but this field should be repeatable. If we would like to make def rules for holding we have to change folio holdings (and maybe Items also) https://folio-org.atlassian.net/wiki/display/FOLIJET/MARC+Holdings-Default+Mapping+profile
  3. What would be the impact for the imports? For example what to do if the user will import files with a subfield which does not exist in definition.
  4. Possible impact for electronic resources? We don’t know anything about the format for these kind resources. What if they are in Dublin Core? Need to have another solution for mapping Dublin Core to the MARC and then to Folio? Or just Dublin Core to Folio and we don’t need to think about it. https://folio-org.atlassian.net/browse/DEBT-59
  5. Do we need to create default rules? It will be useful for the users who will change rules for ex. for tests and then would like to make a restore. How many of these default rules should be ? Manufacturer rules? User rules? Do we need default rules for every type of material or authority?
  6. We need to think of additional tools. Imagine importing of 10000 bib records, when half of them are not validated by my rules but they are imported. I need to know which of them are not validated. I need to make global changes to validate these records properly. Is it possible in this kind of system architecture? How could it be done?
  7. Do we plan logic for fixed length fields? For example:
  • in bib choose 008/22=d and then fill out 385$a? At least any validation?
  • in authority define 008/08=f (language of catalog)

Last, but not least question: can that approach be applied in building bibframe functionality for LoC?

Scope of definition

Bibliographic Rec.

Action: CRUD - it means all contents are editable, changeable, deletable

Fixed - length field 

  1. MARC Tag
  • Name
  • Description
  • List of data elements for all types of material
  • List of possible values and natural language description for all of data elements
  • Default data element

Variable length field

  1. MARC Tag
  • Name
  • Description
  • Repeatable / Non-repeatable
  • Obligatory / Optional
  1. Indicators
  • On / Off
  • Name
  • Acceptable content
  • Default content
  • Logic: Nonfiling characters (YES/NO)
  • Supported field by nonfiling characters
  1. Subfields
  • Name
  • Description
  • Repeatable / Non-repeatable
  • Subfield order
  1. Authority Control

Connection between authority and bib on subfield level. So you will be able to decide which subfield in bib rec. is controlled by which subfield in authority. Examples: 650$a is controlled by field 150$a

650$a$x$z$y$v is controlled by the 150$a$x$z$y

385$m$a is controlled by the 150$a (subfield 385$m is uncontrolled)

  1. Every subfield can be assigned to the type of material determined by LDR/06 and LDR/07 as a obligatory or optional
  2. Every subfield can be obligatory or optional on the type of material level
  3. On the type of material level user can define:
  • User can define order of subfield on type of material level making his own validation rules for example: 260$a$b$c, 260$a$b$a$b$c, 260$a$a$b$c etc.
  • User can define signs before subfields [like: .,=: etc.] making his own validation rules. For example before subfield 245$b he can allow signs like :.=
  • User can define sign on the end of the field making, making his own validation rules
  • User can define signs inside the subfield for example () for subfield 100$d

Authority Records

Action: CRUD -  it means all contents are editable, changeable, deletable

 

Fixed - length field 

  1. MARC Tag
  • Name
  • Description
  • List of data elements f
  • List of possible values and natural language description for all of data elements
  • Default data element

Variable length field

Action: CRUD

  1. MARC Tag
  • Name
  • Description
  • Repeatable / Non-repeatable
  • Obligatory / Optional
  1. Indicators
  • On / Off
  • Name
  • Acceptable content
  • Default content
  • Logic: Nonfiling characters (YES/NO)
  • Supported field by nonfiling characters
  1. Subfields
  • Name
  • Description
  • Repeatable / Non-repeatable
  • Subfield order
  1. Type of authority records
  • user can define type of authority records by the 008/15, 008/14, 008/16
  • user assign subfield to the type of authorities as an obligatory or optional
  1. On the type of authority records level user can define:
  • User can define order of subfield on type of material level making his own validation rules
  • User can define signs before subfields [like: .,=: etc.] making his own validation rules
  • User can define sign on the end of the field making, making his own validation rules
  • User can define signs inside the subfield for example: () for subfield 100$d

Subject heading system/thesaurus:

  • support for 2nd indicator in the bib records, for fields which are controlled by the authority
  • support for 008/11 in the authority records which means
  • user can define his own content for both: 2nd indicator and 008/11 and make his own unique thesaurus

Generated at Fri Feb 09 00:38:03 UTC 2024 using Jira 1001.0.0-SNAPSHOT#100246-sha1:7a5c50119eb0633d306e14180817ddef5e80c75d.