Skip to end of banner
Go to start of banner

2021-09-01 Data Import Subgroup meeting

Skip to end of metadata
Go to start of metadata

You are viewing an old version of this page. View the current version.

Compare with Current View Page History

« Previous Version 8 Next »

 Recordings are posted Here

Slack channel for Q&A, discussion between meetings

Additional discussion topics in Subgroup parking lot

Attendees: Ann-Marie Breaux (Deactivated) Jennifer Eustis Monica Arnold , Tim Watters, Lisa McColl, leeda.adkins@duke.edu, Khalilah Gambrell Christie Thomas, Lloyd Chittenden,  Jose Alexander

Development update: 

  • Kiwi planning - dashboard where you can see the current scope and status of Data Import work for Kiwi
  • Current Data Import feature development dashboard and bugfix support
  • Sprint demo for sprints 120-121: Recording
    • Folijet demo'd the fix for cat date not respecting the tenant time zone, and UI unit test work
  • Juniper hotfix
    • Necessitated by some libraries having MARC records with multiple 001s in SRS (not via Data Import UI, but via direct insertion into SRS, bypassing UI and business logic)
    • Juniper migration script assumed each SRS MARC only has one 001 field; multiples cause the migration script to fail, so some libraries cannot upgrade to Juniper
    • Updated migration script and Multiple 001 check script to deal with this
    • Will add more info in Iris and Juniper release notes
    • Will also create a wiki page listing SRS cleanup scripts that can be run periodically to check for invalid data

Agenda topics: 

  • System updates and validation rules for MARC Authority records: current plans. Any concerns or questions? (development will be handled by the Spitfire dev team)  
    • Jennifer E:
      • inconsistent with MARC Bibs/Holdings
      • Having the manipulation and being able to add prefixes (the standard 001/003/035 manipulation) is helpful
    • Christie:
      • Would have to do pre-processing of all MARC authorities before they were loaded; would either need to delete them or move them somewhere else; if deleted, then might or might not replace the 001 with something else
      • if 001 is left as-is, might have accidental duplication if multiple authority vendors
      • How to identify different sources, if receiving records from multiple sources
      • A little strange that work is being done in the quickMARC group, without more consultation of the Entity Management group
      • If this was temporary, then it might be OK, but would need to have more options/become more robust in the future
      • If not going to manipulate, and is duplicate to 010 or other 0xx number field, maybe remove the 001/003?
    • Leeda: 003 has the source of the 001
      • 010 or 016 is Duke's update match point
    • Khalilah:
      • QM group has talked about being able to add default data to distinguish different sources (which can be accomplished with MARC modifications in the DI field mapping profile)
      • When get to the phase of linking the authority records to correct bib/instance holdings, then will definitely involve the Entity Management group
        • Per Christie: when that linking happens, there need to be options on whether an authority record controls headings in linked bib records
    • A-M:
      • 001 is an often used matchpoint
      • 001 as FOLIO HRID - give you a count/sense of size of records
    • Khalilah: 010 and UUID most often cited as numbers/identifiers to search on
    • Jennifer E: use the 001 for matchpoints for ETL; stromg preference is for consistency with handling of HRIDs in all 3 MARC record types
    • 001/003 Options
      • No system manipulation of 001/003 (unless library uses MARC modification to add/change the 003 or 9xx for source)
        • Quickest, but may be tricky to retrofit if different decision in the future
      • Follow the Bib/Holdings 001/003/035 manipulation
        • Would need to add a new settings screen
        • Not the preferred by the QM Subgroup, since HRIDs are not as necessary because they already have a value, and not showing in Inventory (yet)
      • Add the 001/003/035 manipulation at a later date
        • Cleaning up existing records 
      • Make it optional
        • The most work, since would require a new Setting, plus logic to make it optional
        • Would be consistent with what happens to Bibs/Instances and Holdings
    • May also need to revisit the whole 001/003/035 manipulation and refine it (remove duplicates, make it optional) A-M add to backlog
    • A-M: Add bug for 003-009 validation in MARC modification field mapping, to remove validation for subfields and indicators
  • Handling Inventory optimistic locking:  Data Import support work
  • New identifier types being added, will affect the default MARC-to-Instance map: MODDICORE-172
  • Need to return to the scenarios for 4 August 2021 of trying to match 
    • Another reason for doing a match after an action - marc mod to add an 003, then match, then take further actions
    • For field protections - be able to use the qualifier to determine if the field is protected
  • Also will soon provide details for Shared Rancher (sandbox) environment for labs and Import/Export/quickMARC/OAI-PMH noodling


  • No labels