2022-04-20 Data Import Subgroup meeting

Owned by Ann-Marie Breaux (Deactivated)
Last updated: Apr 27, 2022
5 min read

Recordings are posted Here (2022+) and Here (pre-2022)                   Slack channel for Q&A, discussion between meetings

Requirements details Here                                                                    Additional discussion topics in Subgroup parking lot


Attendees: Ann-Marie Breaux (Deactivated) Jennifer Eustis  leeda.adkins@duke.edu Timothy Watters Monica Arnold Jenn Colt Lloyd Chittenden Raegan Wiechert Taylor Smith Christie Thomas 

Lotus

Morning Glory

  • Morning Glory Folijet and Spitfire planning: dashboard where you can see the current scope and status of Data Import work for Morning Glory
  • Completed work:
  • Current work:
    • Inventory matching from POL and VRN
      • Ask Dennis - was the holdings POL connection retroactive to existing holdings, or only applies to holdings related to orders placed after the change?
        • TBD - Dennis and A-M will review and discuss with Acq devs
    • Backend of deleting logs
    • Field protection refinement
      • Distinguish handling for repeatable and non-repeatable fields
      • Making field protections invoke if the job profile action is Update instances (rather than Update MARC Bib)
      • Leeda/Christie - when pairing an Instance update and a blank Update MARC Bib with each other, it would hang
        • Match 001 to Instance HRID
          • Update instance to change status and cat date
        • Match 001 to 001
          • Modify the MARC after the holdings and items are created to delete the acq 9xx data
        • Have not been able to get MARC modify and Instance update to work in the same profile
          • Still pending - A-M needs to triage
      • Jennifer E: 
        • Also trouble with this profile
          • Create instance
          • Create holdings
          • Create item
          • Modify MARC to take out 9xx acq data
        • Check this one also 
          • Still pending - A-M needs to triage
      • Still need to check on the interplay of field protections and MARC Modifications: Per Jenn/Christie, also make sure that the field protections do not prevent modifications on incoming records if the field is protected, e.g. 856 fields)
        • Still pending - A-M needs to triage
    • Flow control (to help Single Record Imports and MARC Authority UI deletes not get stuck)
    • Analysis of additional MARC-MARC matching refinement beyond what was delivered in Lotus
      • Matching from all repeated fields of an incoming MARC to all repeated fields in an existing MARC (e.g. all 035s to all 035s)
      • Adding indicator wildcards
      • Tangential: will analyze whether there is a way to deduplicate OCLC 035s with same number but different prefixes (seems like it might be needed during import and as on-demand)

Agenda topics:

  • Log refinement: finalize a few outstanding questions:
    • See PPT here
    • See Jira feature here
    • Job summary - stories not yet written
    • Make error details easier to find - stories not yet written
      • HOMEWORK: Review slides 9-21: If we want to show more error details on the summary screen (so that user doesn't have to click into the details screen for each error). how could we do that? Several examples of errors from Lotus Bugfest. How could this kind of information be consolidated into something meaningful on the log screen? Please add comments to this wiki page before next week's meeting.
    • Change log text for jobs stopped by User - stories written
    • Change from Load more to Pagination for long logs (like Inventory) - stories written - good change per Mark
    • Navigation from log back to Landing page vs View all page - stories not yet written - A-M review with devs
      • HOMEWORK: Review slides 27-29: Would this be a useful change? Right now users are always taken back to the Landing page, unless you use the browser Back button instead of the "x". If you have filtered jobs on the View all page, you'll lose those filters. Please add comments to this wiki page before next week's meeting.
    • Permissions enhancements - stories written


  • Should we start an implementers topics list like Acq SIG?
    • Development questions/discussion take priority; these would be discussion topics when there are not dev topics.
    • If so, and if A-M creates the page, would someone volunteer to maintain it? Yay Leeda!
    • Yes: A-M create page
      • Done


  • Jennifer: Would be helpful to have better handling of 856s in Holdings (which could also apply to any MARC mappings into Holdings and Item fields)
    • Be able to differentiate by indicator, so that user could specify import the resource's 856 only, or both the resource and a related one (differentiate by indicator)
      • Being able to differentiate by indicator would also allow for assigning different relationship values for different 856s; right now, the user can only pick 1 relationship value for all 856s being mapped into Holdings or Item
    • Making sure multiple 856s are not concatenated
    • Only brings in the first 856
    • Would there be a way to adjust mapping syntax to say "bring in all, bring in first only, pay attention to indicators"
      • A-M to talk with devs - how difficult? would it still be possible to validate in UI?
        • Jira created: UIDATIMP-1157 - Getting issue details... STATUS  Devs still need to evaluate. Will not be delivered in Morning Glory
      • Examples:
        • 856$u/* = import all 856s 
        • 856$u/1 = import first 856 only
        • question: what would 856$u trigger? all 856s or first only?
        • 856[1 ]$u = import only 856 with indicators 1[blank] (and if multiple 856s with the same indicators, would this trigger all to be imported or only the first one?)
        • 856[1 ]$u/1 = import only the first 856 with indicators 1[blank]


  • Future topics:
    • Results of Feb 2022 survey 
    • See if we can make the 005 modifiable, so that the existing one can be copied into a 9xx field via a MARC modification rule so that we can keep the date of last OCLC edit to a record (Lloyd)
    • Updated import stats for Lotus


Zoom chat: 

From Taylor Smith to Everyone 01:00 PM
hello

From Jenn Colt to Everyone 01:07 PM
When the UUIDs for the holdings types got changed. Hosting had to change it back to the old UUID

From Christie Thomas (she/her) to Everyone 01:17 PM
It would be great if both work because I find it useful to not update the instance and holdings if there is not an item match, too.
thank you!

From Jenn Colt to Everyone 01:17 PM
i don't want to redo my profiles 🙂

From Christie Thomas (she/her) to Everyone 01:17 PM
me either!

From Christie Thomas (she/her) to Everyone 01:35 PM
I am pretty sure it was not retroactive.

From Raegan Wiechert to Everyone 01:52 PM
People are talking

From Jenn Colt to Everyone 01:52 PM
i think you aren’t hearing us

From Christie Thomas (she/her) to Everyone 01:52 PM
I think you cannot hear us?

From Leeda Adkins to Everyone 01:52 PM
Can you hear us AM?

From Jennifer Eustis (she/her) to Everyone 01:53 PM
We can hear you

From Christie Thomas (she/her) to Everyone 01:53 PM
we can hear you.
I have to run to another meeting. I will see you all next week.