2022-01-26 Data Import Subgroup meeting

Owned by Ann-Marie Breaux (Deactivated)
Last updated: Jan 26, 2022
3 min read

 Recordings are posted Here

Slack channel for Q&A, discussion between meetings

Additional discussion topics in Subgroup parking lot

Attendees: Ann-Marie Breaux (Deactivated) Jennifer Eustis leeda.adkins@duke.edu Timothy Watters Jennifer Eustis Lisa McColl Monica Arnold Lloyd Chittenden Christie Thomas 

Lotus

Agenda topics: 

  • Bound withs
    • A-M would like some sample MARC Bibs, to make sure the Inventory changes work OK with Data Import. Note: DI does not support import of bound with info yet (A-M check the bug report from this morning)
    • Jenn Colt noted the 400 error in Inventory (nothing to do with DI) many times, like it's constantly polling for bound with info. 
  • Bug questions:
    • MODDATAIMP-427: confirm preferred action
      • Possible actions: 
        • Remove the 999 ff and create a new Instance/SRS (hard to clean up)
        • Discard the record and report an error
        • Do nothing and rely on users to create appropriate profiles (users will make mistakes, especially when using)
      • Preferred action: 
        • Not do nothing
        • Not create new record
          • If 999 ff and action is Create, discard the record
          • Report in the log: action was create, but incoming MARC had a 999 ff field
          • Where do we put error messages when SRS record is discarded? Can we show in the log UI?
      • Separate bug or story: Address Update when importing a file with 999 ff's
        • Use case: Export from 1 env and then import to other env and use an update action
        • If using a non-999 ff match (like 001) can lead to bad errors in the second env
        • If the action is update, then check something in SRS (999 ff $i, Instance UUID in SRS wrapper? something else?)
        • What if added an additional subfield in the 999 ff field to brand the source tenant of the incoming record, and if that didn't match, then stop and throw an error
        • If they don't match then
          • And action is Modify or Update, discard the record
          • Report in the log: action was Update, but incoming MARC had a 999 ff field that didn't match the existing record
      • Add a link for the tech documents that Kate just updated
        • See if it talks about the generation count (per Jenn, if 999 is stripped out, the generation count starts over)
  • Michigan State big invoice
    • POL/Invoice line matching - be careful during data migration (add link to video)
  • Data Import Log Enhancements
    • Updated mockups for purging logs (A-M add link)
      • Show/hide columns would be nice to have, but really mainly useful once user settings can persist across sessions
      • Mark: see Google gmail - has a select all on page, but also an option for select all results; Mark will send A-M screenshots
        • First iteration - only select all on a single page
      • It would be good to show the number selected 
    • Ask Devs - could there be an endpoint that libs could hit, in addition to the UI actions? A-M to check with Devs
    • Jenn: If something you're trying to filter is not on the first page, then you can't filter by it (user who hasn't imported in more than 100 imports, or older dates) - try a name or import profile; dates worked fine
    • Mockups for adding summary (A-M add link)
      • Make it more clear what error means - did it make some updates, or did it reject it completely
    • Still working with devs, and then Kimie, on possible log messages for  
      • Further processing stopped because there was a match/no-match to 1 hit, and then no further action specified in the job profile
      • Further processing stopped because there the match resulted in multiple hits, so no further action taken
  • Agenda for next week:
    • If this is the whole log scope for Morning Glory (e.g. no log exports), is that enough? Can we take care of other log stuff after Morning Glory?
  • Data Import: if we make permissions more granular in Morning Glory, what distinctions would be needed? A-M will send out a link to a document so that we can collect some comments before the meeting