2022-02-02 Data Import Subgroup meeting

Owned by Ann-Marie Breaux (Deactivated)
Last updated: Feb 02, 2022
4 min read

Recordings are posted Here                                           Slack channel for Q&A, discussion between meetings

Requirements details Here                                             Additional discussion topics in Subgroup parking lot


Attendees: Ann-Marie Breaux (Deactivated)  Jennifer Eustis leeda.adkins@duke.edu Timothy Watters Jenn Colt Lisa McColl Raegan Wiechert Christie Thomas 

Lotus

Agenda topics: 

  • Data Import Log Enhancements
    • Data Import Log Enhancements
    • Updated mockups for purging logs (A-M add link)
    • Mockups for adding summary (A-M add link)
      • Maybe default to filtered for Errors, and maybe discards?
      • Distinguish Discards and Rejects
        • Discard: I got to the end of the profile decisions, and there were no more actions
        • Rejects: I couldn't create a item record because there was a problem; per the job profile, another action should have been taken
          • Reject = Error
          • Set up categories, e.g. Missing required data, System error - in the error column, replace the "error" message with the category, and make it a hotlink direct to the JSON info
        • Have the counts add up in each column; have error under each column
        • Have hotlinks for discards and errors = filters
        • Is it more complicated/Expensive to have all the outcomes in the summary be hotlinks?
        • If have these categories, could they be gettable from an API endpoint, so library could create/send summary of the job to the requestor
        • Failed: import died before it could do anything
        • Matches
          • No match:
            • If no match, don't take an action: Discard
            • If no match, and I was supposed to take an action, but it couldn't: Error
          • Multiples:
            • If multiple matches, and I was supposed to take an action, but it couldn't: Error (category: multiple matches, unable to proceed)
      • A-M to do's
        • Look at Magda's bulk edit screen; any ideas to borrow from that?
        • Invalid data in the Leader - a common FOLIO error per Christie
        • Set up spreadsheet and post link - gather errors from real-world scenarios; see how they break out
        • Update mockups with Kimie
      • Get rid of the JSON link? No, but have buckets for the different kinds of errors - have error categories (e.g. missing required info, but show the actual errors in the error column) - can we get a list of errors that DI already understands and accounts for? If no identifiable category, assign Other
    • Still working with devs, and then Kimie, on possible log messages for  
      • Further processing stopped because there was a match/no-match to 1 hit, and then no further action specified in the job profile
      • Further processing stopped because there the match resulted in multiple hits, so no further action taken
    • Is there something else for the logs that would make them more helpful for troubleshooting?
    • If this is the whole log scope for Morning Glory (e.g. no log exports), is that enough? 
    • Maybe a poll to see which log changes are the most important

Next meeting

  • Permissions: if we make permissions more granular in Morning Glory, what distinctions would be needed? 
    • Data Import Permissions Refinement
    • Current UI Permissions: 
      • Data Import: All permissions
      • Settings (Data import): Can view, create, edit, remove
      • Inventory Single Record Import has separate permissions - leave those alone?

        • Inventory: Import single bibliographic records

        • Settings (Inventory): Configure single-record import

Zoom chat: 

From Jennifer Eustis (she/her) to Everyone 01:00 PM
It's supposed to be ice and snow :(

From Christie Thomas (she/her) to Everyone 01:09 PM
discard and reject are two different things.

From Jennifer Eustis (she/her) to Everyone 01:10 PM
will all discards be an error?

From Christie Thomas (she/her) to Everyone 01:10 PM
one no action is taken and that is expected. the other means that something went wrong.

From Leeda Adkins to Everyone 01:19 PM
Our current system calls them New or Match records, and Multi

From Jennifer Eustis (she/her) to Everyone 01:21 PM
brb

From Christie Thomas (she/her) to Everyone 01:21 PM
I guess I thought that the JSON was a temporary status.

From Jenn Colt to Everyone 01:22 PM
My folks definitely forget to go look for the json

From Lisa McColl to Everyone 01:22 PM
Agreed - I thought JSON was stop-gap as well

From Jenn Colt to Everyone 01:23 PM
I think the detail there is good for me but most people don’t need to see it
We also need things to actually fail so there even is a log, and not just hang

From Jennifer Eustis (she/her) to Everyone 01:32 PM
back …

From Christie Thomas (she/her) to Everyone 01:43 PM
I think so
I would consider that a rejection.

From Raegan Wiechert to Everyone 01:43 PM
would that be a fail?

From Christie Thomas (she/her) to Everyone 01:46 PM
i would be comfortable putting that in rejected as long as the error had a label that was accurate for the scenario. So that we know all we need to do is try again.
i think so

From Jenn Colt to Everyone 01:57 PM
One thing I worry about with sorting is that I never feel fully confident if the sort really worked on a big job

From Leeda Adkins to Everyone 01:57 PM
+1 Jenn