2021-08-18 Data Import Subgroup meeting

Recordings are posted Here

Slack channel for Q&A, discussion between meetings

Additional discussion topics in Subgroup parking lot

Attendees: @Ann-Marie Breaux (Deactivated) @Timothy Watters @Jennifer Eustis @Ann Kardos @leeda.adkins@duke.edu @Lisa McColl @Christie Thomas @Monica Arnold @Lloyd Chittenden, Alex

Development update:

Kiwi planning - dashboard where you can see the current scope and status of Data Import work for Kiwi
Current Data Import feature development dashboard and bugfix support
Iris and Juniper release note updates
- EDIFACT invoices:
  - If copying the default field mapping profile, DO NOT remove the mapping for the Vendor Reference Number field, since it will cause the import to fail. Will be fixed in Kiwi.
  - Invoice files must have suffix of .edi or .inv (case insensitive). Starting in Kiwi, they should be able to have any file extension
- PTF analysis and Recommended Maximum File Sizes and Configuration
New wiki info:
- Recommended Maximum File Sizes and Configuration (as of Iris Hotfix 3 and Juniper)
  - Biggest issues to continue to work on:
    - Decreasing the processing size and increasing recommended file size for Inventory updates
    - Once authority records are importing, those will be large files
    - Larger loads for updates are important, but other priorities are higher (log enhancement, and especially update multiple holdings-items from 1 record, per @Jennifer Eustis and @Christie Thomas )
      - A-M to review outstanding reliability/performance work at end of Kiwi to see what if any big wins that could be done quickly in Lotus, and if any could be parked in favor of additional functionality
    - Ensuring that check-in/check-out time is less affected by imports
- Match profiles: Most commonly used
  - Possible lab topic: match on 856, and update holdings (@Jennifer Eustis)
- MARC field mapping syntax table added to Field Mapping profiles: MARC
  - Ability to do if/then based on content, e.g. vocabulary in $2 of a 650 field (in the default MARC-to-Instance map)
  - Ability to do if/then in the field mapping profile (e.g. 856 indicator)
  - A-M to find or create draft features so that we can start gathering use cases

Agenda topics:

Instance, Holdings, Item matching: Match profiles: Most commonly used
- Add a story for matching by stat code (what happens if there are multiples?)
  - Match:
    - If stat code is ?, then stop
    - If stat code is (something), then create a new bib
- Check the matching for locations (by name, code, and name (code))
Adding hotlinks to Import log
- UIDATIMP-988: Add hotlinks to the Import log summary for MARC Bib importsClosed (Inventory)
- UIDATIMP-989: Add hotlinks to the Import log summary for EDIFACT importsClosed (Invoice)
Need to return to the scenarios for 4 August 2021 of trying to match
- Another reason for doing a match after an action - marc mod to add an 003, then match, then take further actions
- For field protections - be able to use the qualifier to determine if the field is protected
Also will soon provide details for Shared Rancher (sandbox) environment for labs and Import/Export/quickMARC noodling

To do's

Finish follow-up from 4 Aug 2021 DI mtg (A-M)

Confirm Rancher environment for Data Import/quickMARC testing (Khalilah and Ann-Marie) - IN PROCESS

Data Import FAQ updates (Ann-Marie) - IN PROCESS

Make videos confirming the following scenarios

For import profiles that match on Holdings or Item IDs (and doesn't need to affect the Instance), make a video showing what happens to SRS MARC (Ann-Marie)

For a brief MARC updating the suppress status on a group of instances. What happens with the SRS MARC (A-M)

Talk with Kimie and see if ideas for updating the UI for implicit (A-M)

If a profile has update instance (to update cat date) which has an implicit update MARC, and also an update MARC (with implicit update instance) that has field protections, will the field protections be honored by the update instance? (A-M to check with devs)

Most important part is 1) making sure this is clearly documented, and 2) preferably, consistent, and 3) preferably, that implicit actions are acknowledged somehow in the import profile (A-M confirm with devs and document)