2021-05-25 Meeting Notes

Data Import Update 20210525

Folijet is the dev team for Data Import. Khalilah's team Spitfire is also helping with Data Import features

Background & Scope

mid Nov 2020 - MARCCat not being dev'd quickly enough

Dec 2020 - need to make sure import/export worked smoothly, but trying to do this and QuickMARC and other issues was too much. All MARCcat work stopped.

Khalilah and SpitFire working on QuickMARC now.  Folijet will focus on performance and infrastructe and import/export data.

Releases

• Iris
  • Import EDIFACT invoices
  • Expand Import log
  • Support Thor's work onInventory Single Record Import
  • Refactor Data import from PubSub using Kafka to Dafda-direct
  • Misc smaller tasks:
    • Change Inv HRID to use w/out leading zeros
    • Update default MARC to Instance profile for 880 field
    • Replace the "secret button" default import profile with a default job profile
    • Support Spitfire's work on QuickMARC/Derive
• Juniper
  • June 4 freeze date
  • Critical hotfixes for Iris
  • Stabilization and performance, working with PTF
  • Provide assistance  knowledge share with Spitfire
    • Store auth records in SRS
    • Changing quickMARC workflow and interaction with SRS to Kafka-direct
  • Implement Keyboard shortcuts
  • These will probably slip to Kiwi
    • Increase security for Data Import
    • Make updates for field mappings and default MARC to Instance mappings
    • refine error handling in Data Import log
    • Karate integration tests
    • Q- when will we be able to test against Inventory?
    • A- can't test MARC imports and orders right now, can test several other things; need real-world cases
    • Q - still a lot of questions about why certain things aren't working
• Kiwi
  • Additional infrastructure stability and performance
  • Sept 24 freeze
  • Continue DI stabilization
  • Other possible at-risk work from Juniper (see italicized list above)
  • UI end-to-end automated tests
  • Will likely push out
    • Implement DI for MARC Invoice data
    • Implement DI for MARC Order data
    • Optimistic locking*
• Beyond
  • Implement DI for MARC Invoice data
  • Implement DI for MARC Order data
  • Optimistic locking*
  • Interactive log
  • create/update multiple hldgs & Items
  • Ability to preview import
  • Refinements for EDIFACT invoice imports
  • Add support for tokens in field mapping profiles

Khalilah - spitfire in Juniper is working on an alert for problems with  mod-import - sort of a troubleshooting tool

Q - overlay - hearing that SRS is causing problems?

AMB - bugs in Iris that we're working on, updates were creating new instances,

Q - From Implementer perspective, please explain "performance" and "stability".  Serious, maybe hidden, data issues will arise, data integrity is an issue.

AMB - yes, we want things to be right, we have to clean up some things before trying to work on new features.  Would welcome new test case scenarios