2021-07-14 Data Import Subgroup meeting

Owned by Ann-Marie Breaux (Deactivated)
Last updated: Jul 14, 2021
4 min read

 Recordings are posted Here

Slack channel for Q&A, discussion between meetings

Additional discussion topics in Subgroup parking lot

Attendees: Ann-Marie Breaux (Deactivated) Jennifer Eustis leeda.adkins@duke.edu Christie Thomas Jenn Colt Tim Watters

Agenda topics:

  1. Open these meetings to the whole Slack channel?
    1. Mention in the channel, for dev, join this mtg, for libraries getting ready to go live, join the lab
  2. Volunteer to report out to MM SIG on a monthly basis (plus a backup)? Date will probably be synced to the SIG reports to the PC.
    1. Jennifer Eustisvolunteered
  3. Current development: Sprint 118:
    1. Support board (Iris hotfixes, Juniper bugfixes)
    2. Development board (Prep for Kiwi, Documentation, Knowledge share with Spitfire (SRS, MARC Holdings) and Vega (PubSub), SME Rancher env)
    3. Other prep for Kiwi: Automated test planning, Bound-with meeting this week, Optimistic locking meeting next week 
  4. Has anyone created successful MARC to Instance Identifier matches for 010, 020, 022, 024, 035, 9xx fields? If so, it would be great to see examples. Developers still need to work on some identifier matching bug fixes (see examples in homework below) (come back to this in a future meeting)
    1. Would it be useful to have a wiki page of successful match examples? 
      1. Yes Ann-Marie Breaux (Deactivated)will set up and advertise
      2. And can we make sure to deal with the MARC-MARC 0xx and 9xx fields, even if the research spike proves difficult?
      3. Most concerning issue with MARC-Instance matches is that will also update the instance, which may not be desired (which circles back to cleaning up field protection)
  5. Bug when creating Instances instead of matching/updating them: MODDATAIMP-427
    1. Review the bug and confirm preferred action
    2. Whether it's Create or Update, wipe out the 001 and 999 ff and create them again
      1. If Create
        1. Do the 001/003/035 manipulation
          1. Which still needs to be cleaned up - for the dups or almost dups; find the Jira and raise priority
          2. Maybe end-goal is to make this manipulation a job profile preference - how much work would it be?
        2. Create the 999 ff with the UUIDs
      2. If Update
        1. After the match, wipe out the 001 and 999 ff
        2. Then populate the 001/999 ff with the numbers from the existing SRS record or the Instance (if no prior SRS)
        3. And find Jenn's bug about extra 001s that are being created
        4. Make sure the FAQ is clear on which UUID is the 999 ff $s - (the matched ID)
    3. Not planned as an Iris hotfix since it's an edge case. Needed as a Juniper bugfix? Or can it wait until Kiwi release?
  6. Preferred UI info when a user cancels an import: MODSOURMAN-492
    1. Cancelled OK as the term? (to distinguish from Completed or Failed
      1. Does FOLIO ever fail anything?
    2. Run by user - should it be the person who started the job or the person who stopped the job?
      1. Can we show both? If only 1, then show the Run person
  7. Testing
    1. Per Christie Thomastesting is getting overwhelming, with the 2 Bugfests, folio-snapshot, Rancher, local test envs
      1. For go-live libraries, most important is the sandbox that has your real data in it (and maybe functionality that doesn't exist in the shared envs)
      2. For libraries not yet going live, which environment is closest to your go-live env? probably Iris Bugfest or Juniper Bugfest
      3. For functionality that is not yet released (Juniper bugfixes that are not yet merged, Iris hotfixes that are not yet merged, Kiwi development): folio-snapshot-load, SME Rancher env 
    2. Per Jenn Colt, spent a ton of time on folio-snapshot and Iris bugfix, sometimes to the detriment of their local testing
  8. Juniper Bugfest
    1. Testing feedback?
    2. Exploratory tests?
      1. Christie Thomasplanning to adopt some of the exploratory tests to test their most important workflows
    3. EDIFACT invoice importing has a regression in Juniper. Working on figuring that out, plus expanding the documentation
      1. EDI invoice questions should go to Data Import
  9. Ann-Marie Breaux (Deactivated)on vacation next week, so 21 July meeting cancelled

HOMEWORK from last meeting:

  1. Has anyone created successful MARC to Instance Identifier matches for 010, 020, 022, 024, 035, 9xx fields? If so, it would be great to see examples. Developers still need to work on some identifier matching bug fixes

    • Lisa (since I won't be able to make the meeting today, putting notes here): In Iris bugfest I was able to overlay with exact matches on the 010$a, 022$a, 024$a, and 035$a. The 020 to ISBN did not work I think because it appears the 020 in bugfest was not mapped to the identifier of ISBN in the Instance record. Example: HRID: in871746

      Inventory ViewView in quickMARC

    • Jennifer: I tested this about 2 weeks ago about for the match on 010, 020, 035 only. And I was lucky that there weren't multiple OCLC matches. This seemed to work and here's the video: https://umass-amherst.zoom.us/rec/share/yeOUyWSxxgRdf3YZielNYFZw-uskf0nGA3SmrMpcF3fYAc_G3YqYaxIQtgKd5wZZ.N83MYbSHPlqF9TqC. Note: The video will only 90 days online. Ann-Marie Breaux (Deactivated)to see if we can download the video and move to Data Import Google drive

To do's (plus ones above)

  • Confirm Rancher environment for Data Import/quickMARC testing (Khalilah and Ann-Marie) - IN PROCESS
  • Data Import FAQ updates (Ann-Marie) - IN PROCESS
  • Make videos confirming the following scenarios
    • For import profiles that match on Holdings or Item IDs (and doesn't need to affect the Instance), make a video showing what happens to SRS MARC (Ann-Marie)
    • For a brief MARC updating the suppress status on a group of instances. What happens with the SRS MARC (A-M)
  • Talk with Kimie and see if ideas for updating the UI for implicit (A-M)
  • If a profile has update instance (to update cat date) which has an implicit update MARC, and also an update MARC (with implicit update instance) that has field protections, will the field protections be honored by the update instance? (A-M to check with devs)
  • Most important part is 1) making sure this is clearly documented, and 2) preferably, consistent, and 3) preferably, that implicit actions are acknowledged somehow in the import profile (A-M confirm with devs and document)
    1. QM edit: increments
    2. Import MARC Modify: increments, we think
    3. Import MARC Update: does not increment
    4. Import Instance Update (implicit MARC update): increments