2023-03-15 Data Import Subgroup meeting

Owned by Ann-Marie Breaux (Deactivated)
Last updated: Mar 15, 2023
4 min read

Recordings are posted Here (2022+) and Here (pre-2022)                   Slack channel for Q&A, discussion between meetings

Requirements details Here                                                                    Additional discussion topics in Subgroup parking lot


Attendees: Ann-Marie Breaux (Deactivated) Jennifer Eustis Lynne Fors Jenn Colt Kim Wiljanen Monica Arnold Lloyd Chittenden Lisa Smith 

Current development (Orchid)

Agenda: 

    • FOLIO Production Library Import Statistics
      • Please add statistics, especially for large files
      • Helpful to have EDIFACT stats too; per Kim, a file of 100 invoice lines takes ca. 10 mins
      • Per Jenn, impact on circ has decreased, but large imports still impact single record imports (even with the express lane for them) and catalogers; slowdowns and errors
        • A-M add a standard PTF/Folijet test for importing a large file, plus running sequential ISRIs
    • Importing MARC Bibs to create multiple holdings/items
      • See slide deck - any outstanding questions?
        • A-M: Make sure these are accounted for
          • If there are multiple 945s and the ones for a particular holdings are not next to each other is that OK? 
          • If there are multiple 945s to create multiple items, but for the same holdings, does the holdings perm loc/identifier need to be in the MARC Bib or not?
          • Can the holdings HRID or UUID be in the item 9xx and used for matching instead of the holdings perm loc 
      • Is there a non-edge use case for trying to create/update multiple holdings with the same perm loc from the same MARC Bib?
        • No one spoke up for it; in the rare cases where the same perm loc should be in 2 different holdings, for now, plan to manually straighten it out once import has created/updated in Inventory
        • If we can use Holdings HRID/UUID to differentiate and MARC record had that info in it, then DI needs to be able to handle it 
      • Will need to have some tests for multi-copy orders and inventory interactions; A-M will add task for that, plus various cases
        • P/E mix
        • Multiple copies for same location
        • Multiple copies spread across multiple locations
    • Bugfest and Orders UAT
      • Released some fixes to the UAT environment and Orchid Bugfest; more on the way
      • Automatic setting of Create inventory value for Open orders - can be tested on UAT env
      • Still need to make a couple UI changes to inactivate the Create inventory fields when Order status = Open
      • Create inventory automatic value is set based on 3 things:
        • Status = Open (determines that the system controls the Create inventory field; if status = pending, then system does not control the Ceate inventory value, and the user assigns the value in the field mapping profile)
        • Order format = P, E, P/E mix, Other (controls whether the Physical and/or E-resources Create inventory field is populated)
        • Inventory actions in the associated job profile (controls whether the Create inventory field is populated with None; Instance; Instance, holdings; Instance, holdings, item)
      • Seems to make sense to folks, but they need to work with it some more 
      • Just a few failed Bugfest cases so far
        • Log counts will be fixed in Poppy
        • Jenn reported a couple bugs where there's a different result when creating versus updating; developers are looking at those
      • Manual QAs are writing lots of TestRails and testing after every sprint; that has cut the amount of work that comes late in the release
      • Please finish any additional BF testing by Friday

Upcoming meetings/agenda topics:

  • 22 March
    • Review work from log subgroup and proposed updates in Poppy
    • Final review of ISRI multiple profile UAT, Orders UAT, Bugfest
  • 29 March 2023
    • Discuss Poppy scope
  • Misc
    • Discuss/review mockups for MARC updates refinements
    • POL/VRN matching
      • For invoices, we only consider open POs
      • For Instance, Holdings, Items, we currently (Orchid and before) only consider open POs. Should we change the Inventory matching logic to allow matching on closed POs as well?
    • Deleting outdated versions of SRS records
      • Can we define a cutoff date? 90 days ago? 1 year ago?
        • Different for records that are used during import and then not consulted again? (e.g. EDIFACT invoices, MARC bibs that only create/update orders, holdings, items)
      • Effects on the import log
    • OCLC number cleanup
      • Confirm 035 structure, aim for it to be consistent across all FOLIO tenants
    • Downloading log info
      • Lots of interest, especially for errors
      • Including identifiers for everything
      • What would UI look like?
      • What would output look like? 
    • Variation between PTF and production library performance results - why?



Chat

Kimberly Wiljanen  to  Everyone 1:14 PM
Are you interested in EDI invoice imports also?

Lisa Smith, Mich State  to  Everyone 1:23 PM
Kim - Mich State

Lynne Fors  to  Everyone 1:35 PM
Could you create using the Location Code instead of the full location name?  Our names are extremely long

Lisa Smith, Mich State  to  Everyone 1:46 PM
We are still creating instance only!

Kimberly Wiljanen  to  Everyone 2:01 PM
  I like it!

Lynne Fors  to  Everyone 2:02 PM
I think I need to play with it first