2022-04-06 Data Import Subgroup meeting

Owned by Ann-Marie Breaux (Deactivated)
Last updated: Apr 06, 2022
5 min read

Recordings are posted Here (2022+) and Here (pre-2022)                   Slack channel for Q&A, discussion between meetings

Requirements details Here                                                                    Additional discussion topics in Subgroup parking lot


Attendees: Ann-Marie Breaux (Deactivated) Timothy Watters leeda.adkins@duke.edu Jennifer Eustis Taylor Smith Heather MacFarlane (Deactivated) Jenn Colt Lloyd Chittenden Lynne Fors Monica Arnold Raegan Wiechert Autumn Faulkner 

Lotus

Morning Glory

  • Morning Glory Folijet and Spitfire planning: dashboard where you can see the current scope and status of Data Import work for Morning Glory
  • Current work:
    • Adding Admin notes to the 3 Inventory field mapping screens
    • Field protections: fixed 1 bug for Lotus, rest of the fixes will be in Morning Glory
    • Starting log refinements (deleting logs)

Agenda topics:

Lotus Bugs

  • Deleting job logs
    • Will be possible from the UI on the Data Import landing page and the View all page, plus by directly hitting an endpoint in the backend
    • OK that the logs are delete and cannot be restored? OK per the SMEs
    • OK that (for now) deleting logs will be part of the Data Import: All permissions?
      • Rename DI: all permissions to be more descriptive: upload, import, view logs
      • Data Import - delete logs: separate from Data Import: All permissions
      • Settings (Data import): all permissions
      • Settings (Data import): view only
      • Would be nice to have a permission difference between EDI and MARC - after DI is feeling stable


Start here next week (just the questions). Main topic next week is MARC-MARC matching


  • MARC-MARC matching
    • Ann-Marie: Make sure the Lotus MARC-MARC matching tests show in Lotus BF, and assign to Jenn C
    • Is there a way to update the MARC-MARC matching so that an incoming 019 $a or 035 $a or 035 $z could match to 035 $a or 035 $z or 019 $a?
      • Is it possible to do this now? (with multiple match options from and to - would take 9 match profiles?)
    • Lotus: Allows for any field in a MARC record except
    • Are these needed in Morning Glory?
      • Matching for 100-899 fields? (I think they work, but not heavily tested yet)
      • Repeatable fields (e.g. 024, 035)
        • Incoming record: Only first version of the field is considered (NOTE: It's the first field that has the indicator(s) and/or subfield specified in the match profile)
        • It takes Ind 1, Ind 2, Subfield into account (in addition to the data)
        • Does FOLIO need to check all incoming 024s against all 024s in the existing SRS records? Or just the first?
        • Wildcards for Ind 1, Ind 2, Subfield (repeatable or non-repeatable fields)
          • Needed?
    • Additional info from A-M/Igor:

      • Let's pretend that these fields are in an incoming record: (Field Ind1 Ind2 Subfield)

        • 024 _ _ $a 12345
          024 1 1 $a 45678
          024 1 _ $x 67890
          024 2 2 $x 67890

      • And the fields in the existing SRS record are

        • 024 2 2 $x 67890
          024 _ _ $a 12345
          024 1 _ $x 13579
          024 1 1 $a 45678
          024 1 _ $x 67890

      • I understand that for repeatable fields, FOLIO Lotus only pays attention to the first incoming field, not the rest, but compares to any matching fields in the existing record.

      • Now - setting up different match profiles, I want to be sure I understand the logic that is in place now:

      • If the match profile is 024 _ _ $a: 

        • Matches, because the incoming first 024 looks for an existing 024 with blank indicators and $a and the same value (even though that is the second 024 in the existing record)


        If the match profile is 024 1 1 $a:

        • Matches, because the first incoming 024 with indicators 11 and $a (which is the second 024 in the incoming file) looks for an existing 024 with indicators 11 and $a and the same value (which is the fourth 024 in the existing record)


        If the match profile is 024 1 _ $x:

        • Matches, because the first incoming 024 with indicators 1_ and $x (which is the third 024 in the incoming file) looks for an existing 024 with indicators 1_ and $x and the same value (which is the fifth 024 in the existing record)


        If the match profile is 024 2 2  $x:

        • Matches, because the first incoming 024 with indicators 22 and $x (which is the fourth 024 in the incoming file) looks for an existing 024 with indicators 22 and $x and the same value (which is the first 024 in the existing record)
      • However!
        Let’s pretend the incoming record looks like this:
        • 024 1 1 $a 12345
          024 1 1 $a 45678
      • And the existing SRS record is
        • 024 1 1 $a 45678
      • If the match profile is 024 1 1 $a, SRS does not match, even though “024 1 1 $a 45678” is present in both incoming and existing records.
        SRS starts searching a field, that is specified in match profile, scrolling the incoming record from the very beginning, as usual, and takes the first occurrence of <024 1 1 $a>. The first occurrence is “024 1 1 $a 12345". So, SRS takes “024 1 1 $a 12345” and can’t find it in the existing record 


    • Future topics:
      • Results of Feb 2022 survey 
      • See if we can make the 005 modifiable, so that the existing one can be copied into a 9xx field via a MARC modification rule so that we can keep the date of last OCLC edit to a record (Lloyd)
      • Updated import stats for Lotus

Zoom chat

From Taylor Smith to Everyone 12:59 PM
hello!
sorry, my mic isn't working great right now so I'm just on keyboard

From Jenn Colt to Everyone 01:04 PM
Nick 🙁
lol i thought that was going to end differently

From Autumn Faulkner (she/her) to Everyone 01:24 PM
thanks so much for sharing that!
I have been anxious.

From Lynne Fors to Everyone 01:42 PM
Data import configuration permissions?

From Heather MacFarlane to Everyone 01:44 PM
So sorry, but I have to leave for another meeting...thank you!

From Jenn Colt to Everyone 01:45 PM
like i wish some people could do only edi or only marc loading

From Jennifer Eustis (she/her) to Everyone 01:56 PM
019 to 039 too
opps 035

From Jenn Colt to Everyone 01:59 PM
does do multiple match profiles not work?
doing…

From Jenn Colt to Everyone 02:07 PM
and also a qualifier?
ohhhh ok

From Taylor Smith to Everyone 02:08 PM
by everyone