2022-06-15 Data Import Subgroup meeting

Owned by Ann-Marie Breaux (Deactivated)
Last updated: Jun 15, 2022
6 min read

Recordings are posted Here (2022+) and Here (pre-2022)                   Slack channel for Q&A, discussion between meetings

Requirements details Here                                                                    Additional discussion topics in Subgroup parking lot


Attendees: Ann-Marie Breaux (Deactivated) Jennifer Eustis leeda.adkins@duke.edu Timothy Watters Autumn Faulkner Christie Thomas Heather MacFarlane (Deactivated) Jenn Colt Lloyd Chittenden Taylor Smith 

Morning Glory

Agenda topics:

  • Questions from devs: If there is a 999 ff in the incoming MARC Bib record, and the Instance action is Create – should that automatically fail? Are there any use cases where it would be valid for an incoming MARC record that is creating to already have a 999 ff in it?
    • If this mainly happens due to a user selecting a Create action instead of an Update, then is it more helpful to constrain the Create action or rely on more user experience/training? 
    • It should automatically fail, especially since not currently easy to resolve extra instances that are created
      • Bulk delete of instances not scheduled until Queen Ann's Lace, but Inventory delete of instances may be sooner. A-M confirming with Charlotte
    • If doing an Inventory Single Record Import to create, and pulling a record from another FOLIO tenant, it would have the other tenant's 999 and 001. Seems like those should be stripped, and new IDs assigned in the new tenant. Not aware of any library doing this currently. Would mean an extra bit of handling for Inventory Single Record Import Creates.


  • MARC field protections - Initial User Acceptance testing for Morning Glory
    • A-M will try more tonight, see if there's a workable job profile structure, see if can make a chart
    • Use tomorrow's lab for more experimenting
      • Can there be a MARC modify and MARC update action in the same job?
      • If overriding field protections, does it matter if the Update MARC action is before or after the Update instance action?
      • If match Instance-MARC, can a job with Update Instance and Update MARC succeed? Does it matter which order the actions are in?
      • If match MARC-MARC, can a job with Update Instance and Update MARC succeed? Does it matter which order the actions are in?
      • If 2 separate matches (Instance-MARC, and MARC-MARC), can a job with Update Instance and Update MARC succeed? Does it matter which order the actions are in?
    • From Christie: So if I want to export a record, change a protected field and then reimport the record and have only the changes I made in MarcEdit, how would I do that?
    • From Jennifer: If MARC modification in the job profile, field protections not working as expected, regardless of whether the Modify action was at the beginning or end of the job profile
    • From Christie: I have only been able to get that to work with creates and not with updates. Have you ever been able to do a marc update + a marc modify in the same job? I have not.
    • From Leeda: I feel like modify should be treated as an incoming record action, and therefore should not invoke field protections.
Field protection settingLibrary/UserWorks as expected?NotesAdd a TestRail?
035 9\

Jennifer Eustis 

Not the way I expected. I created inventory for 15 eResources with a temporary code in the 035 9\ $a(5LHFCJSP1). Then I exported those so I could match on the instance HRID to create an additional holding and item. Then I included a marc modify action linked to a marc modifications mapping that removed the 1 to have 035 9\$a(5LHFCJSP). This didn't work. I didn't see anywhere where I could override the field protections in the marc modify action or mapping profile. Here I was taking action on the incoming record.Right now the only way we are able to make this change is to update the instances. If field protections are on update instances then we will not be able to do these batch loads in FOLIO at all. We simply don't have the resources to do this manually and there's no bulk edit.
035 9\

Jennifer Eustis 

Since the marc modification didn't work on the incoming record in my other test, I tried to use an update marc action linked to a mapping. This is an update where I match on the instance HRID to create an additional holding and item and then update the marc. Right now this job is stuck with in progress. And an hour later it's still stuck.

856

Jennifer Eustis 

Didn't work. I wanted to create inventory for eResources so I need the 856 to populate the Electronic access in the holdings record. The last action in the job is to modify the marc to remove the 856. This didn't happen. the log has "multiple" for instance and "updated" for srs marc.

856Jenn Colt I need to protect the 856 so that overlays with single record import don't delete our local 856s. But I need to update 856s when I import updated vendor records. Because the field protections were extended to instances, the 856 was not updated as needed and instead the 856 was duplicated. If this is implemented as is it will break ALL OUR VENDOR UPDATE WORKFLOWS. If field protections are extended to instances we need to be able to override them at the job level. We (and many others) use instance overlay to correct a large number of data import problems. We need to have an instance overlay profile that ignores field protections without having to manually turn them all off.

035/003/001Jenn Colt When I updated my records, the instance ID got turned into an 035 with the incoming 003 even though this was a pure vendor record update (ie records not exported from folio). I assume this is field protection related but I don't understand why it happened. Screenshot attached.

035/856

Jennifer Eustis 

I created a marc modify action linked to a mapping that removed fields not in the protected field lists (029, 891, etc), and that removed the 856 (in the protected fields list) and tried to remove the 2-3 letters before the OCLC number. This job was to create an instance (and implicitly a marc bib srs) only. The Marc modify action came before a create instance.

This didn't work as expected. The 856 wasn't removed. The 2-3 letters before the OCLC number were removed with the 035 already present. However, the 035 generated from the 001 and 003 still had the 3 letters before the OCLC number. My expectation is that all 035s whether generated by the system or already present in the record would undergo that marc modification mapping.

This is a test to try and get a profile up and running for single record import that removes fields we don't want from OCLC and those letters before the OCLC number which 5C doesn't want.




























Chat:

From Taylor Smith to Everyone 01:02 PM
hello
From Autumn Faulkner (she/her) to Everyone 01:14 PM
per the bulk edit roadmap, I don't think it's planned till 2023
https://docs.google.com/spreadsheets/d/108EpsW4Um1T27Kvl4rh-VwbCQlgJzEEq/edit?pli=1#gid=1948078897
Instances delete comes in QAL
ohhhhhh right
yay!

From Christie Thomas (she/her) to Everyone 01:29 PM
+1 Leeda

So if I want to export a record, change a protected field and then reimport the record and have only the changes I made in MarcEdit, how would I do that?
I have only been able to get that to work with creates and not with updates.
Have you ever been able to do a marc update + a marc modify in the same job? I have not.

From Leeda Adkins to Everyone 01:30 PM
I feel like modify should be treated as an incoming record action, and therefore should not invoke field protections.

From Christie Thomas (she/her) to Everyone 01:34 PM
+1 Leeda re: marc modify applying to the incoming record and not invoking field protections.

From Jenn Colt to Everyone 01:35 PM
same in voyager

From Christie Thomas (she/her) to Everyone 01:38 PM
IN our previous system we had global protections and job level protections.
And we could override any of the global protections in the job.
But they will both update the instance and the srs - so you will be updating the instance twice and the srs twice and I have never been able to make that work either.

From Jenn Colt to Everyone 01:59 PM
I have a 2pm I can't skip. I'll be at lab tomorrow.

From Christie Thomas (she/her) to Everyone 02:00 PM
Me, too.