MARC field protections apply to MARC modifications of incoming records when they should not
Description
CSP Request Details
CSP Rejection Details
Potential Workaround
Attachments
blocks
defines
relates to
Checklist
hideTestRail: Results
Activity
Charlotte Whitt July 31, 2023 at 1:48 PM
and - should the findings Jennifer reports above be filed as a new bug ticket and then given the label `regression`?
Jennifer Eustis July 27, 2023 at 5:05 PM
Hi and ,
We are testing our dry run Orchid right now. As part of that, we are testing marc modifications on single record import and overlay. Single record import works as expected.
Single record overlay doesn't. I have a profile that is working. However, the marc modifications which is to remove fields from the incoming record are removing those fields form the incoming record and the existing srs marc record in FOLIO. This seems like a regression.
Using the profile below, we tested this with a MARC source file that had a 583 $5 for our EAST commitment which is protected. When we overlaid, that 583 in our existing record was no longer there.
I also tried building a profile where the marc modifications came at the beginning. See image below. I tested this for overlaying a source marc file and the same result, the 583 in the existing srs marc file was removed.
Ann-Marie Breaux September 7, 2022 at 4:43 PM
Hi and Tested the 3 non-Draft TestRails linked to this bug, which includes the 2 scenarios in the bug. All worked properly on MG Bugfest. Also added info to each TestRail about the log Updated/Multiple text being fixed in a separate bug in the future. Closing this issue.
Thank you for sorting it out, !
Oleksii Petrenko September 7, 2022 at 11:13 AM
Deployed to MG bf. Please proceed with verification.
CC
Charlotte Whitt September 6, 2022 at 11:19 AM
Support SIG: or can this work be deployed, so we can get it verified in MG Bugfest environment?
Details
Details
Assignee
Ruslan LavrovReporter
Jenn ColtPriority
P2
Overview: When bringing in a file of MARC records we sometimes want to change that file using MARC modifications. That file should be able to be updated freely, without regard for field protection because that MARC does not yet exist in FOLIO.
Current workaround: None
General Steps to Reproduce:
NOTE: This is corrected in Morning Glory, but has not been backported to Lotus or previous versions
Log into kiwi bugfest (also observed on Cornell's Juniper production system)
Create a job profile that has MARC modifications for an incoming MARC file and make sure those modifications include fields that are protected (protections fire via MARC Update actions; MARC modifications fire via MARC Modify Actions). It is easiest to see with a create record profile rather than update one.
Run the job using a file that has the fields that are supposed to be modified and that are also protected.
Expected Results:
The MARC modifications are made on the incoming file and loaded into FOLIO, even if the modified field is normally a protected one
The SRS MARC status in the log is Created (since this MARC Bib did not exist in SRS prior to this import)
The Instance status in the log is Created (since this Instance did not existing prior to this import
Actual Results:
The protected fields are not modified
The SRS MARC status in the log is Updated
The Instance status in the log is Multiple, with no Instance hotlink (which makes the user think multiple instances have been created)
Specific scenarios to reproduce
Scenario 1
See attached Scenario1-Part1_MODDICORE-248.mp4, Scenario1-Part2_MODDICORE-248.mp4, Scenario1_MODDICORE-248.mrc, and TestRail https://foliotest.testrail.io/index.php?/cases/view/350678
Go to Settings/Data import/MARC field protections, and make sure these protections are in place, or create them if not
Field = *, Ind 1 = *, Ind 2 = *, Subfield = 5, Data = NcD
Field = 979, Ind 1 = *, Ind 2 = *, Subfield = *, Data = *
Have a MARC Bib file that you can import with 1 record that does NOT have a FOLIO 001 HRID or 999 UUID field. Otherwise, export a MARC Bib from FOLIO via Data Export and edit it to remove the 999 ff field, and change the 001/003 fields back to the original system ID and source (e.g. 035 $a(OCoLC)12345 would be removed
Edit the MARC Bib in the file to have the following MARC fields:
Field = 977, Ind 1 = [blank], Ind 2 = [blank], Subfield: a, Data: Test NON-protection based on Field 977
Field = 978, Ind 1 = [blank], Ind 2 = [blank], Subfield: a, Data: Test protection based on subfield5, Subfield: 5, Data: NcD
Field = 979, Ind 1 = [blank], Ind 2 = [blank], Subfield: a, Data: Test protection based on field 979
Create a field mapping profile:
Name: Remove extraneous MARC fields
Incoming record type: MARC Bib
FOLIO record type: MARC Bib
Field mappings for MARC: Modifications
Details:
Action: Delete, Field: 977, all other on the row: *
Action: Delete, Field: 978, all other on the row: *
Action: Delete, Field: 979, all other on the row: *
Create an action profile:
Name: Remove extraneous MARC fields
Action: Modify
FOLIO record type: MARC Bibliographic
Link the field mapping profile created above
Create job profile
Name: Create bib and instance, but remove some MARC fields first
Accepted data type: MARC
Overview:
No match profile
Attach the Modify MARC action created above
Attach the Default create instance action profile
Go to Data Import
Upload the MARC Bib record (which has 977, 978, and 979 fields in it)
Assign the just created job profile, and run the job
Once it completes, make sure the log shows "Completed"
Click on the file name to view the log details
Click on the the Title and copy the Instance HRID from the 001 field
Go to the Instance app, change to the Instance HRID search, and paste the HRID that was just copied
In the Instance detail view, go to Actions/View source
Scenario 1 Expected results
Log summary
Summary should show 1 SRS MARC and 1 Instance Created, 0 Updated
SRS MARC should say Created
Instance should say Created
View source
977, 978, and 979 fields are NOT there. None of the incoming fields should have been protected, since field protections should only apply to existing SRS records in FOLIO, and this incoming record did not previously exist in FOLIO
Scenario 1 Actual results
Log summary
Summary shows 1 SRS MARC and 1 Instance Created, 1 SRS MARC and 1 Instance Updated ("1 updated" should be corrected in the scope of a separate bug MODSOURMAN-819)
SRS MARC shows Updated (should be corrected in the scope of a separate bug MODSOURMAN-819)
Instance shows Multiple (will be corrected in the scope of a separate bug MODSOURMAN-819)
Inventory view source
977 field is not there, but 978, and 979 ARE there.
Scenario 2
See attached Scenario2_MODDICORE-248.mp4, Scenario2_MODDICORE-248.mrc, and TestRail ????
Same as Scenario 1, EXCEPT the job profile is in opposite sequence
Create job profile
Name: Create bib and instance, but remove some MARC fields last
Accepted data type: MARC
Overview:
No match profile
Attach the Default create instance action profile
Attach the Modify MARC action created above
Scenario 2 Expected results
Log summary
Summary should show 1 SRS MARC and 1 Instance Created, 0 Updated
SRS MARC should say Created
Instance should say Created
View source
977, 978, and 979 fields are NOT there. None of the incoming fields should have been protected, since field protections should only apply to existing SRS records in FOLIO, and this incoming record did not previously exist in FOLIO
Scenario 2 Actual results
Log summary
Summary shows 1 SRS MARC and 1 Instance Created, 1 SRS MARC and 1 Instance Updated ("1 updated" should be corrected in the scope of a separate bug MODSOURMAN-819)
SRS MARC shows Updated (should be corrected in the scope of a separate bug MODSOURMAN-819)
Instance shows Multiple (will be corrected in the scope of a separate bug MODSOURMAN-819)
Inventory view source
977 field is not there, but 978, and 979 ARE there.