/
Expand MARC Update field mapping profiles

Expand MARC Update field mapping profiles

 

Umbrellaleaf scope: https://folio-org.atlassian.net/browse/UXPROD-4080

Contents

Summary

Create Action options (Add, Delete, Replace) for individual field updates on Update MARC-to-MARC field mapping profiles in Data Import, replacing the current default repeatable/non-repeatable behavior. 

Current Situation or Problem 

Currently, Data Import MARC Update field mapping profiles apply a non-configurable rule based on whether a MARC field is repeatable: 

  • Non-repeatable fields = REPLACE 

  • Repeatable fields = ADD 

This creates several problems: 

  • No way to replace a repeatable field. If a library wants to overlay a repeatable field, such as replacing a broken 856 URL, or correcting an erroneous 035 OCLC number, the current logic adds the incoming value alongside the existing one rather than replacing it. This could cause duplicate, conflicting, or outdated data in the record 

  • No way to insert a repeatable field without duplicating. Conversely, there's no explicit "Add" control separate from Replace for cases where accumulation is the actual intent, such as legitimately adding a second, distinct 856 

  • No way to remove a non-repeatable field. Current logic only supports replacing a non-repeatable field's value, not deleting it entirely, which might be needed when in cases like stripping an erroneous 043 or a 010 that shouldn't have a replacement value at all 

  • Authority linking issues: When a repeatable field has MARC field protection configured on specific subfields (e.g., $0, $9 for authority linking), and the incoming record's version of that field omits those protected subfields, current logic retains the existing field and adds a new field from the incoming data which produces a duplicate occurrence rather than a merge 

Requirements 

Mockups: https://drive.google.com/drive/folders/1CHoJm9uVOZyNv6rCTJq3_I5Rhqem_sBh  

  • Create an Action option for each field row in an Update MARC-to-MARC field mapping profile 

  • Add 

    • Adds the field(s)/subfield(s) from the incoming file to the existing SRS MARC record as specified in the mapping profile. Does not remove existing data.

    • Applies to both repeatable and non-repeatable fields

    • Alert users if a non-repeatable field already exists; the record will contain two instances of that field

      • Ask Kate if there is a pattern for alerting the user about a non-repeatable field elsewhere 

  • Delete 

      • Removes all matching field(s)/subfield(s) from the existing SRS MARC record as specified in the mapping profile 

    • Both repeatable and non-repeatable fields 

  • Replace 

    • Removes the matching existing field(s)/subfield(s) as specified in the mapping profile, then adds the incoming file's version in their place 

    • Both repeatable and non-repeatable fields 

  • Interaction with MARC field protection, defined per Action:  

    • Add: unaffected by field protection, since Add never modifies existing occurrences, potentially UI enhancement so catalogers don't mistake Add for a merge option 

    • Delete: when a Delete is scoped (via * or explicit subfield list) to remove a field/subfields that include protected subfield, the delete must be blocked for that occurrence unless Override is explicitly selected. Blocked or partial deletes must be surfaced in the Data Import job log 

    • Replace: when the field mapping profile scopes to specific subfields (not *), and some of those subfields are protected, Replace must merge to preserve protected subfield values from the existing occurrence while applying incoming values to unprotected subfields, rather than wiping and recreating the whole occurrence. When scoped to * (all subfields) and any subfield is protected, the same merge behavior should apply at the subfield level unless Override is explicitly selected 

    • Migration idea: Migrate all existing MARC Update jobs fields to REPLACE action when upgrading to release containing this feature’s enhancements. 

Scenarios (in progress)

Action

Section

Scenario

Scenario steps

DI SIG feedback

Status

Action

Section

Scenario

Scenario steps

DI SIG feedback

Status

1

Add

Add non-repeatable (support but warn) 

“Add” action selected for a non-repeatable field warns but does not block 

 

Given a user is configuring an Update MARC-MARC field mapping profile AND the field is non-repeatable AND the user selects the Action “Add” for that field row 

When the import runs 

Then the system should warn the user that “Add” on a non-repeatable field may create an invalid duplicate field if one already exists 

 

*or just have a static message?

 

Having a warning without the action being blocked is fine.

UNDER REVIEW

2

Add

Add non-repeatable (support but warn) 

“Add” action on a non-repeatable field inserted into a record where one does not exist 

Given a field mapping profile with the “Add” action on a 250 field AND there is an existing SRS MARC bib record with no occurrence of a 250 field 

When the import runs 

Then the existing SRS MARC bib record should contain exactly one occurrence of field 250 

This makes sense.

UNDER REVIEW

3

Add

Add repeatable field 

Add action on a repeatable field appends a new occurrence without affecting existing occurrences 

Given an existing SRS MARC bib record with one occurrence of repeatable field 856 containing "$u http://example.com/existing" AND an Update field mapping profile with Action “Add” configured for field 856 containing "$u http://example.com/new" 

When the import runs 

Then the existing SRS MARC bib record should contain two occurrences of 856 AND the existing occurrence "$u http://example.com/existing" should remain unchanged AND a new occurrence "$u http://example.com/new" should be present 

This makes sense, especially the part where existing occurrences aren’t changed.

UNDER REVIEW

4

Delete

Delete non-repeatable 

Delete action removes a non-repeatable field entirely 

Given an existing SRS MARC bib with a non-repeatable 010 field containing “n12345678” AND an update field mapping profile with Action “Delete” configured for 010 field and subfield * 

When the import runs 

Then the existing SRS MARC bib record should contain no occurrence of 010 field 

What happens when a field required to be valid MARC is removed such as LDR? Maybe we need a warning here too?

UNDER REVIEW

5

Delete

Delete non-repeatable 

Delete action on a non-repeatable field with no matching field 

Given an existing SRS MARC bib record with no occurrence of 043 field AND an Update field mapping profile with Action “Delete” is configured for 043 field subfield * 

When the import runs 

Then no changes should be made to the SRS MARC bib record 

This makes sense. It would be great to know how the log will interpret this and provide information in the log summary.

UNDER REVIEW

6

Delete

Delete repeatable 

Delete action removes all occurrences of a repeatable field with subfield * 

Given an existing SRS MARC bib record with two occurrences of 856 AND an Update field mapping profile with Action “Delete” configured for 856 subfield * 

When the import runs 

Then the existing SRS MARC bib record should contain no 856 fields 

This is correct or the field all subfields and value should be removed.

UNDER REVIEW

7

Delete

Delete repeatable 

Delete action removes only the specified subfield within repeatable field 

Given an existing SRS MARC bib record with 650 containing "$a Costume design $x African American influences $5 InU" AND an Update field mapping profile with Action “Delete” configured for 650 field subfield x 

When the import runs 

Then the resulting 650 field should contain "$a Costume design $5 InU" AND should not contain x 

If we open up the contains to the subfields a and x, can we assume that if the values are in a different order so let’s say the 650 is $5 Inu then $a Costume design $x African American influences that the fields are still removed?
What happens if that subfield that is being removed is the last subfield present - in which case we would want the MARC field to also be removed?

UNDER REVIEW

8

Replace

Replace non-repeatable

Replace overwrites the value of an existing non-repeatable field 

Given an existing SRS MARC bib record with 245 containing “Outdated title” AND an Update field mapping profile with Action “Replace” configured for 245 subfield * AND the incoming MARC bib record has 245 “New title” 

When the import runs 

Then the existing SRS MARC bib record has 245 containing "New title” AND no duplicate occurrence of 245 should exist 

Yes.

UNDER REVIEW

9

Replace

Replace non-repeatable

Replace action inserts a non-repeatable field when it does not already exist 

Given an existing SRS MARC bib record with no occurrence of 250 AND an Update field mapping profile with Action “Replace” configured for 250 subfield * AND the incoming MARC bib record has field 250 containing $a “2nd ed." 

When the import runs 

Then the existing MARC bib record should contain exactly one occurrence of 250 with $a “2nd ed." 

Yes.

 

UNDER REVIEW

10

Replace

Replace repeatable

Replace action removes all existing occurrences of a repeatable field and inserts the incoming occurrence(s) 

Given an existing SRS MARC bib record with two occurrences of 856 AND an updated field mapping profile with Action “Replace” configured for 856 subfield * AND the incoming MARC bib record has one occurrence of field 856 

When the import runs 

Then the existing record should contain exactly one occurrence of an 856 AND the outdated 856 fields should be removed 

Yes

UNDER REVIEW

11

Replace

Replace repeatable

Replace action scoped to a specific subfield only touches that subfield, leaving other subfields on the same occurrence untouched 

Given an existing SRS MARC Bib record with field "035" containing "$a (OCoLC)wrong123 $9 local-tracking-001" AND an Update field mapping profile with Action “Replace” configured for 035 $a AND the incoming MARC bib record has field 035 $a “(OCoLC)correct456” 

When the import runs 

Then the resulting 035 field should contain both the new $a “(OCoLC)correct456” AND the preexisting $9 localtracking-001" 

Yes.

Though many of us have 035 with a first or second indicator of 9. In this case, how will the system know that you want to replace just the OCLC number and not any other system numbers that go in the 035? Similar question for 856’s where indicators help differentiate the type of value (FTP, URL, etc.)? Would we have to make sure that the match profile has a use of a qualifier and/or only compare part of the value?

UNDER REVIEW

12

Add

Authority linking & field protection: Add (WIP) 

Add action is unaffected by field protection and always creates a new, unlinked occurrence 

 

 

ANALYSIS IN PROGRESS

13

Delete

Authority linking & field protection: Delete (WIP)  

 

 

 

ANALYSIS IN PROGRESS

14

Replace

Authority linking & field protection: Replace (WIP) 

 

 

 

ANALYSIS IN PROGRESS