/
Archived Data Import Topics

Archived Data Import Topics

Introduction

This page is meant to track archived topics that are now closed. When a topic from The Data Import Tracker closes, then remove it from that page of current issues to this spreadsheet below.

How to contribute to other people's discussion topics:

  1. Do not add detail to closed or discussed topics as your comments may be overlooked. In this situation, it might be best to Add your details as a new topic and reference the previous topic.

  2. To contribute to an existing topic. Add a new paragraph to the description column.

  3. @mention yourself at the beginning of the paragraph

How to indicate you are also interested in a topic:

  1. @mention yourself in the "Interested parties" column and add your institution name

How are topics archived:

When a topic status is set to closed by it's "Owner". The topic must also be moved to the Data Import Topic Tracker Archive.

  1. Copy the topic and paste it at the top of the Archived topics page that is nested under this page

  2. Delete the topic from this page

Closed Topics

Status

Topic

Description/use case

Date Added

Provided By (Name/Institution)

Interested Parties

Has Been Discussed (Link to agenda/minutes)

Jira Link

Action Required

Status

Topic

Description/use case

Date Added

Provided By (Name/Institution)

Interested Parties

Has Been Discussed (Link to agenda/minutes)

Jira Link

Action Required

1

CLOSED

MARC-MARC Matching Enhancements

MARC-MARC matches and MARC-Inventory matches have differing use cases. Pairing a MARC-MARC match with a more specific MARC-Instance or MARC–Holdings or MARC–Item match allows for identifying a specific record to be updated, or confirms that a new record is needed.

 

We want to ensure that MARC-MARC matching works properly for repeatable and non-repeatable fields, especially 0XX/9XX fields, and that they can pair well with Inventory submatches.

In scope:

  • MARC-MARC matches that result in multiple possible hits can be narrowed to single records with MARC-Inventory or static value submatches

  • Review any existing matching bugs and plan to resolve as part of this feature

Out of scope:

  • After a MARC-MARC or MARC-Instance match, a user can include both Instance and MARC Bib actions afterwards (need examples from users)

  • Confirm that MARC matches are working properly using indicator wildcards (asterisks) versus blanks

  • Currently ISBN matches do not translate 10-digit and 13-digit so that they can be matched against each other, Include in this feature, or handle as a separate feature in the future? See MODSOURMAN-269

  • Should we add a bug for not being able to have an override action for field protections under a MARC-Instance match?

  • What else?

Use case(s):

  • SMEs: Please add examples*
     * MARC-MARC match on OCLC number and then submatch by Instance status

  • MARC-MARC match on 001 and then submatch for holdings by permanent location

  • Need a use case that results in multiple SRS hits that then need to be narrowed down by Inventory match

 

2020-05-13

All

All

2024-1-17 Data Import Subgroup meeting

https://folio-org.atlassian.net/browse/UXPROD-2742

More Use Cases and scenarios

2

closed

Data Import removes duplicate 856s in SRS

Overview:  When updating an SRS record via Data Import, some MARC fields are duplicated while others are de-duped without notification or guidelines.    
**

Steps to Reproduce:

  1. Log into any environment on Orchid or Nolana.

  2. Identify any record in inventory and export the MARC record.

  3. Add two exact duplicate 856 fields to the record.

    1. Exact duplicates including indicators and subfields.  Copy & paste.

  4. Import using a simple overlay job profile matching on a subfield of the 999 to update the SRS.  Do not use MARC modifications.

Expected Results: The SRS record contains duplicate 856 fields.
Actual Results: The SRS record contains only one of the duplicate 856 fields.
**

Additional Information:  We know that Data Import does not de-dupe the 903 field, for example, during an update but it does the 856 field.  Data Import jobs which create new SRS records includes the duplicate 856 fields.  This raises several questions:

  • Why does the system de-dupe during an update without explicit instructions from the user?  

  • Which fields does the system de-dupe and when?

  • Is this a bug?  Or by design?  

From testing, there appears to be no difference between de-duping of the 856 when field protections are applied or not.

 

 

2023-06-13

@Corrie Hutchinson (Unlicensed) 

All

2024-1-17 Data Import Subgroup meeting

2024-1-10 Data Import Subgroup meeting

 

https://folio-org.atlassian.net/browse/MODDATAIMP-879

Need to define what deduplication means

Clarify expectations
Deduping in the UI vs SRS
Deduping the incoming record
3

CLOSED

Adding MARC modifications to imports with update actions creates broken records

Overview: 

Steps to Reproduce:

  1. Log into Morning Glory bugfest

  2. Go to inventory and select import using the "OCLC with MARC modifications" option and import OCLC number 31934425

  3. The record should import successfully and the title will have received an obvious MARC modification

  4. Attempt to overlay the same record with the same OCLC number.

Expected Results:

Overlaying the record works. The MARC is modified as described in the profile and the instance and SRS are updated.

Actual Results:

The instance is not updated. A modified SRS record is created but still has the original OCLC 001 and 003. QuickMARC will not work on the record.

Additional Information:
URL:  Update profile: https://bugfest-mg.int.aws.folio.org/settings/data-import/job-profiles/view/1c351fa4-d578-434f-a02a-7ff46af16f06?query=single&sort=name

An example of an instance with this issue: https://bugfest-mg.int.aws.folio.org/inventory/view/53e28701-dccc-49be-a01d-9adaa15f4cb6?query=neuromancer&sort=title&xidtype=0dd718cf-a09a-4f1c-be6a-0cf0de58b424

Job profiles

BE Notes:

  • First Modify action saves the record in SRS, subsequent Match on MARC Bib in SRS returns multiple match error, because it finds not only the original record, but also the one that was just saved.

  • During modify action post-processing an attempt appears to change instance hrid because of incoming record doesn't contain actual instance hrid in 001
    that causes error on instance update.

Note for QAs:
When this bug is fixed, create a new TestRail for modification, followed by match, followed by update action, as outlined in the repro steps above. Then unlink this Jira from TestRail C350914

 

2022-08-04

@Jenn Colt 

All

 

https://folio-org.atlassian.net/browse/MODDATAIMP-710

 

4

CLOSED

Adding MARC modifications to single record overlay doesn't respect field protections

Overview: When a MARC modification action is added to the end of the single record overlay job, protected fields from the existing MARC SRS are removed rather than from the incoming file.

Steps to Reproduce:

  1. Log into some FOLIO snapshot environment 

  2. Go to Settings/Data Import

  3. Duplicate the field mappings for Inventory Single Record - Default for the update marc bib and update instance

  4. Create a new field mapping for a marc modification where the fields 029, 506, 856, 583 are deleted

  5. Create action profiles linked to the field mappings you just created

  6. Duplicate the inventory single record match for no srs and existing srs

  7. In field protections, make sure that you add 583 with subfield 5 and data MU in addition to 856 with subfield 5 and data MU.

  8. Create a job with a match to existing marc srs. On matches update marc srs. On no matches, another match to no srs and then under that update instance. Then at the level of the 1st match, add the modify marc action (see screenshot)

  9. Bring in a record and add a 583 with a $5 MU and a 856 $5 MU

  10. Overlay

Expected Results: The existing and protected fields should remain in the record.

Actual Results: The protected fields have been removed.

Additional Information:
URL: The job profile on orchid bugfest is https://bugfest-orchid.int.aws.folio.org/settings/data-import/job-profiles/view/b590fc78-f069-42a1-bfdd-3988c7d6be00?query=fc%20test&sort=name. 

 

2023-08-23

@Jennifer Eustis 

All

2024-2-21 Data Import Subgroup meeting

https://folio-org.atlassian.net/browse/MODDATAIMP-897

Functionality audit being prepared in DI lab

5

CLOSED

Single record overlay creates duplicate OCLC # / 035

Overview:  When "Overlay source bibliographic record' is employed for the first time, duplicate 035 fields are created.

Steps to Reproduce:

  1. Log into bugfest-Orchid 

  2. Open Inventory and search for an instance (random records were chosen and the behavior was duplicated each time) 

  3. From Actions, choose 'Overlay source bibliographic record':

    1. External target = OCLC WorldCat

    2. Profile = Inventory single record - default update instance (Default)

    3. Enter the OCLC# of the instance currently being viewed

  4. Click 'Import'

Expected Results: The bibliographic record and instance are updated with the latest OCLC version.
Actual Results: The bibliographic record and instance are updated but with duplicate 035 fields for those OCLC#s with a prefix.  
Additional Information: Testing shows that this happens on the initial overlay, but subsequent overlays do not continue to 'add' duplicates.  

Records tested in bugfest-orchid: in523951 and in2486915 (screenshots of before overlay and after are attached) 

Duplicate data causes issues with integrations and other functions that rely on the OCLC# as a match point.  

 

2023-04-12

@Corrie Hutchinson (Unlicensed) 

All

 

https://folio-org.atlassian.net/browse/MODDICORE-339

 

6

CLOSED

Not able to use the system generated match profiles

I am attempting to create a new job profile for single record import to get rid of the junk fields. I was able to do this a while back on our test tenant and today I was finally able to get around to creating it on our production tenant. 

 

2023-09-14

 

 

 

https://folio-org.atlassian.net/browse/UIDATIMP-1521

 

7

CLOSED

Data Import log displays ambiguous information for successful matches on anything but 999i/instance uuid

f your match profile matches incoming MARC 020 on instance ISBN, and the incoming MARC record matches an instance, the reported status in the log of the SRS record will be Updated if the MARC record also contains a 999-ff-i matching the instance's UUID, but Created if the incoming MARC record does not contain a 999-ff-i. In both cases the status reported for the instance in the log is Updated.

 

2022-05-30

 

 

 

https://folio-org.atlassian.net/browse/MODSOURMAN-848

 

8

closed

data import sorts protected fields out of order after update

Field protections moves the protected field to the first Nxx field. For instance, if the protected field is a 541, the protected 541 becomes the first of all 5xx tags. (see screen shot attached.)

 

2023-05-08

 

 

 

https://folio-org.atlassian.net/browse/MODDICORE-358

 

9

CLOSED

Job summary: error column does not display errors

When there is any error related to an instance/authority/orders/invoice, the error column does not display it.

2024-01-25

 

 

 

https://folio-org.atlassian.net/browse/UIDATIMP-1590

 

10

CLOSED

Field mapping profiles: state of the final form fields is not set

When switching between Folio record types fields with the same name do not reset the state (value, dirty, etc.), although the field values are equal to the initial.

Current workaround: start over/refresh page

2021-06-30

 

 

 

https://folio-org.atlassian.net/browse/UIDATIMP-947

 

11

CLOSED

Incorrect quantity is displayed in the cell of no action and error rows at the individual import job's log

The '1' number of Instance is displayed in cell in the row with the 'No action' and 'Error' rows header in the 'Summary table' at the individual import job's log.

2024-01-19

 

 

 

https://folio-org.atlassian.net/browse/MODSOURMAN-1117

 

12

CLOSED

The status of srs marc is created after match+modify action

Expected Results: The status of SRS MARC is 'Updated' in the Import log after uploading MARC file for update.

2023-03-07

 

 

 

https://folio-org.atlassian.net/browse/MODDATAIMP-1008

 

13

CLOSED

PMSystem displayed as source in quickmarc view when record was created by non matches action of job profile

"PMSystem" displayed as source (instead of User's last and first name) in "Edit MARC authority record" view when record was created by "Non-matches" action of job profile.

2023-03-07

 

 

 

https://folio-org.atlassian.net/browse/MODSOURCE-608

 

14

CLOSED

Single record overlay creates duplicate oclc #/035

When "Overlay source bibliographic record' is employed for the first time, duplicate 035 fields are created.   **  

2023-04-12

 

 

 

https://folio-org.atlassian.net/browse/MODDICORE-339

 

15

CLOSED

Invoice level adjustments do not work

When loading an EDIFACT invoice using a field mapping profile with invoice-level adjustments, the adjustments error

2021-03-29

 

@Kimberly Pamplin 

 

https://folio-org.atlassian.net/browse/MODDICORE-125