Skip to:

  1. Skip to Jira Navigation
  2. Skip to Side Navigation
  3. Skip to Main Content

Data Import - job fails with fk constraint error

Description

One of EBSCO's hosted libraries – Skidmore – has reported an issue attempting to import a file with 2922 records. The job completes with errors. From the ui log – each record fails with error SRS MARC Bib record Discarded- we see an error similar to this for each file –

SQL [null]; ERROR: insert or update on table "raw_records_lb" violates foreign key constraint "fk_raw_records_records" Detail: Key (id)=(92a43de3-277b-41f9-b10f-6c3302e917d3) is not present in table "records_lb".

Attaching the job profile (which was used for the job) - the library is
"attempting to use a match profile to update the 856 field for several thousand records"

I have asked for the marc file that they are testing with (or a similar smaller sample), and it's attached. I've attached a part of mod-srs logs for 1 of the records at the time of the error.

Was looking through Jira issues - Is there a known issue with 'fk_raw_records_records' or something possibly in error with their job profile which might lead to this situation/error?

At the time of the error - mod-data-import-converter-storage restarted due to memory pressure

The library is running Juniper HotFix-4:
Source Record Manager Module (mod-source-record-manager-3.1.3)
Source Record Storage Module (mod-source-record-storage-5.1.8)

CSP Request Details

None

CSP Rejection Details

None

Potential Workaround

None

Attachments

6
  • 05 Jan 2022, 09:21 AM
  • 28 Dec 2021, 02:36 PM
  • 27 Dec 2021, 10:47 AM
  • 27 Dec 2021, 10:47 AM
  • 27 Dec 2021, 10:47 AM
  • 27 Dec 2021, 10:47 AM

Confluence content

mentioned on

Checklist

hide

TestRail: Results

Activity

Show:

Ann-Marie Breaux January 25, 2022 at 10:23 AM

Hi and I'm going to close this. Could one of you create a P3 or P4 priority Spike to investigate validation mechanism for profile configuration, and link to this Jira? Thank you!

Aliaksandr Fedasiuk January 24, 2022 at 1:23 PM
Edited

Hi , client got fk constraint because the logic worked by an unexpected way because client was using incorrect profile configuration. For fix it we have to implement some validation mechanism for profile configuration, but this is quite a complicated job with architecture decisions. At the moment, the user is responsible for validating a job profile.

Carole Godfrey January 24, 2022 at 2:40 AM

 and   – the library has reworked DI profiles and are no longer experiencing this issue.

Still would be helpful to understand why fk constraint happened and if an incorrectly configured profile could result in this error

Ann-Marie Breaux January 20, 2022 at 2:20 PM

Hi Fingers crossed that an updated profile will help. I was looking at it with Skidmore and couldn't see a problem with the profile. It it's just a profile/settings problem, then it sounds like no Kiwi HF would be needed. I'll wait til we get this resolved before definitely making a decision about HF.

Note to self: if recommended profile works, consider adding to Tips and Tricks page, and possible release note.

Aliaksandr Fedasiuk January 13, 2022 at 5:21 PM
Edited

Hi , I prepared profile for update 856 field and described it in RRT chanel to . If this decision is enough and I will describe it here and we can close this task. Now we will wait for a response from Skidmore. I think Skidmore used inappropriate profile settings.

Cannot Reproduce

Details

Assignee

Reporter

Labels

Priority

Story Points

Development Team

Folijet Support

Parent

Release

Lotus R1 2022

RCA Group

Institution specific (Data)

Affected Institution

Skidmore College

TestRail: Cases

Open TestRail: Cases

TestRail: Runs

Open TestRail: Runs
Created December 27, 2021 at 10:46 AM
Updated January 25, 2022 at 11:19 AM
Resolved January 25, 2022 at 10:23 AM
TestRail: Cases
TestRail: Runs

Flag notifications