File stuck in Data Import app when importing large amount of records

Description

Overview:
File stuck in Data Import app when importing large amount of records (f.e., 10k MARC Bib records)

Steps to reproduce:

  1. Login to evrk2

  2. Go to “Data import” app

  3. Click on “or choose file“ button

  4. Select a file with thousands of MARC Bib records (you can use the attached .mrc file with 10k records)

  5. Select “Default - Create instance and SRS MARC Bib“ job profile

  6. Click on “Actions” in the third pane → Select “Run“ option

  7. Click on “Run” button in appeared modal

Expected result:

  • After some time, several log rows for chunks (parts) of the total amount of the uploaded records appear in the second pane.

  • All rows have “File name” value equal to name of the uploaded file with digit postfixes (“-1“, “-2”, etc.)

  • All rows have “Completed“ status.

  • The sum of “Records“ values for all rows equals to the total amount of uploaded records.

Actual result:

  • Some of the rows have “Completed with error“ status.

  • When such logs opened, if you click on the record titles in the log, you can see on of the the following error messages on “Incoming record“, “SRS MARC“, “Instance“ tabs:

    • io.vertx.core.impl.NoStackTraceThrowable: {"errors":[{"type":"UnauthorizedException","code":"authorization_error","message":"Unauthorized"}],"total_records":1}

    • No action

  • The import job remains shown in the first pane in “Running“ accordion until it is manually stopped. The progress is stuck on a values < 100%, and some job parts are not finished.

  • Sometimes, further import jobs (with lesser amount of records) are then also stuck in progress. This is NOT consistently reproducible

Additional information:

 

Environment

None

CSP Request Details

None

CSP Rejection Details

None

Potential Workaround

None

Estimation Notes and Assumptions

None

RCA Group Details

None

Attachments

4

Checklist

hide

Activity

Show:

Harry Rumble October 25, 2024 at 1:03 PM

Just want to say thank you for the support on this!

Yauhen Vavilkin October 21, 2024 at 10:52 AM

The bug was cloned into mod-inventory project for Folijet team

Yauhen Vavilkin October 18, 2024 at 3:06 PM

While investigating the issue, I found that the mod-inventory module caches a token, and if a job lasts longer than the lifetime of this token, operations fail with 401 code (A link to the code).
The issue can not be fixed from Eureka side.
I would also recommend reviewing similar areas in the logic to prevent such errors in the future.

Also, I can confirm that increasing the token's lifetime in our environment resolved the issue and it no longer occurs on edev-eureka (2 hours access token ttl).
However, I don't believe that extending the token's lifetime is a valid fix for this issue; it should be resolved in the code.


Harry Rumble October 17, 2024 at 7:07 PM

/ I have a meeting with LOC to review reported bugs/issues, any update on this I can provide to them is appreciated, thank you!

Harry Rumble October 15, 2024 at 2:25 PM

just wanted to check in on the status of this bug fix? Thank you!

Done

Details

Assignee

Reporter

Development Team

Eureka

RCA Group

Implementation coding issue

Story Points

Sprint

Priority

TestRail: Cases

Open TestRail: Cases

TestRail: Runs

Open TestRail: Runs

Created July 4, 2024 at 6:57 AM
Updated December 12, 2024 at 4:22 PM
Resolved October 25, 2024 at 12:38 PM
TestRail: Cases
TestRail: Runs