Skip to:

  1. Skip to Jira Navigation
  2. Skip to Side Navigation
  3. Skip to Main Content

Timeout executing GET /metadata-provider/jobLogEntries/{jobExecutionId} when split is disabled on the environment

Description

Overview:
The issue reproduced on https://folio-testing-sprint-cs00000int.ci.folio.org/ environment when bulk edit of ~100K MARC Instances was done in Central tenant (split mode was disabled for Data Import on the environment). Bulk edit of MARC Instances is using Data Import under the hood. After Data Import job completed the request was sent to get log entries, and timeout error occurred

ERROR DirectJDKLog         Servlet.service() for servlet [dispatcherServlet] in context with path [] threw exception [Request processing failed: org.folio.bulkops.exception.DataImportException: feign.RetryableException: timeout executing GET http://metadata-provider/jobLogEntries/b175ae3d-9024-48ff-9a08-fa187d27902f?limit=2147483647] with root cause 2024-11-28T18:01:26.090219287Z java.net.SocketException: Socket closed

Steps to Reproduce:

  1. Go to “Bulk edit“ app

  2. Select “Inventory - instances“ record type

  3. Select “Instance UUIDs“ record identifier

  4. Click “or choose file“ button => Upload file with ~100K MARC Instances UUIDs

  5. Click “Actions“ > “Instances with source MARC (POC)“

  6. Select any options/actions to edit MARC fields of Instance records (e.g. add 500 field)

  7. Click “Confirm changes“

  8. Click “Commit changes“

Expected Results:

Commit completed successfully, the Preview of record changed is displayed

In scope of this ticket this is expected that when split mode is disabled GET /metadata-provider/jobLogEntries/{jobExecutionId} supports limit, offset parameters to get log entries by at least 1000 records

Actual Results:

Timeout error occurred even for limit=100, job fails, the Preview of record changed is not displayed

 

IMPORTANT NOTE: ignore that in request we used limit MAX_INT, after we tried limit=100 and we couldn’t get any results for the job with 100k updated records

CSP Request Details

None

CSP Rejection Details

None

Potential Workaround

None

Attachments

16

Checklist

hide

Activity

Show:

Mikita Siadykh March 4, 2025 at 2:38 PM

moving to Folijet as performance improvements are required to retrieve log entries (described in previous comment)

Tatsiana Hryhoryeva February 27, 2025 at 6:36 AM

Hi ,

After improvements on errors saving done in verification on large dataset for bulk edit of ~40K MARC Instances done on https://folio-etesting-sprint-fs09000000.ci.folio.org/ environment with split disabled in DI

  • While committing - Data Import job progress was not displayed

  • After Data Import job completed Bulk edit job completed and Confirmation screen is displayed

  • While Data import job was processing the response time with which log entries returned via Postman using limit, offset parameters was increasing from ~1s to ~44s

  • Upon Data import job completion the response time was 40+ sec or timeout error returned in response to the request for log entries using limit, offset parameters

  • File with errors saved in Bulk edit includes errors about timeouts, e.g. ERROR,,Failed to get 1000 log entries, reason: timeout executing GET http://metadata-provider/jobLogEntries/3c2f7348-036d-4cc3-ba83-a7f6a5d1308f?offset=13000&limit=1000

cc:

Viachaslau Khandramai February 5, 2025 at 10:51 AM

, it is necessary to finish to avoid potential issues in retrieving errors from data-import.

Javokhir Abdullaev February 5, 2025 at 9:52 AM

discussed with
it would be better to get errors by chunks similar to other types of records being retrieved by bulk-ops
without setting max_limit value for errors
cc

Viachaslau Khandramai February 4, 2025 at 12:10 PM

, issues with GET /metadata-provider/jobLogEntries/{jobExecutionId} can potentially affect bulk operations in retrieving errors from data-import because we make call with limit=Integer.MAX.

Details

Assignee

Reporter

Labels

Priority

Story Points

Sprint

Development Team

Folijet

Parent

Release

Sunflower (R1 2025) Bug Fix

TestRail: Cases

Open TestRail: Cases

TestRail: Runs

Open TestRail: Runs

Created November 29, 2024 at 12:04 PM
Updated last week
TestRail: Cases
TestRail: Runs