Job status in UI changes FAILED -> COMPLETED after error in Data Export job (Kiwi)

Description

Purpose/Overview:

Data Export job status changes from FAILED -> COMPLETED in Data Export UI even after there were errors during the run

Approach:

Environment:

https://pre-kiwi-cap1.int.aws.folio.org/

mod-data-export v4.2.1

Use case:

Run Data Export job to Export 500K instance records.

Observation:

After exporting 236400 records, Data Export failed in UI and Chrome console log shows as

{
    "id" : "e74bc816-57cc-4c19-a075-ceda913a5adb",
    "hrId" : 7635,
    "exportedFiles" : [ {
      "fileId" : "dcbce2b8-7e84-4ca4-866f-07a2760a2d98",
      "fileName" : "kcp1-DE-500k-7635.mrc"
    } ],
    "jobProfileId" : "937d6256-8532-442b-9286-cbc3396fa18d",
    "jobProfileName" : "holdings and items",
    "progress" : {
      "exported" : 236400,
      "failed" : 0,
      "total" : 500000
    },
    "completedDate" : "2021-10-29T19:11:10.626+00:00",
    "lastUpdatedDate" : "2021-10-29T19:42:30.320+00:00",
    "startedDate" : "2021-10-29T16:39:46.971+00:00",
    "runBy" : {
      "firstName" : "folio",
      "lastName" : "folio"
    },
    "status" : "FAIL"
  }

mod-data-export failed at 10/29/2021 3:11 PM Eastern as per the Chrome console log. But as per CPU and memory utilization, export continued to run.
mod-inventory, mod-inventory-storage, mod-source-record-storage continue to work even if job failed in UI. There are no errors in module logs.

{    "id": "e74bc816-57cc-4c19-a075-ceda913a5adb",    "hrId": 7635,    "exportedFiles": [        {            "fileId": "dcbce2b8-7e84-4ca4-866f-07a2760a2d98",            "fileName": "kcp1-DE-500k-7635.mrc"        }    ],    "jobProfileId": "937d6256-8532-442b-9286-cbc3396fa18d",    "jobProfileName": "holdings and items",    "progress": {        "exported": 500000,        "failed": 0,        "total": 500000    },    "completedDate": "2021-10-29T22:57:24.532+00:00",    "lastUpdatedDate": "2021-10-29T22:57:24.474+00:00",    "startedDate": "2021-10-29T16:39:46.971+00:00",    "runBy": {        "firstName": "folio",        "lastName": "folio"    },    "status": "COMPLETED"}

After failed testing in UI, I did not stop running backend modules. After a few hours, DE jobs status changed from FAILED -> COMPLETED and so does the status in UI.

Additional information:
The reverse order can be also possible when the job status changes from completed to failed. It might be worth investigating if this is not related to the restarting of the module.

Acceptance criteria:

Investigate why status changes from FAILED -> COMPLETED when data export job seems to be still running

Environment

None

Potential Workaround

None

Attachments

03 Nov 2021, 01:56 PM
03 Nov 2021, 01:55 PM

Linked issues

relates to

MDEXP-470

Export fails after /data-export/expire-jobs endpoint invocation

Checklist

hide

TestRail: Results

Activity

Show:

Cannot Reproduce

Details

Assignee

Unassigned

Reporter

Varun Javalkar

Priority

Story Points

Development Team

Firebird

TestRail: Cases

Open TestRail: Cases

TestRail: Runs

Open TestRail: Runs

Created November 3, 2021 at 1:49 PM

Updated February 6, 2025 at 6:48 PM

Resolved February 6, 2025 at 6:48 PM

Configure

TestRail: Cases

TestRail: Runs