[FOLIO-3863] Spike: Analyze impact of data file chunking approach when user cancels a job Created: 25/Jul/23  Updated: 04/Dec/23  Resolved: 04/Dec/23

Status: Closed
Project: FOLIO
Components: None
Affects versions: None
Fix versions: None

Type: Task Priority: P3
Reporter: Khalilah Gambrell Assignee: Taras Spashchenko
Resolution: Done Votes: 0
Labels: SolutionArchitecture
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original estimate: Not Specified

Sprint:
Development Team: Data Import Task Force
Release: Poppy (R2 2023)
RCA Group: TBD

 Description   

Context: Currently cancelling a job does not revert the job and sometimes that leads to issues such as MODSOURMAN-899 Closed , MODSOURCE-631 Open

Ideally, cancelling the job should stop the entire job and complete processing of records so that no records are broken in the system. Flow should support allowing records to be re-imported without any issue. 

Spike

  • Investigate the impact of DI task force development on cancelling a job especially when 
    • Loading file 
    • Queue management 
    • Single cluster environment vs. multi-cluster environment 
  • Ensure that development does not lead to broken records/problems with loading records/problems with re-loading records. 
  • Document findings and present to DI task force and Folijet dev lead 

 



 Comments   
Comment by Khalilah Gambrell [ 12/Sep/23 ]

Hey Tetiana Gusar Lee Braginsky Taras Spashchenko any updates on this Spike?

 

cc: Kathleen Moore 

Comment by Tetiana Gusar [ 14/Sep/23 ]

Hi Khalilah Gambrell, sorry for the delay with the reply. Taras Spashchenko hasn't worked on this spike yet, but according to his comments, this doesn't affect anything, since we didn't touch the mechanism responsible for processing at all, there should not be changes in the system's behavior here.

Taras, please add more details on that.

Comment by Taras Spashchenko [ 15/Sep/23 ]

Hello Khalilah Gambrell, sorry for the delay.
The "Data file chunking approach" has no effect on the cancellation of Data Import ExecutionJobs, and canceling such a composite job is no different from canceling a larger unchunked job. Those records that have already been processed will be processed in the same way, and those that have not yet been sent for processing will be canceled because when you cancel a composite Data Import ExecutionJobs, it cancels all its subordinate Data Import ExecutionJobs as well. Since we haven't made any changes to the processing mechanism responsible for handling this, there are no changes in the system's behavior in this regard.

Generated at Thu Feb 08 23:31:21 UTC 2024 using Jira 1001.0.0-SNAPSHOT#100246-sha1:7a5c50119eb0633d306e14180817ddef5e80c75d.