[UIHA-1] Processing of xml files by the harvester Created: 25/Aug/21  Updated: 29/Nov/22  Resolved: 29/Nov/22

Status: Closed
Project: ui-harvester
Components: None
Affects versions: None
Fix versions: None

Type: Bug Priority: P3
Reporter: Charlotte Whitt Assignee: Unassigned
Resolution: Cannot Reproduce Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original estimate: Not Specified

Sprint: Sif Sprint 153, Sif Sprint 147, Sif Sprint 154, Sif Sprint 148, Sif Sprint 149, Sif Sprint 150, Sif Sprint 151, Sif Sprint 152
Development Team: Sif

 Description   

Overview:
The harvester runs its first job:
2021-08-19 08:49:09,224 DEBUG [BulkRecordHarvestJob(10001 GBV)] BulkRecordHarvestJob downloading list: http://ouf-minerva2/picaxml/
...
2021-08-19 08:49:11,943 INFO [BulkRecordHarvestJob(10001 GBV)] Committed 1 adds, 0 deletes. 0 in total (pending warming of index).
2021-08-19 08:49:11,943 WARN [BulkRecordHarvestJob(10001 GBV)] No default or custom recipients specified, notification will not be sent
Sets its nextbegin timestamp to 08:49:12
2021-08-19 10:32:30,675 INFO [BulkRecordHarvestJob(10001 GBV)] Conditional request If-Modified-Since: Thu, 19 Aug 2021 08:49:12 GMT
Which is not correct because in the time between 08:49:09 (retrival of xml file list) and 08:49:12 (end of harvester job + 1 sec.) there are new xml files created which are never processed.
Begintimestamp of the next harvesterjob needs to be starttimestamp + 1 second.
Steps to Reproduce:

  1. Log into some FOLIO environment as User X
  2. Click this

Expected Results:
Having the timestamp with miliseconds should avoid corner cases.

Actual Results:
In the time between 08:49:09 (retrival of xml file list) and 08:49:12 (end of harvester job + 1 sec.) there are new xml files created which are never processed.

Additional Information:
URL:
Interested parties: Niels Erik Nielsen Dennis Benndorf



 Comments   
Comment by Ann-Marie Breaux (Inactive) [ 30/Nov/21 ]

Hi Charlotte Whitt Not sure if this should be Prok or some other team. The UIHA project is not in the wiki matrix: https://folio-org.atlassian.net/wiki/display/REL/Team+vs+module+responsibility+matrix. Could you review and decide? Thank you!

Comment by Holly Mistlebauer [ 08/Dec/21 ]

This ticket does not have a Priority set. In your opinion, what should the Priority be? Thanks for your help with this...

Comment by Charlotte Whitt [ 17/Feb/22 ]

Ann-Marie Breaux Holly Mistlebauer - this is not Prokopvych as developement team.

It's all correct as noted in the Development matrix - where it was Thor work and now it is Sif work.

Comment by Ann-Marie Breaux (Inactive) [ 17/Feb/22 ]

Cool - thank you for sorting it out, Charlotte Whitt

Comment by Charlotte Whitt [ 21/Feb/22 ]

Ann-Marie Breaux - This is work done by Sif. No need to monitor this by the FOLJIET team

Comment by Charlotte Whitt [ 22/Aug/22 ]

Felix Hemme charlesNiels Erik Nielsen - can this ticket be closed now?

Comment by Felix Hemme [ 22/Aug/22 ]

Hi Denis Mönch , I remember this was an issue a while ago. Has it been fixed? If so, please indicate in a comment and then Charlotte can close the ticket as "Done". Thanks

 

Comment by Charlotte Whitt [ 28/Aug/22 ]

Denis Mönch - a gentle reminder see above.

Comment by Charlotte Whitt [ 28/Nov/22 ]

GBV meeting note: We will keep it open until Denis Mönch is back and can confirm that this ticket can be closed.

Comment by Denis Mönch [ 29/Nov/22 ]

Hi all ...

I will do some checks and get back to you. As far as I can tell at the moment, this problem did not occur for a while. But I have to be absolutely sure about this before closing the case.

Comment by Denis Mönch [ 29/Nov/22 ]

Okay. The problem has gone. You can close this ticket.

Comment by Charlotte Whitt [ 29/Nov/22 ]

Denis Mönch - sounds good. Thank you so much for picking this up right away

Generated at Thu Feb 08 22:33:32 UTC 2024 using Jira 1001.0.0-SNAPSHOT#100246-sha1:7a5c50119eb0633d306e14180817ddef5e80c75d.