|
Overview:
The harvester runs its first job:
2021-08-19 08:49:09,224 DEBUG [BulkRecordHarvestJob(10001 GBV)] BulkRecordHarvestJob downloading list: http://ouf-minerva2/picaxml/
...
2021-08-19 08:49:11,943 INFO [BulkRecordHarvestJob(10001 GBV)] Committed 1 adds, 0 deletes. 0 in total (pending warming of index).
2021-08-19 08:49:11,943 WARN [BulkRecordHarvestJob(10001 GBV)] No default or custom recipients specified, notification will not be sent
Sets its nextbegin timestamp to 08:49:12
2021-08-19 10:32:30,675 INFO [BulkRecordHarvestJob(10001 GBV)] Conditional request If-Modified-Since: Thu, 19 Aug 2021 08:49:12 GMT
Which is not correct because in the time between 08:49:09 (retrival of xml file list) and 08:49:12 (end of harvester job + 1 sec.) there are new xml files created which are never processed.
Begintimestamp of the next harvesterjob needs to be starttimestamp + 1 second.
Expected Results:
Having the timestamp with miliseconds should avoid corner cases.
Actual Results:
In the time between 08:49:09 (retrival of xml file list) and 08:49:12 (end of harvester job + 1 sec.) there are new xml files created which are never processed.
Additional Information:
URL:
Interested parties: Niels Erik Nielsen Dennis Benndorf
|