[GMU-28] Error log provides duplicate entries for the same error Created: 16/Nov/21  Updated: 24/Mar/22  Resolved: 21/Feb/22

Status: Closed
Project: generate-marc-utils
Components: None
Affects versions: None
Fix versions: 1.4.0

Type: Bug Priority: P3
Reporter: Magda Zacharska Assignee: Oleksandr Bozhko
Resolution: Done Votes: 0
Labels: bugfest_R3.2021
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original estimate: Not Specified

Attachments: PNG File image-2021-11-16-18-48-51-832.png     PNG File image-2021-11-16-18-53-04-747.png     PNG File kiwi-export.PNG     PNG File kiwi-multiple-errors.PNG     PNG File testing-export.PNG     PNG File testing-only-one-error.PNG    
Issue links:
Defines
defines UXPROD-3340 Firebird - Lotus R1 2022 Enhancements... Closed
Sprint: Firebird Sprint 133
Story Points: 3
Development Team: Firebird
Release: Lotus R1 2022
RCA Group: Incomplete/missing requirements

 Description   

Here are some examples of the duplicated entries:

Kiwi Bugfest - export job 7689

Error log: https://bugfest-kiwi.folio.ebsco.com/data-export/log/2f710fd6-a288-4b70-8d34-aeb09d9616f3

Kiwi Bugfest - export job 7685

Error log: https://bugfest-kiwi.folio.ebsco.com/data-export/log/f2c90758-735a-4709-acab-04d426278cd2



 Comments   
Comment by Oleksandr Bozhko [ 17/Feb/22 ]

According to the investigation (translation exception is duplicated), this story should be moved to GMU project.

Comment by Oleksandr Bozhko [ 20/Feb/22 ]

This story was reproduced on kiwi and verified on testing environment.

1. Login on kiwi as admin and find instances with source FOLIO:

GET https://okapi-bugfest-kiwi.folio.ebsco.com:443/search/instances?limit=10&query=(source=="FOLIO") 
X-Okapi-Tenant: fs09000000
Content-Type: application/json
x-okapi-token: {token}
 
HTTP/1.1 200 OK
{
    "instances": [
           {all instances}
    ]
}

2. Select any instance (for example instance with id b0f9fd25-c40e-4834-a445-4e3f9d052b97 was selected):

 
GET https://okapi-bugfest-kiwi.folio.ebsco.com:443/instance-storage/instances/b0f9fd25-c40e-4834-a445-4e3f9d052b97
X-Okapi-Tenant: fs09000000
Content-Type: application/json
x-okapi-token: {token}
 
HTTP/1.1 200 OK
{
"id" : "b0f9fd25-c40e-4834-a445-4e3f9d052b97",
"_version" : 14,
"hrid" : "ak00005285486",
"source" : "FOLIO",
"title" : "Postgastrectomy syndromes /by Frederic L. Bushkin and Edward R. Woodward.",
"alternativeTitles" : [ ],
"editions" : [ ],
"series" : [ ],
"identifiers" : [ {
"value" : "9780721622088",
"identifierTypeId" : "59378436-9645-4add-b05f-6afc69bdc8d1"
} ],
"contributors" : [ {
"name" : "Bushkin, Frederic L., 1944-",
"contributorNameTypeId" : "35b01201-597f-4938-a0d1-110245ba9d67"
} ],
"subjects" : [ ],
"classifications" : [ ],
"publication" : [ {
"publisher" : "Saunders",
"dateOfPublication" : "1976"
} ],
"publicationFrequency" : [ ],
"publicationRange" : [ ],
"publicationPeriod" : {
"start" : 1976
},
"electronicAccess" : [ ],
"instanceTypeId" : "1c43b4c1-b138-4386-a91d-0d305e4ddf23",
"instanceFormatIds" : [ ],
"instanceFormats" : [ ],
"physicalDescriptions" : [ ],
"languages" : [ ],
"notes" : [ ],
"previouslyHeld" : false,
"discoverySuppress" : false,
"statisticalCodeIds" : [ ],
"statusId" : "7141660c-a8c2-4bb0-bf68-1110059275a2",
"tags" : {
"tagList" : [ ]
},
"metadata" : {
"createdDate" : "2020-08-20T12:44:41.611+00:00",
"createdByUserId" : "9eb67301-6f6e-468f-9b1a-6134dc39a684",
"updatedDate" : "2022-02-16T23:16:30.885+00:00",
"updatedByUserId" : "9eb67301-6f6e-468f-9b1a-6134dc39a684"
},
"holdingsRecords2" : [ ],
"natureOfContentTermIds" : [ ]
}

3. Change identifierTypeId to make it invalid (for example, last digit was changed from 0 to 1) and update instance:

PUT https://okapi-bugfest-kiwi.folio.ebsco.com:443/instance-storage/instances/b0f9fd25-c40e-4834-a445-4e3f9d052b97
X-Okapi-Tenant: fs09000000
Content-Type: application/json
x-okapi-token: {token}
{
"id": "b0f9fd25-c40e-4834-a445-4e3f9d052b97",
"_version": 14,
"hrid": "ak00005285486",
"source": "FOLIO",
"title": "Postgastrectomy syndromes /by Frederic L. Bushkin and Edward R. Woodward.",
"alternativeTitles": [],
"editions": [],
"series": [],
"identifiers": [
{
"value": "9780721622088",
"identifierTypeId": "59378436-9645-4add-b05f-6afc69bdc8d1"
}
],
"contributors": [
{
"name": "Bushkin, Frederic L., 1944-",
"contributorNameTypeId": "35b01201-597f-4938-a0d1-110245ba9d67"
}
],
"subjects": [],
"classifications": [],
"publication": [
{
"publisher": "Saunders",
"dateOfPublication": "1976"
}
],
"publicationFrequency": [],
"publicationRange": [],
"publicationPeriod": {
"start": 1976
},
"electronicAccess": [],
"instanceTypeId": "1c43b4c1-b138-4386-a91d-0d305e4ddf23",
"instanceFormatIds": [],
"instanceFormats": [],
"physicalDescriptions": [],
"languages": [],
"notes": [],
"previouslyHeld": false,
"discoverySuppress": false,
"statisticalCodeIds": [],
"statusId": "7141660c-a8c2-4bb0-bf68-1110059275a2",
"tags": {
"tagList": []
},
"metadata": {
"createdDate": "2020-08-20T12:44:41.611+00:00",
"createdByUserId": "9eb67301-6f6e-468f-9b1a-6134dc39a684",
"updatedDate": "2022-02-16T23:16:30.885+00:00",
"updatedByUserId": "9eb67301-6f6e-468f-9b1a-6134dc39a684"
},
"holdingsRecords2": [],
"natureOfContentTermIds": []
}
 
HTTP/1.1 204 No Content

4. Save this instance id in csv file.

5. Go to Data export, upload csv file created above, select Default job profile and run the export:

6. Click on the job and see multiple:

Now it needs to make sure the issue above was fixed testing environment.

1. Login on testing as admin and find instances with source FOLIO:

GET https://folio-testing-okapi.dev.folio.org:443/search/instances?limit=10&query=(source=="FOLIO") 
X-Okapi-Tenant: diku
Content-Type: application/json
x-okapi-token: {token}
 
HTTP/1.1 200 OK
{
    "instances": [
           {all instances}
    ]
}

2. Select any instance (for example instance with id 7fbd5d84-62d1-44c6-9c45-6cb173998bbd was selected):

GET https://folio-testing-okapi.dev.folio.org:443/instance-storage/instances/7fbd5d84-62d1-44c6-9c45-6cb173998bbd
X-Okapi-Tenant: diku
Content-Type: application/json
x-okapi-token: {token}

HTTP/1.1 200 OK
{
"id" : "7fbd5d84-62d1-44c6-9c45-6cb173998bbd",
"_version" : 1,
"hrid" : "inst000000000006",
"source" : "FOLIO",
"title" : "Bridget Jones's Baby: the diaries",
"alternativeTitles" : [ ],
"editions" : [ "First American Edition" ],
"series" : [ ],
"identifiers" : [ {
"value" : "ocn956625961",
"identifierTypeId" : "5d164f4b-0b15-4e42-ae75-cfcf85318ad9"
} ],
"contributors" : [ {
"name" : "Fielding, Helen",
"contributorNameTypeId" : "2b94c631-fca9-4892-a730-03ee529ffe2a"
} ],
"subjects" : [ "Jones, Bridget", "Pregnant women", "England", "Humorous fiction", "Diary fiction" ],
"classifications" : [ {
"classificationNumber" : "PR6056.I4588",
"classificationTypeId" : "ce176ace-a53e-4b4d-aa89-725ed7b2edac"
} ],
"publication" : [ {
"publisher" : "Alfred A. Knopf",
"place" : "New York",
"dateOfPublication" : "2016"
} ],
"publicationFrequency" : [ "A frequency description" ],
"publicationRange" : [ "A publication range" ],
"publicationPeriod" : {
"start" : 2016
},
"electronicAccess" : [ {
"uri" : "http://www.folio.org/",
"linkText" : "Electronic resource (audio streaming)",
"materialsSpecification" : "Novel",
"publicNote" : "Access to audio file"
} ],
"instanceTypeId" : "6312d172-f0cf-40f6-b27d-9fa8feaf332f",
"instanceFormatIds" : [ ],
"instanceFormats" : [ ],
"physicalDescriptions" : [ "219 pages ; 20 cm." ],
"languages" : [ "eng" ],
"notes" : [ {
"note" : "Bridget Jones finds herself unexpectedly pregnant at the eleventh hour. However, her joyful pregnancy is dominated by one crucial but awkward question --who is the father? Could it be honorable, decent, notable human rights lawyer, Mark Darcy? Or, is it charming, witty, and totally despicable, Daniel Cleaver?",
"staffOnly" : false,
"instanceNoteTypeId" : "6a2533a7-4de2-4e64-8466-074c2fa9308c"
} ],
"administrativeNotes" : [ "Cataloging data" ],
"previouslyHeld" : true,
"staffSuppress" : true,
"discoverySuppress" : true,
"statisticalCodeIds" : [ ],
"statusUpdatedDate" : "2022-02-20T03:22:04.245+0000",
"metadata" : {
"createdDate" : "2022-02-20T03:22:04.244+00:00",
"updatedDate" : "2022-02-20T03:22:04.244+00:00"
},
"holdingsRecords2" : [ ],
"natureOfContentTermIds" : [ ]
}

3. Change identifierTypeId to make it invalid (for example, first digit was changed from 5 to 6) and update instance:

PUT https://folio-testing-okapi.dev.folio.org:443/instance-storage/instances/7fbd5d84-62d1-44c6-9c45-6cb173998bbd
PUT /instance-storage/instances/7fbd5d84-62d1-44c6-9c45-6cb173998bbd HTTP/1.1
X-Okapi-Tenant: diku
Content-Type: application/json
x-okapi-token: {token}
{
"id": "7fbd5d84-62d1-44c6-9c45-6cb173998bbd",
"_version": 1,
"hrid": "inst000000000006",
"source": "FOLIO",
"title": "Bridget Jones's Baby: the diaries",
"alternativeTitles": [],
"editions": [
"First American Edition"
],
"series": [],
"identifiers": [
{
"value": "ocn956625961",
"identifierTypeId": "6d164f4b-0b15-4e42-ae75-cfcf85318ad9"
}
],
"contributors": [
{
"name": "Fielding, Helen",
"contributorNameTypeId": "2b94c631-fca9-4892-a730-03ee529ffe2a"
}
],
"subjects": [
"Jones, Bridget",
"Pregnant women",
"England",
"Humorous fiction",
"Diary fiction"
],
"classifications": [
{
"classificationNumber": "PR6056.I4588",
"classificationTypeId": "ce176ace-a53e-4b4d-aa89-725ed7b2edac"
}
],
"publication": [
{
"publisher": "Alfred A. Knopf",
"place": "New York",
"dateOfPublication": "2016"
}
],
"publicationFrequency": [
"A frequency description"
],
"publicationRange": [
"A publication range"
],
"publicationPeriod": {
"start": 2016
},
"electronicAccess": [
{
"uri": "http://www.folio.org/",
"linkText": "Electronic resource (audio streaming)",
"materialsSpecification": "Novel",
"publicNote": "Access to audio file"
}
],
"instanceTypeId": "6312d172-f0cf-40f6-b27d-9fa8feaf332f",
"instanceFormatIds": [],
"instanceFormats": [],
"physicalDescriptions": [
"219 pages ; 20 cm."
],
"languages": [
"eng"
],
"notes": [
{
"note": "Bridget Jones finds herself unexpectedly pregnant at the eleventh hour. However, her joyful pregnancy is dominated by one crucial but awkward question --who is the father? Could it be honorable, decent, notable human rights lawyer, Mark Darcy? Or, is it charming, witty, and totally despicable, Daniel Cleaver?",
"staffOnly": false,
"instanceNoteTypeId": "6a2533a7-4de2-4e64-8466-074c2fa9308c"
}
],
"administrativeNotes": [
"Cataloging data"
],
"previouslyHeld": true,
"staffSuppress": true,
"discoverySuppress": true,
"statisticalCodeIds": [],
"statusUpdatedDate": "2022-02-20T03:22:04.245+0000",
"metadata": {
"createdDate": "2022-02-20T03:22:04.244+00:00",
"updatedDate": "2022-02-20T03:22:04.244+00:00"
},
"holdingsRecords2": [],
"natureOfContentTermIds": []
}
 
HTTP/1.1 204 No Content

4. Save this instance id in csv file.

5. Go to Data export, upload csv file created above, select Default instances job profile, Instances and run the export:

6. Click on the job and see only one error:

As you can see above, there are no error duplicates anymore.

Comment by Denis [ 24/Mar/22 ]

Replaced release field with Lotus R1 2022 release, as the ticket was released under Fix Version 1.4.0, that was Lotus release itself.

Data export has the following version on GMU - 1.4.0. It's available in pom.xml file
https://github.com/folio-org/mod-data-export/blob/master/pom.xml 

cc Magda Zacharska Viachaslau Khandramai Oleksandr Bozhko

Generated at Thu Feb 08 22:16:15 UTC 2024 using Jira 1001.0.0-SNAPSHOT#100246-sha1:7a5c50119eb0633d306e14180817ddef5e80c75d.