[MODEXPW-47] Save errors in csv format Created: 24/Sep/21 Updated: 16/Dec/21 Resolved: 16/Dec/21 |
|
| Status: | Closed |
| Project: | mod-data-export-worker |
| Components: | None |
| Affects versions: | None |
| Fix versions: | None |
| Type: | Story | Priority: | P3 |
| Reporter: | Magda Zacharska | Assignee: | Oleksandr Bozhko |
| Resolution: | Done | Votes: | 0 |
| Labels: | bulk-edit-pilot | ||
| Remaining Estimate: | Not Specified | ||
| Time Spent: | Not Specified | ||
| Original estimate: | Not Specified | ||
| Attachments: |
|
||||||||||||||||
| Issue links: |
|
||||||||||||||||
| Sprint: | Firebird Sprint 129 | ||||||||||||||||
| Story Points: | 3 | ||||||||||||||||
| Development Team: | Firebird | ||||||||||||||||
| Description |
|
Purpose/Overview: Requirements/Scope:
Approach: Acceptance criteria:
Notes |
| Comments |
| Comment by Illia Daliek [ 17/Nov/21 ] |
|
Magda Zacharska, should we also store the CSV files with the errors in S3 for some period of time until it gets expired, or does the user has to get them only once? |
| Comment by Magda Zacharska [ 19/Nov/21 ] |
|
Illia Daliek please do not make changes to the description of the jira after it was estimated by the team. Any required changes should be discussed and documented in the comments to the ticket. |
| Comment by Magda Zacharska [ 19/Nov/21 ] |
|
For the pilot project we do not need to store the error files in S3 |
| Comment by Oleksandr Bozhko [ 09/Dec/21 ] |
|
Magda Zacharska - this story has been verified at snapshot environment. 1. Create a new job: POST https://folio-snapshot-okapi.dev.folio.org:443/data-export-spring/jobs X-Okapi-Tenant: diku Content-Type: application/json x-okapi-token: $TOKEN { "type": "BULK_EDIT_IDENTIFIERS", "exportTypeSpecificParameters": {}, "entityType": "USER", "identifierType": "BARCODE" } HTTP/1.1 201 Created Content-Type: application/json { "id":"82e29654-921f-4d20b6dd617cf02a980d", "name":"000008", "source":"diku_admin", "isSystemSource":false, "type":"BULK_EDIT_IDENTIFIERS", "exportTypeSpecificParameters":{}, "status":"SCHEDULED", "metadata": { "createdDate":1639059432556, "createdByUserId":"c9eb3fe1-d774-5ef7-934f-be8a2315fb74", "createdByUsername":"diku_admin", "updatedDate":1639059432556, "updatedByUserId":"c9eb3fe1-d774-5ef7-934f-be8a2315fb74", "updatedByUsername":"diku_admin" }, "identifierType":"BARCODE", "entityType":"USER" } 2. Upload attached user_barcodes_with_one_corrent_one_not_exists_one_duplicated.csv POST https://folio-snapshot-okapi.dev.folio.org:443/bulk-edit/82e29654-921f-4d20-b6dd-617cf02a980d/upload X-Okapi-Tenant: diku Content-Type: multipart/form-data; x-okapi-token: $TOKEN Accept: */* Content-Disposition: form-data; name="file"; filename="user_barcodes.csv" HTTP/1.1 200 OK 3. Get job by id: GET https://folio-snapshot-okapi.dev.folio.org:443/data-export-spring/jobs/82e29654-921f-4d20-b6dd-617cf02a980d X-Okapi-Tenant: diku Content-Type: application/json x-okapi-token: $TOKEN Accept: */* HTTP/1.1 200 OK Content-Type: application/json { "id":"82e29654-921f-4d20-b6dd-617cf02a980d", "name":"000008", "source":"diku_admin", "isSystemSource":false, "type":"BULK_EDIT_IDENTIFIERS", "exportTypeSpecificParameters":{}, "status":"SUCCESSFUL", "files":["https://folio-snapshot-export.s3.us-east-1.amazonaws.com/user_barcodes.csv?X-Amz-Algorithm=AWS4-HMAC-SHA256&X-Amz-Credential=AKIA2VGNPE73JV77RBGE%2F20211209%2Fus-east-1%2Fs3%2Faws4_request&X-Amz-Date=20211209T142031Z&X-Amz-Expires=604800&X-Amz-SignedHeaders=host&X-Amz-Signature=bec05255f08ec384fd6fff1862cd04b6bc8a04fcb293f96db8a8562f4d6373d1", "https://folio-snapshot-export.s3.us-east-1.amazonaws.com/2021-12-09-Errors-user_barcodes.csv?X-Amz-Algorithm=AWS4-HMAC-SHA256&X-Amz-Credential=AKIA2VGNPE73JV77RBGE%2F20211209%2Fus-east-1%2Fs3%2Faws4_request&X-Amz-Date=20211209T142031Z&X-Amz-Expires=604800&X-Amz-SignedHeaders=host&X-Amz-Signature=64114cb3394bc51538382263251f5134c127b3fa1202816d6cae6def4a5fe37b"], "startTime":1639059630889, "endTime":1639059630995, "metadata": { "createdDate":1639059432556, "createdByUserId":"c9eb3fe1-d774-5ef7-934f-be8a2315fb74", "createdByUsername":"diku_admin", "updatedDate":1639059631114, "updatedByUserId":"c9eb3fe1-d774-5ef7-934f-be8a2315fb74", "updatedByUsername":"diku_admin" }, "identifierType":"BARCODE", "entityType":"USER" } 4. Use the second link in "files" and open 2021-12-09-Errors-user_barcodes.csv file:
GET https://folio-snapshot-export.s3.us-east-1.amazonaws.com/2021-12-09-Errors-user_barcodes.csv?X-Amz-Algorithm=AWS4-HMAC-SHA256&X-Amz-Credential=AKIA2VGNPE73JV77RBGE%2F20211209%2Fus-east-1%2Fs3%2Faws4_request&X-Amz-Date=20211209T142031Z&X-Amz-Expires=604800&X-Amz-SignedHeaders=host&X-Amz-Signature=64114cb3394bc51538382263251f5134c127b3fa1202816d6cae6def4a5fe37b
HTTP/1.1 200 OK
163901525593756911,No match found
1639015255937569116,Duplicate entry
Result is attached in 2021-12-09-Errors-user_barcodes.csv |