[MODEXPW-45] Get matching records based on provided identifiers Created: 03/Sep/21  Updated: 16/Dec/21  Resolved: 16/Dec/21

Status: Closed
Project: mod-data-export-worker
Components: None
Affects versions: None
Fix versions: 1.3.0

Type: Story Priority: P3
Reporter: Magda Zacharska Assignee: Siarhei Charniak
Resolution: Done Votes: 0
Labels: bulk-edit-pilot
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original estimate: Not Specified

Attachments: File userIds-2.csv     File userIds.csv    
Issue links:
Defines
defines UXPROD-3229 Identify user records for bulk edit Closed
Duplicate
is duplicated by MODEXPW-49 Get matching records CSV file by user... Closed
Gantt End to Start
has to be done before MODEXPS-48 Remove expired files Closed
has to be done before MODEXPW-46 Get matching records based on provide... Closed
has to be done before MODEXPW-51 Records processing errors handling Closed
has to be done after MODBULKED-2 [SPIKE]:File upload functionality Closed
Sprint: Firebird Sprint 129
Story Points: 5
Development Team: Firebird

 Description   

Purpose/Overview:
After uploading a CSV file with users' UUIDs the user should be able to see the preview of matched records(top 10) and be able to download a CSV file with matched records, so the records can be reviewed and later modified.

Requirements/Scope:

  1. All records returned are saved in csv
  2. Keys are replaced with values
  3. The file name follows the convention: <yyyy-mm-dd-Matched-Records-<name of the file with identifiers>.csv

Approach:
1) Update the JobCommand with the new field that will define the export type for bulk edit (since in future we have to work with barcodes CSV file instead of UUIDs only and work with different record types like Users, Items and etc we have to know which Job we need to launch, this information must be put to the JobCommand)
2) Implement an endpoint for saving UUIDs CSV files locally in mod-data-export-worker.
/bulk-edit/jobId/upload, where jobId is the id of created JobCommand.
3) Implement a base Job for exporting records to CSV files.
4) Reference data caching.

Acceptance criteria:

  • File is created and can be saved
  • File contains all fields from the user record
  • Keys are replaced with corresponding values
  • The copy of a file is saved.
  • File name follows the convention
  • Unit and API tests are written where applicable


 Comments   
Comment by Magda Zacharska [ 19/Nov/21 ]

Illia Daliek please do not make changes to the description of the jira after it was estimated by the team. Also, please do not make changes to the acceptance criteria without letting PO know. Any required changes should be discussed and documented in the comments to the ticket.

Comment by Siarhei Charniak [ 08/Dec/21 ]

Magda Zacharska
Verified on folio-snapshot environment.

1. Create a new Job

curl --location --request POST 'https://folio-snapshot-load-okapi.dev.folio.org:443/data-export-spring/jobs' \
--header 'x-okapi-tenant: diku' \
--header "x-okapi-token: $TOKEN" \
--header 'Content-Type: application/json' \
--data-raw '{
  "type": "BULK_EDIT_IDENTIFIERS",
  "exportTypeSpecificParameters": {},
  "identifierType": "BARCODE",
  "entityType": "USER"
}'
{
    "id": "7feb90ef-9f84-4da9-ac1a-3149c4695afe",
    "name": "000001",
    "source": "diku_admin",
    "isSystemSource": false,
    "type": "BULK_EDIT_IDENTIFIERS",
    "exportTypeSpecificParameters": {},
    "status": "SCHEDULED",
    "metadata": {
        "createdDate": 1638938131732,
        "createdByUserId": "cd8fdbda-7db5-5734-a7f3-5e4315ef8f6a",
        "createdByUsername": "diku_admin",
        "updatedDate": 1638938131732,
        "updatedByUserId": "cd8fdbda-7db5-5734-a7f3-5e4315ef8f6a",
        "updatedByUsername": "diku_admin"
    },
    "identifierType": "BARCODE",
    "entityType": "USER"
}

2. Upload attached userIds.csv file with user barcodes:

curl --location --request POST 'https://folio-snapshot-load-okapi.dev.folio.org:443/bulk-edit/7feb90ef-9f84-4da9-ac1a-3149c4695afe/upload' \
--header 'x-okapi-tenant: diku' \
--header "x-okapi-token: $TOKEN" \
--header 'Content-Type: multipart/form-data' \
--form 'file=@"userIds.csv"'
200 OK

3. Check job status - SUCCESSFUL

curl --location --request GET 'https://folio-snapshot-load-okapi.dev.folio.org:443/data-export-spring/jobs/7feb90ef-9f84-4da9-ac1a-3149c4695afe' \
--header 'x-okapi-tenant: diku' \
--header "x-okapi-token: $TOKEN" \
--header 'Content-Type: application/json'
{
    "id": "7feb90ef-9f84-4da9-ac1a-3149c4695afe",
    "name": "000001",
    "source": "diku_admin",
    "isSystemSource": false,
    "type": "BULK_EDIT_IDENTIFIERS",
    "exportTypeSpecificParameters": {},
    "status": "SUCCESSFUL",
    "files": [
        "https://folio-snapshot-load-export.s3.us-east-1.amazonaws.com/userIds.csv?X-Amz-Algorithm=AWS4-HMAC-SHA256&X-Amz-Credential=AKIA2VGNPE73JV77RBGE%2F20211208%2Fus-east-1%2Fs3%2Faws4_request&X-Amz-Date=20211208T043603Z&X-Amz-Expires=604800&X-Amz-SignedHeaders=host&X-Amz-Signature=fa917db496918c3c3a7ff2bade167e7ed60414ee0587abd560a6616dd9352a6d"
    ],
    "startTime": 1638938162877,
    "endTime": 1638938163471,
    "metadata": {
        "createdDate": 1638938131732,
        "createdByUserId": "cd8fdbda-7db5-5734-a7f3-5e4315ef8f6a",
        "createdByUsername": "diku_admin",
        "updatedDate": 1638938163793,
        "updatedByUserId": "cd8fdbda-7db5-5734-a7f3-5e4315ef8f6a",
        "updatedByUsername": "diku_admin"
    },
    "identifierType": "BARCODE",
    "entityType": "USER"
}

The result is in attached userIds-2.csv file.

Generated at Thu Feb 08 22:30:06 UTC 2024 using Jira 1001.0.0-SNAPSHOT#100246-sha1:7a5c50119eb0633d306e14180817ddef5e80c75d.