Data import: circulation notes created via import cause blank references, rendering items uneditable within the UI (short term fix) (Orchid CSP 6 Clone)

Description

Overview: Data import field mapping profiles for items allow the addition/update/removal of circulation notes. However, when adding circulation notes to items via data import, null ID values are added to the circulation notes. This prevents records from being editable in the UI after the fact. Once the circulation notes are removed via import or API, the records are editable again.

Here is a sample JSON response:

"circulationNotes": [
       {
           "id": null,
           "noteType": "Check in",
           "note": "Check for discs.",
           "staffOnly": true,
           "source":

{                "id": null,                "personal": null            }

,
           "date": null
       },
       {
           "id": null,
           "noteType": "Check out",
           "note": "Check for discs and confirm number with patron.",
           "staffOnly": true,
           "source":

{                "id": null,                "personal": null            }

,
           "date": null
       }
   ],

Steps to Reproduce:

  1. Log into https://bugfest-orchid.int.aws.folio.org/ as a user with admin permissions.

  2. Open the Inventory app to the item search segment.

  3. Narrow to material type = molly

    1. Two records should appear.

  4. Click Actions > Save instance UUIDs

    1. A CSV file will be downloaded to your machine containing two instance UUIDs.

  5. Open the Data export app.

  6. Upload the CSV file that downloaded from step 4.

  7. Run using the SRS + Item HRID data export job profile (0e0599dc-1ac9-41ae-9814-c6ca7aa3e05c) and indicating you've provided instance UUIDs.

  8. Download the resulting .mrc file.

    1. This should have item HRIDs in the 900ff$a.

  9. Open the Data import app.

  10. Upload the .mrc file that was saved to your computer during step 8.

  11. Run using the Molly Circ Notes Test profile (3a0c92b1-1e0d-40a4-9fa4-00e118d652c3).

    1. This profile matches incoming 900ff$a to the item HRID with a static submatch to 'molly' as the material type, then adds check out and checkin notes to the item records when a match is found.

  12. Once the job completes, open the log, and click on one of the updated items.

    1. This will route you to the item record in Inventory.

    2. You should see the circulation notes on the record.

  13. Click Actions > Edit.

  14. Try to make a change to the record, then Save & close.

Expected Results: Item record saves with edits.

Actual Results: The save & close button is grayed out for a time, but then resets and the record is not saved. 

The dev tools network tab shows a canceled status for the PUT request with "Failed to load response data: No data found for resource with given identifier.

The console shows request timeout errors.

Additional Information: Originally discovered in a Nolana environment, escalated to EBSCO hosting who diagnosed the null values.

Replicated on Orchid Bugfest, so escalating to Support SIG for resolution within the Community.

BE: When Item is created/updated in inventory-storage - check for circ notes and generate UUIDs for each note that does not have the id assigned

Test info

  1. Test 1: Create 2 items, 1 with mapped circ notes, 1 without

    • Instance: use default create instance profile

    • Holdings field mapping: default a permanent location, no other data

    • Item field mapping: default Material type, perm loan type, status

    • Map the Circulation notes from the incoming file

      • Default: Note type: default to Check-out note type, 901$aCirc note text, default the staff only only checkbox

    • In the MARC file, have 2 records

      • Record 1: has 1 901 fields: with 901$a only

      • Record 2: has NO 901 field

    • Import to create instance, holdings, item

    • Results

      • Record 1: should have a check-out note, with text from 901$a, and marked staff only

      • Record 2: should not have any circulation note: no circ note type, no text, no staff only checkbox

  2. Test 2: Update those 2 items

    • Export those 2 instances, with item HRID in 900$a

    • Edit the export records

      • Record 1: remove the 901

      • Record 2: add a 901$a with check-in text

    • Match 900$a to item record HRID

    • Item action: Update item

    • Item field mapping:

      • Circ notes: Remove existing and add new

      • Mapping: Note type: default to check-in note, with text from 901$a, staff only: do NOT mark the checkbox

    • Create import profile

      • Match to item

      • Update item

    • Results:

      • Item 1: Check-out note is removed, and there's no new circ note added to the item, because no 901$a in the incoming record

      • Item 2: No note to remove, but new check-in note added to the item, using defaults and 901$a data

ORCHID Critical service patch details

  1. Describe issue impact on business If a field mapping references a MARC field as one with notes data, and there is no notes data, it will create a note field in the item with a blank note, AND make the item record un-editable via the UI. That is a significant data issue. This fix allows the item to be editable via the UI, but we're making a more significant fix in Poppy that will not create the item note field at all, if there is no actual note text to populate into it. So this is a short-term fix, which creates the item circ note field (with note type and staff only values, but no note text) and allows the item to be edited.

  2. What institutions are affected? (field “Affected Institutions” in Jira to be populated) All

  3. What is the workaround if exists? None, without editing the item in the backed via API

  4. What areas will be impacted by fix (i.e. what areas need to be retested) Data Import smoke and critical path tests, plus the manual test associated with this Jira

  5. Brief explanation of technical implementation and the level of effort (in workdays) and technical risk (low/medium/high). Work is complete and level effort is low. Very minimal risk.

  6. Brief explanation of testing required and level of effort (in workdays). Provide test plan agreed with by QA Manager and PO. Test cases already exist and level of effort is .5 day

  7. What is the roll back plan in case the fix does not work? Revert to previous version

CSP Request Details

Orchid CSP requested 1 Sept 2023 on Slack. Followed up 5, 12, 19 Sept. Approved 21 Sept by ...

CSP Rejection Details

None

Potential Workaround

From Molly: Labeled a P2 because the workaround is onerous – must remove notes via data import or API.

Attachments

6

Checklist

hide

TestRail: Results

Activity

Show:

Maksym Ishchenko September 26, 2023 at 9:36 AM

Hello  

Ticket verified on Orchid Bugfest env. Circulation notes remains empty when there is no field in incoming file. Result jobs here: https://bugfest-orchid.int.aws.folio.org/data-import/job-summary/bb54b8fa-ab3d-4d2c-b5fe-4b0cad80e364 and  https://bugfest-orchid.int.aws.folio.org/data-import/job-summary/8841f622-4731-4997-b2c6-d7c22377aba0

 

Please doublecheck and if all looks good to you close the ticket

Maksym Ishchenko September 6, 2023 at 12:06 PM

Return ticket to "In QA" column because it's needed Testrail update

Kateryna Senchenko September 6, 2023 at 11:38 AM

For records that don't have the value specified in the mapping profile notes (with empty values) will not be created. It was fixed in scope of

Ann-Marie Breaux September 1, 2023 at 3:30 PM

Please make sure we have TestRails that match the testing descriptions laid out in the Jira description. Thank you!

Molly Driscoll August 28, 2023 at 5:01 PM

Results:

 

Done

Details

Assignee

Reporter

Priority

Story Points

Sprint

Development Team

Folijet

Fix versions

Release

Orchid (R1 2023) Service Patch #6

RCA Group

Implementation coding issue

CSP Approved

Yes

Affected releases

Orchid (R1 2023)
Nolana (R3 2022)

Affected Institution

!!!ALL!!!

TestRail: Cases

Open TestRail: Cases

TestRail: Runs

Open TestRail: Runs

Created May 11, 2023 at 4:22 PM
Updated October 13, 2023 at 11:29 AM
Resolved September 21, 2023 at 2:29 PM
TestRail: Cases
TestRail: Runs