Batch Importer (Bib/Acq) (UXPROD-47)

[UXPROD-3173] Folijet support work for Inventory Optimistic Locking (Data Import and SRS) Created: 09/Jul/21  Updated: 07/Mar/22  Resolved: 07/Mar/22

Status: Closed
Project: UX Product
Components: None
Affects versions: None
Fix versions: Lotus (R1 2022)
Parent: Batch Importer (Bib/Acq)

Type: New Feature Priority: P2
Reporter: Ann-Marie Breaux (Inactive) Assignee: Ann-Marie Breaux (Inactive)
Resolution: Done Votes: 0
Labels: data-import, epam-folijet, optimistic_locking
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original estimate: Not Specified

Issue links:
Defines
defines UXPROD-47 Batch Importer (Bib/Acq) Analysis Complete
is defined by MODDATAIMP-541 SPIKE: investigate causes of inconsis... Closed
is defined by UIDATIMP-995 Test Data Import with Optimistic Lock... Closed
is defined by UIDATIMP-996 Test Data Import with Optimistic Lock... Closed
Relates
relates to UXPROD-3058 Optimistic Locking In Progress
relates to UIIN-1527 Error popup on HTTP error status code... Closed
relates to UXPROD-3089 Inventory. Implementing Optimistic Lo... Closed
relates to UIIN-1245 Implement optimistic locking in Inven... Closed
Release: Lotus R1 2022
Epic Link: Batch Importer (Bib/Acq)
Back End Estimate: Small < 3 days
Development Team: Folijet
PO Rank: 112
Rank: Cornell (Full Sum 2021): R1
Rank: U of AL (MVP Oct 2020): R1

 Description   

NOTE: This feature assesses how much impact the Inventory Optimistic Locking work has on Data Import. If any UI changes are needed (for the Data Import log or error messages in the UI), then a follow-on feature for Lotus will be added

Current situation or problem: Inventory is implementing Optimistic locking in Kiwi. Folijet needs to review and determine how Data Import and SRS may be affected. There are already conflicts when 2 processes, or a person and a process are acting on the same record at the same time. Currently, one set of changes is discarded silently, with no error messages or UI indication. UXPROD-3058 In Progress lays out more details so that apps can return messages indicating when record edit conflicts occur. Data Import and SRS will need to react to those messages when the conflict (one of the simultaneous edits) is caused by Data Import.

FOLIO-wide technical proposal: https://folio-org.atlassian.net/wiki/display/DD/Optimistic+locking+proposal
PO doc on detecting and resolving conflicts: https://folio-org.atlassian.net/wiki/display/COMMUNITY/Optimistic+Locking+-++Detecting+and+Resolving+Conflicts+Requirements

In scope

  • Any Data Import or SRS activity that causes changes to any Inventory record (Instance, Holdings, Item) at the same time as another automated process
  • Any Data Import or SRS activity that causes changes to any Inventory record (Instance, Holdings, Item) at the same time as a user editing manually in the Inventory app
  • Any Data Import UI pop-ups and handling user response, if necessary
  • Any Data Import UI log messages or details to reflect conflicts or resolution of those conflicts

Out of scope

  • 2 users manually editing the same record in Inventory at the same time
  • 2 users manually editing the same record in quickMarc at the same time

Use case(s)

  • Data Import is trying to update an instance, holdings, or item at the same time that a user is manually updating the same record
  • Data Import is trying to create a holdings or item on an instance that a user is manually updating at the same time
  • Data Import is trying to create an item related to a holdings that a user is manually updating at the same time
  • Data Import is updating an instance, holdings, or item at the same time that another automated process is updating it (e.g. circulation, aged to lost, another Data Import job)
  • SRS MARC Bib is updated by a Data Import job at the same time that a user is manually updating the same record in quickMARC (or leaves the record open in Edit mode on their screen for an extended period of time)
  • SRS MARC Bib is updated by 2 Data Import jobs running at the same time
  • Would there be use cases where OL would affect matching? (if trying to match from static or from incoming record to Instance, Holdings, or Item value?)
  • Is there a separate implementation that DI will need to do in case an SRS MARC is being acted upon by import B or by QM at the same as import A?

Possible use cases (validate with Kate and Ivan; maybe consolidate with above)

  • Cannot create holdings because instance was in use
  • Cannot create item because instance was in use
  • Cannot create item because holdings was in use
  • Cannot update instance because instance was in use
  • Cannot update holdings because holdings was in use
  • Cannot update holdings because instance was in use
  • Cannot update item because item was in use
  • Cannot update item because holdings was in use
  • Cannot update item because instance was in use
  • Would there be use cases where OL would affect matching? (if trying to match from static or from incoming record to Instance, Holdings, or Item value?)
  • Is there a separate implementation that DI will need to do in case an SRS MARC is being acted upon by import B or by QM at the same as import A?

Proposed solution/stories

Links to additional info

Questions



 Comments   
Comment by Ann-Marie Breaux (Inactive) [ 10/Aug/21 ]

Hi Debra Howell and Tom Wilson I see that you've marked this as P1, and I think it's totally appropriate. We're hoping to work on this in Kiwi, and we have a placeholder for it, but we don't have any clear user stories for UI or backend requirements. I'm hoping that we'll be able to start filling in these details in the next few weeks. So all of that to say - stay tuned!

Comment by Ann-Marie Breaux (Inactive) [ 12/Aug/21 ]

Next steps in defining this feature:

  1. Taisiya Trunova scheduling meeting for next week with Vladimir Shalaev, Julian Ladisch, Kateryna Senchenko and optional Jakub Skoczen, Mark Veksler to go over OL and implications for DI backend
  2. Vladimir Shalaev to outline design for required DI BE work
  3. Khalilah Gambrell to finalize Inventory UI requirements (https://folio-org.atlassian.net/browse/UIIN-1245 and https://folio-org.atlassian.net/wiki/display/COMMUNITY/Optimistic+Locking+-++Detecting+and+Resolving+Conflicts+Requirements)
  4. Ann-Marie Breaux to finalize DI UI requirements with Folijet UI devs and the SMEs
Comment by Ann-Marie Breaux (Inactive) [ 25/Aug/21 ]

Grooming: for Kiwi focus on unblocking the problems when Julian merges existing changes. Will need to test DI performance before and after the merge. If any problems with versioning, then the job will be completed with errors.

Next steps: Kateryna Senchenko to write the BE stories and Vladimir Shalaev to review

If a record is being used, then we will end up with "discarded" on the summary page. Once DI can accept OL errors, try and see if we can trigger the error. Then meet with Ivan and Kimie to discuss UI options, which may need to be in Lotus.

Comment by Ann-Marie Breaux (Inactive) [ 01/Sep/21 ]

Grooming:

  1. Folijet resolves the PTF P1 bug MODDATAIMP-541 Closed
  2. PTF test DI performance before the OL changes in MODINV UIDATIMP-995 Closed
  3. Merge the MODINV and MODINVSTOR changes (Prokopovych)
  4. PTF retests DI performance after the changes UIDATIMP-996 Closed
  5. We're confident that DI will not stop/die when it encounters OL error message from Inventory. Folijet may describe or test to ensure that DI doesn't stop or get stuck when an Inventory record is being manually edited, or SRS MARC is being manually edited in QM, while import is also trying to act on it. See if we can figure out how to trigger it, but it may be difficult to trigger UIDATIMP-998 Closed
  6. Look at the error message that is triggered for DI and see if we need any UI story, or if existing error message is clear enough
Comment by Ann-Marie Breaux (Inactive) [ 22/Nov/21 ]

Hi Martin Tran Just wanted to check on whether UIDATIMP-995 Closed and UIDATIMP-996 Closed can be tested in the next couple of weeks. I don't think there's additional work that Folijet needs to do for Kiwi, but it would be good to have confirmation on these issues so that it's clear if any work is needed in Lotus. Thank you!

Comment by Martin Tran [ 22/Nov/21 ]

Hi Ann-Marie Breaux, yes, the PTF can test DI with/without OL in the next couple of weeks.

Generated at Fri Feb 09 00:29:55 UTC 2024 using Jira 1001.0.0-SNAPSHOT#100246-sha1:7a5c50119eb0633d306e14180817ddef5e80c75d.