Platform, DevOps and Release Management
(UXPROD-1814)
|
|
| Status: | Closed |
| Project: | UX Product |
| Components: | None |
| Affects versions: | None |
| Fix versions: | R1 2021 | Parent: | Platform, DevOps and Release Management |
| Type: | New Feature | Priority: | P3 |
| Reporter: | Jakub Skoczen | Assignee: | Jakub Skoczen |
| Resolution: | Done | Votes: | 0 |
| Labels: | Showstopper-Cornell, Support, platform-backlog, round_iv | ||
| Remaining Estimate: | Not Specified | ||
| Time Spent: | Not Specified | ||
| Original estimate: | Not Specified | ||
| Attachments: |
|
||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
| Issue links: |
|
||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
| Epic Link: | Platform, DevOps and Release Management | ||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
| Back End Estimate: | XXL < 30 days | ||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
| Development Team: | Core: Platform | ||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
| Rank: Chalmers (Impl Aut 2019): | R1 | ||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
| Rank: Chicago (MVP Sum 2020): | R1 | ||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
| Rank: Cornell (Full Sum 2021): | R1 | ||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
| Rank: Duke (Full Sum 2021): | R1 | ||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
| Rank: 5Colleges (Full Jul 2021): | R1 | ||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
| Rank: FLO (MVP Sum 2020): | R1 | ||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
| Rank: GBV (MVP Sum 2020): | R1 | ||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
| Rank: hbz (TBD): | R1 | ||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
| Rank: Lehigh (MVP Summer 2020): | R1 | ||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
| Rank: Leipzig (Full TBD): | R1 | ||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
| Rank: Leipzig (ERM Aut 2019): | R1 | ||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
| Rank: MO State (MVP June 2020): | R1 | ||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
| Rank: TAMU (MVP Jan 2021): | R1 | ||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
| Rank: U of AL (MVP Oct 2020): | R1 | ||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
| Description |
|
NOTE: Optimistic locking is the solution described in this feature. Libraries ranked this based on the idea of preventing update conflicts, not necessarily based on the specific solution of optimistic locking. Problem statement In FOLIO, most storage modules follow the "last writer wins" strategy for handling record updates. From the UI perspective this may lead to a situation when a stale record (older version of a give record) previously loaded into the UI may override a more recent version on the server. Hence relevant updates may get lost in the process and the user is not made aware of what has happened. Scope: The scope of this issue is to create platform support for optimistic locking which modules can make use of on a case-by-case basis (opt-in). Focus of this feature is on simple "detection" and "prevention" (identifying when a collision has occurred and preventing it). Additional tools and mechanisms for handling collisions when they occur (e.g. diffs, merges etc.)). There are 3 phases, two of which are in scope for this feature:
Proposed solution Handling of updates in FOLIO should rely on more explicit semantics, both in the storage (backend) APIs and the way it is communicated to the user through the UI. From the storage and API perspective, optimistic locking is the proposed strategy to handle conflicts:
In general, optimistic locking is used when the risk of collisions (updates to the same record) is low and when the lock granularity is high ((ie duration of any given update is short). Use cases collected from community add others that seem likely
User impact This approach will not prevent collisions, but it would notify the user when they happen and offer them a choice. Something like, "Sorry AgentB has already updated the record and your working copy might not be up-to-date? Would you like to:" (a) Update anyway (b) Reload.
This situation can happen when multiple data imports are happening at the same time (or data import and a user acting on the same record at the same time) and can affect many records at the same time. Cleanup can then be very time-consuming and confusing. |
| Comments |
| Comment by Cate Boerema (Inactive) [ 29/Jul/19 ] |
|
Jakub Skoczen I am assigning this to you as PO since it's Core Platform. Can you add the PO rank? |
| Comment by Theodor Tolstoy (One-Group.se) [ 15/Jan/20 ] |
|
Should w e add the NFR tag to this? |
| Comment by Ann-Marie Breaux (Inactive) [ 09/Apr/20 ] |
|
Cate Boerema Jakub Skoczen Theodor Tolstoy (One-Group.se) Also added a note in the description that this may happen in cases of data import or batch editing, where large numbers of records may be affected at once. Based on convo at MM SIG today. |
| Comment by Erin Nettifee [ 11/May/20 ] |
|
Asking same question as Theodor Tolstoy (One-Group.se) - is this NFR? It's shown up on the Round IV list, but this seems like tech debt and I'm not sure how we would rank this other than go live. |
| Comment by Holly Mistlebauer [ 17/Jun/20 ] |
|
Cornell comment from Round IV Outliers spreadsheet: This outlier is a showstopper for Cornell |
| Comment by Holly Mistlebauer [ 04/Sep/20 ] |
|
Cate Boerema: As we are re-planning what was formerly Q4 2020 (and is now simply Iris), it seems that we should revisit this feature. This is a go-live feature for every institution and is currently causing issues (see Support issue
|
| Comment by Anya [ 10/Sep/20 ] |
|
Cate Boerema is there an update on this ticket - I am being asked about this on many fronts |
| Comment by Cate Boerema (Inactive) [ 11/Sep/20 ] |
|
Hi Anya. I think Jakub Skoczen would be better positioned to provide an update. I know this is being discussed regularly in TC and I think Core Platform is working on a design. Last I discussed with Jakub, we needed to (1) design and implement a platform solution (Core Platform team) and then (2) roll it out incrementally in FOLIO apps starting with Inventory (Core Functional etc). I have already created a draft story for Inventory here:
|
| Comment by Holly Mistlebauer [ 14/Sep/20 ] |
|
Discussed at Capacity Planning Team meeting this morning... Jakub reports that the Spike for optimistic blocking (
|
| Comment by Anya [ 16/Sep/20 ] |
|
There is a worry, that is being expressed to me, that the solution being proposed is to have better error messages... even though having better error messages are good - this doe not solve the issue of preventing or incorporating changes that are made when more than one user is editing a record. Please help me understand the proposed solution(s) for this issue. |
| Comment by Marc Johnson [ 17/Sep/20 ] |
I'm not sure where folks have got their understanding of this from. In the current system, most of the time when two potentially conflicting changes are made, there wouldn't be an error at all.
There are ongoing conversations about the specifics of the solution. I'll try to share my understanding of the general approach being taken. Let's say we have two users, Bob and Alice, editing the same instance record, Harry Potter and the Goblet of Fire. They both started around the same time (and their starting points were the same). They both make unrelated changes to the record. Bob saves the record first and then Alice. In the current system, Alices changes are remembered, and Bob's are likely lost. In the proposed solution, Bob's changes would be remembered, and Alice would receive an error that the record had changed since she started editing it. Does that make sense? Jakub Skoczen Does that match your understanding of the general approach? |
| Comment by Jakub Skoczen [ 17/Sep/20 ] |
|
Anya Yes, it's correct that the changes proposed here would result a prevention of mid-air collision by rejecting updates. It does not address specific functionality for automatically incorporating changes to a single record made by multiple users. |
| Comment by Charlotte Whitt [ 17/Sep/20 ] |
|
Marc Johnson and Jakub Skoczen - re:
Then my question is: What happens to Alice's changes, e.g. if she did a lot of updates in quickMARC? The preferred behavior will of course be, that Alice's edits has not been lost. Another question, what if Alice is not a person, but e.g. the Data Import module, doing an update at the same time as a person (Bob)? What will happen in Data Import? |
| Comment by Marc Johnson [ 17/Sep/20 ] |
That depends upon what the client, in your example the quickMARC UI, does. It could attempt to reapply Alice's changes on top of Bob's. However this assumes that it knows specifically what changes Alice made. I don't think the current UI tracks them sufficiently to do this easily. My interpretation of Jakub Skoczen's comment above:
is that kind of recovery is out of scope of this feature.
Indeed, I was trying to keep the explanation simple by only considering people interacting with the system via the UI. In your example, data import is merely a specialised client, it has to decide how it wants to react. It could attempt to reapply it's changes on top of the new update or it could abort that record and present an error to the person who imported the file. |
| Comment by Björn Muschall [ 17/Sep/20 ] |
|
Could Alice be given a UI choice between canceling and applying her changes? Then she can talk to Bob on the phone and sort things out. Or is that unrealistic in practice? The technical implementation sounds feasible, I think, but I am not a developer. It would possibly extend the function with simple methods. EDIT: I see it's already in the description, sorry. |
| Comment by Erin Nettifee [ 17/Sep/20 ] |
|
Can Alice be warned when she opens the record that Bob is already viewing/editing it? Or is this scenario intended to understand what happens if they open it at the exact same time? |
| Comment by Julian Ladisch [ 17/Sep/20 ] |
|
A warning does not prevent that they both try to save their changes. |
| Comment by Ann-Marie Breaux (Inactive) [ 17/Sep/20 ] |
|
Good point Julian Ladisch Data import may be trying to update an SRS MARC, Instance, Holdings, or Item at the same time that a user is manually updating it. If there's a person watching the import, they won't actually be on the same record as User A, so backend would need to provide some sort of message to the log that the record could not be updated because it was in use by another user or process. |
| Comment by Jenn Colt [ 17/Sep/20 ] |
|
From my data import user perspective, it is not my expectation that this issue provide warnings about things being in use, I am happy with the initial scope described as a place to start. For new records in data import, this isn't an issue. For updating records, I would expect data import to open the record at a given version(3) and then write the updated record. If someone had updated the record to version 4 between data import's read and data import's write then I would expect my data import change to be rejected with a reason in the log. I wonder if this model will hold in the bulk APIs? If at the same time a user had the record open and I saved version 4 before they did, then I would expect the user save to fail. I'm not expecting warnings about things being in use from this. We might want to give data import users the ability to override this behavior, but only with a lot of care. I think data export will need to add version # to exports if we want to be able to use that as part of an ETL chain (for marc records perhaps the 005 is enough though, if DI allows that comparison). |
| Comment by Jakub Skoczen [ 18/Sep/20 ] |
|
On the FOLIO API level there's little distinction between different types of clients: some may be connected to interactive sessions in the FOLIO UI (user editing a record) some may be connected to batch processes running in the background (and potentially using more specialised batch APIs). I think a solution where the client is allowed to "override" collision detection is certainly possible but may need to be used with extreme care. It could quickly become a source of the problem we are trying to solve. |
| Comment by Holly Mistlebauer [ 21/Sep/20 ] |
|
Update from Capacity Planning Team meeting...
|
| Comment by Jacquie Samples [ 23/Sep/20 ] |
|
Why is this in the work-around list? This is fundamental to operations (locking records when in-process) and there is no work-around listed. |
| Comment by Julian Ladisch [ 23/Sep/20 ] |
|
Jacquie Samples Which work-around list do you refer to, can you post a link to it? |
| Comment by Jacquie Samples [ 23/Sep/20 ] |
|
Hi Julian Ladisch |
| Comment by Julian Ladisch [ 23/Sep/20 ] |
|
Thanks. The "Potential Workaround" field of this issue contains this text:
I agree with Jacquie that this doesn't mention any workaround. CPT says that they are not affected in production, the others say that they are affected in production. |
| Comment by Ann-Marie Breaux (Inactive) [ 01/Oct/20 ] |
|
Meeting today: Jakub Skoczen, Julian Ladisch, Craig McNally, Ann-Marie Breaux, Charlotte Whitt, Dennis Bridges Key points: (please add any I missed in the comments)
|
| Comment by Ann-Marie Breaux (Inactive) [ 08/Oct/20 ] |
|
Comments from the Data Import Subgroup meeting 7 Oct 2020:
|
| Comment by Holly Mistlebauer [ 12/Nov/20 ] |
| Comment by Holly Mistlebauer [ 12/Nov/20 ] |
|
I've requested (on behalf of Cornell) that a meeting be set up for interested library parties. It would be good for all of us to get on the same page. It appears that this JIRA issue (
|
| Comment by Charlotte Whitt [ 13/Nov/20 ] |
|
Hi Holly Mistlebauer - will you also invite to this meeting the most directly involved POs and SIG conveners? |
| Comment by Kristin Martin [ 16/Nov/20 ] |
|
Thanks - Holly Mistlebauer are there other Jira numbers that now need to be ranked? |
| Comment by Anya [ 23/Nov/20 ] |
|
Has this meeting been set up? This has come up in some support cases as well... |
| Comment by Ann-Marie Breaux (Inactive) [ 23/Nov/20 ] |
|
Not that I know of, Anya |
| Comment by Holly Mistlebauer [ 30/Nov/20 ] |
|
'Watchers': I will ask about the meeting and the other JIRAs at tomorrow's Cap Planning meeting. |
| Comment by Holly Mistlebauer [ 30/Nov/20 ] |
|
Charlotte Whitt: The meeting will be open to anyone. |
| Comment by Holly Mistlebauer [ 01/Dec/20 ] |
|
Hi Watchers! Jakub is scheduled to discuss optimistic locking at the FOLIO Implementation Group meeting on December 15 at 11:00 AM US Eastern Time. The Zoom URL is |
| Comment by Holly Mistlebauer [ 09/Dec/20 ] |
|
Jakub's attendance at the FOLIO Implementation Group meeting has been rescheduled for January 5 at 11:00 AM US Eastern Time. The Zoom URL is still https://zoom.us/j/244921097. |
| Comment by Anya [ 22/Mar/21 ] |
|
Jakub Skoczen is there an update on this?
|
| Comment by Julian Ladisch [ 23/Mar/21 ] |
|
raml-module-builder (RMB) since version 32.0.0 ships with optimistic locking support (
Documentation: https://github.com/folio-org/raml-module-builder#optimistic-locking RMB based modules need to explicitly enable optimistic locking to use it. mod-inventory-storage since version 20.0.0 (
Iris ships with a mod-inventory-storage version >= 20.0.0. See also UXPROD-2994 "OL: coordinate rollout of "failOnConflict" to select modules and APIs". |
| Comment by Jakub Skoczen [ 20/Apr/21 ] |
|
Closing this as the RMB support for OL has been shipped in RMB 32. However, this functionality on it's own does not provide any conflict resolution facilities for individual APIs. The follow-up feature relevant to implementers is
|