[UXPROD-3503] Update Kafka Message Queues for Select Circulation Storage Record Types to Use Outbox Pattern Created: 13/Jan/22 Updated: 08/May/23 |
|
| Status: | Draft |
| Project: | UX Product |
| Components: | None |
| Affects versions: | None |
| Fix versions: | TBD |
| Type: | New Feature | Priority: | P2 |
| Reporter: | Brooks Travis | Assignee: | Tim Auger |
| Resolution: | Unresolved | Votes: | 0 |
| Labels: | check-in, consortia, d2ir-circulation-flow, kafka, loans, requests, resourceaccess, volaris-candidate | ||
| Remaining Estimate: | Not Specified | ||
| Time Spent: | Not Specified | ||
| Original estimate: | Not Specified | ||
| Issue links: |
|
||||||||
| Front End Estimate: | Out of scope | ||||||||
| Back End Estimate: | XXL < 30 days | ||||||||
| Development Team: | Volaris | ||||||||
| PO Rank: | 0 | ||||||||
| Rank: Cornell (Full Sum 2021): | R1 | ||||||||
| Description |
|
Current situation or problem: Initial implementation of Kafka messaging for loan, request, and check-ins was made without outbox pattern recommended in https://folio-org.atlassian.net/wiki/display/DD/Eventual+consistency+for+duplicated+data (and likely to be the standard approach for data sync in FOLIO). We need to circle back and update these implementations to use this pattern to enhance reliability in future releases. In scope
Out of scope
Use case(s): Proposed solution/stories
Links to additional info Questions |
| Comments |
| Comment by Dima Tkachenko [ 28/Jan/22 ] |
|
I wouldn't recommend making any serious changes (like adding new features) without re-writing the module into Spring. In its current state it's hard to extend the module because of the code compleхity and making further changes will make it even worse. Alternatively the code can be refactored staying with Vert'x / RMB frameworks, but I'd suggest switching to Spring as it's more simple from dev perspective and suits better to existing business tasks. Putting an estimate ONLY for outbox pattern implementation. Let me know if I should include refactoring effort also cc: Oleksandr Dmytriiev , Khalilah Gambrell , Oleksii Petrenko |
| Comment by Khalilah Gambrell [ 06/Sep/22 ] |
|
Hey Brooks Travis and Tim Auger, how can this be In Refinement when there are no user stories tied to it? |
| Comment by Tim Auger [ 05/May/23 ] |
|
Brooks Travis It is not clear to me what problem we are trying to solve. Can you help me understand? Thanks. |
| Comment by Brooks Travis [ 08/May/23 ] |
|
Failed message send. Right now, the Kafka message is published as part of the logic that writes the changes to circulation storage. If Kafka is unavailable, the message could be dropped and never received by the subscribing parties (mod-inn-reach in this case). Outbox pattern for Kafka messages was the recommendation for implementing them more widely, but there was a problem with how the RMB modules were implemented that made it basically impossible without a rewrite, so they just implemented them like inventory. This should probably be classed as tech debt. |