[UXPROD-3503] Update Kafka Message Queues for Select Circulation Storage Record Types to Use Outbox Pattern Created: 13/Jan/22  Updated: 08/May/23

Status: Draft
Project: UX Product
Components: None
Affects versions: None
Fix versions: TBD

Type: New Feature Priority: P2
Reporter: Brooks Travis Assignee: Tim Auger
Resolution: Unresolved Votes: 0
Labels: check-in, consortia, d2ir-circulation-flow, kafka, loans, requests, resourceaccess, volaris-candidate
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original estimate: Not Specified

Issue links:
Cloners
clones UXPROD-3380 Add Kafka Message Queues to Select Ci... Closed
Front End Estimate: Out of scope
Back End Estimate: XXL < 30 days
Development Team: Volaris
PO Rank: 0
Rank: Cornell (Full Sum 2021): R1

 Description   

Current situation or problem:

Initial implementation of Kafka messaging for loan, request, and check-ins was made without outbox pattern recommended in https://folio-org.atlassian.net/wiki/display/DD/Eventual+consistency+for+duplicated+data (and likely to be the standard approach for data sync in FOLIO). We need to circle back and update these implementations to use this pattern to enhance reliability in future releases.

In scope

Out of scope

  • Adding additional queues to CIRCSTORE

Use case(s):

Proposed solution/stories

  • Migrate CIRCSTORE to Spring Way to use existing libraries available for Spring Way that can implement this pattern

Links to additional info

Questions



 Comments   
Comment by Dima Tkachenko [ 28/Jan/22 ]

Hi Brooks Travis

I wouldn't recommend making any serious changes (like adding new features) without re-writing the module into Spring. In its current state it's hard to extend the module because of the code compleхity and making further changes will make it even worse. Alternatively the code can be refactored staying with Vert'x / RMB frameworks, but I'd suggest switching to Spring as it's more simple from dev perspective and suits better to existing business tasks.

Putting an estimate ONLY for outbox pattern implementation. Let me know if I should include refactoring effort also

cc: Oleksandr Dmytriiev , Khalilah Gambrell , Oleksii Petrenko

Comment by Khalilah Gambrell [ 06/Sep/22 ]

Hey Brooks Travis  and Tim Auger, how can this be In Refinement when there are no user stories tied to it? 

Comment by Tim Auger [ 05/May/23 ]

Brooks Travis It is not clear to me what problem we are trying to solve. Can you help me understand? Thanks.

Comment by Brooks Travis [ 08/May/23 ]

Failed message send. Right now, the Kafka message is published as part of the logic that writes the changes to circulation storage. If Kafka is unavailable, the message could be dropped and never received by the subscribing parties (mod-inn-reach in this case). Outbox pattern for Kafka messages was the recommendation for implementing them more widely, but there was a problem with how the RMB modules were implemented that made it basically impossible without a rewrite, so they just implemented them like inventory. This should probably be classed as tech debt.

Generated at Fri Feb 09 00:32:32 UTC 2024 using Jira 1001.0.0-SNAPSHOT#100246-sha1:7a5c50119eb0633d306e14180817ddef5e80c75d.