| Page Properties | ||||||||
|---|---|---|---|---|---|---|---|---|
|
Overrides/Supersedes
...
This is a solution design document aimed to provide details, alternatives and decision for FOLIO cross-module data consistency problem.
Data consistency refers to whether the same data kept at different places do or do not match. Being a distributed microservices-based platform, FOLIO is a number of separate modules with own data schemas and storages. So, FOLIO follows the principle to keep each module’s persistent data private to that module and accessible only via its API. A module’s transactions only involve its database.
With that, this approach has some drawbacks, among them -
- Implementing business transactions that span multiple services is not straightforward,
- Implementing queries that join data that is now in multiple databases is challenging.
ARCH-5 - Spike : Data consistency approach for Folio OPEN
Basing on currently known Folio issues with Data consistency one can see that FOLIO has difficulties caused by both mentioned shortcomings. Those difficulties can be divided into the following groups:
- PRIORITY 1 eventual consistency for redundant / duplicated data when some data is duplicated in 2 storages and is to be synchronized,
- PRIORITY 2 consistency for dangling / lost references when an item is deleted from one module leaving lost references to it in other modules, a problem that is succinctly, if frustratingly, captured in the PR discussion related to UITEN-128.
- PRIORITY 3 data consistency during distributed business operations when data in several separate storages is to be modified (mod-finance-storage, mod-invoice-storage, mod-orders-storage),
- updates collisions ... (check with Jacub? )
...
- Pros
- N/A
- Cons
- N/A
Other Related Resources
References
On distributed updates and eventual consistency
2021-05-12 Meeting notes - there was a lot of discussion on related issues
Meetings notes
Agenda
...