Data consistency and message driven approach
NOTICE
This decision has been migrated to the Technical Council's Decision Log as part of a consolidation effort. See: DR-000029 - Data consistency and message driven approach
Status | IN PROGRESS |
|---|---|
Stakeholders | Front-end and back-end devs who meet issues with data consistency |
Outcome |
|
Created date | Jun 30, 2021 |
Owner | @Raman Auramau |
Changes list
Data | Author | Comment |
|---|---|---|
|
|
|
Jun 9, 2021 | Raman Auramau | More details on Consistency for Dangling References |
May 17, 2021 | Raman Auramau |
|
Apr 26, 2021 | Raman Auramau | Filled in the section with Eventual Consistency for redundant data |
Apr 22, 2021 | Raman Auramau | Initial document |
Summary
This is a solution design document aimed to provide details, alternatives and decision for FOLIO cross-module data consistency problem.
Problem Statement
Data consistency refers to whether the same data kept at different places do or do not match. Being a distributed microservices-based platform, FOLIO is a number of separate modules with own data schemas and storages. So, FOLIO follows the principle to keep each module’s persistent data private to that module and accessible only via its API. A module’s transactions only involve its database.
With that, this approach has some drawbacks, among them -
Implementing business transactions that span multiple services is not straightforward,
Implementing queries that join data that is now in multiple databases is challenging.
https://folio-org.atlassian.net/browse/MODORDERS-399
Basing on currently known Folio issues with Data consistency one can see that FOLIO has difficulties caused by both mentioned shortcomings. Those difficulties can be divided into the following groups:
Priority 1 eventual consistency for redundant / duplicated data when some data is duplicated in 2 storages and is to be synchronized,
Priority 2 consistency for dangling / lost references when an item is deleted from one module leaving lost references to it in other modules, a problem that is succinctly, if frustratingly, captured in the PR discussion related to UITEN-128.
Priority 3 data consistency during distributed business operations when data in several separate storages is to be modified (mod-finance-storage, mod-invoice-storage, mod-orders-storage),
updates collisions ... (check with Jacub? )
References
On distributed updates and eventual consistency
2021-05-12 Meeting notes - there was a lot of discussion on related issues
Eventual consistency for duplicated data
Brief context: a source module owns an entity; a particular entity field is duplicated into 1+ entities of another module (e.g., for search, or filtering, or sorting). If an original field value in source module is changed, the change is to be replicated everywhere.
Identified cases:
Pair of RefNumber and RefType should be in consistence state between POL and invoice line https://folio-org.atlassian.net/browse/MODORDERS-421
mod-orders → mod-invoice
1-to-1 relation - one pair of refNumber/refNumberType to one invoice record ( - not sure, need to confirm)
VendorCode should be in consistence state between Organization record and purchaseOrder.vendorCode https://folio-org.atlassian.net/browse/MODORDERS-398
mod-organizations → mod-orders
1-to-many relation - one vendor code can be used in many orders
FundCode should be in consistence state between Fund record and pol.fundDistribution.code
mod-finance → mod-orders
1-to-many relation - one fund code can be used in many orders
Outstanding questions
Status | Item | Details, comments, decisions |
|---|---|---|
@Raman Auramau How many data (rows) can be affected? | @Raman Auramau Assumption is up to dozens of thousand (e.g. changing of vendor or fund code can affect thousands+ orders | |
@Raman Auramau Are there any specific performance requirements (i.e. how fast data are to be synchronized)? |
| |
@Raman Auramau What is an expected behavior in case synchronization fails? Options - rollback, continue from the same record, retry (1..N times), report an error |
| |
@Raman Auramau What is the allowable lag between changing a value in module-source and updating it in module-recipient? |
| |
|
|
|
Evaluated Alternatives
Alternative | Reasoning |
|---|---|
API Composition is a pattern for micro-service-based platforms for implementing queries that span services. In this approach the application performs the data join rather than the database. For example, a service (or the API gateway) could retrieve a customer and their orders by first retrieving the customer from the customer service and then querying the order service to return the customer’s most recent orders. | Redundant data are currently used for both visualization and filtering / search. API Composition approach likely will add additional internal API calls which might impact on overall performance and efficiency. Also, certain re-thinking of work flow will be required. |
Command Query Responsibility Segregation - maintain one or more materialized views that contain data from multiple services. The views are kept by services that subscribe to events that each services publishes when it updates its data. For example, the online store could implement a query that finds customers in a particular region and their recent orders by maintaining a view that joins customers and orders. The view is updated by a service that subscribes to customer and order events. | The views mechanism requires certain efforts for implementation and resources for keeping views in an actual state. Meanwhile currently known use cases are pretty straightforward. So, this approach seems to be to complex for this particular case. |
Domain-event pattern for change notifications, and data normalization to simplify data synchronization - implement a notification channel for easy (though guaranteed) delivery of changes, and improve data normalization to achieve 1-to-1 updates. | Enable keeping current FOLIO approach for certain data redundancy while solve eventual consistency issue with minimum efforts. |