Apache Kafka usage in inventory-storage
Status | Done |
|---|---|
Stakeholders | @Mikhail Fokanov @Marc Johnson @VBar |
Outcome | Apache Kafka will be used directly in inventory-storage as a special case |
Created date | Nov 12, 2020 |
Owner | @Mikhail Fokanov |
NOTICE
This decision has been migrated to the Technical Council's Decision Log as part of a consolidation effort. See: DR-000006 - Apache Kafka Usage in Inventory Storage
Background
Apache Kafka should be used directly in inventory-storage for implementation of domain event pattern, which is needed for both remote-storage integration and new search module. This direct usage of Apache Kafka should be considered as a special case, as it was for data-import. Also it is special case, that storage module use more than one data storage. For search module the performance is significant. Kafka partitioning provide system with out-of-the-box mechanism for scaling out modules.
It will be one fault tolerant solution for sending notification to other applications, when data is changed in inventory-storage. This notifications allow us to make business logic segregation and avoid implementation of not related business logic in inventory module:
Remote-storage module will start accession process based on these notifications.
Search will need to reindex data based on this notifications, so system cannot live with missing even one update notification, because the data would be inconsistent in such case.
In future the same notifications may be consumed by other modules (e.g. possible implementation of INN-Reach)