Background
Apache Kafka should be used directly in inventory-storage for implementation of domain event pattern, which is needed for both remote-storage integration and new search module. This direct usage of Apache Kafka should be considered as a special case, as it was for data-import. Also it is special case, that storage module use more than one data storage. However, Kafka is used here not for data storage, but for event provision for interaction between modules. Also it least for search module, the performance is significant. Kafka partitioning provide system with out-of-the-box mechanism for scaling out modules if there is a high-load.
It will be one fault tolerant solution for sending notification to other applications, when data is changed in inventory-storage. This notifications allow us to make business logic segregation and avoid implementation of not related business logic in inventory module:
- Remote-storage module will start accession process based on these notifications.
- Search will need to reindex data based on this notifications, so system cannot live with missing even one update notification, because the data would be inconsistent in such case.
- In future the same notifications may be consumed by other modules (e.g. possible implementation of INN-Reach)
Action items
- Implement sending of Kafka messages (domain events) from inventory storage, when instance/holding/item is changed. There should separate topics for each type of entities. The id of the changed entity and difference (between old/new) should be provided in the body of the message.