Apache Kafka usage in inventory-storage

Background

Apache Kafka should be used directly in inventory-storage for implementation of domain event pattern, which is needed for both remote-storage integration and new search module. This direct usage of Apache Kafka should be considered as a special case, as it was for data-import. Also it is special case, that storage module use more than one data storage. For search module the performance is significant. Kafka partitioning provide system with out-of-the-box mechanism for scaling out modules.

It will be one fault tolerant solution for sending notification to other applications, when data is changed in inventory-storage. This notifications allow us to make business logic segregation and avoid implementation of not related business logic in inventory module:

• Remote-storage module will start accession process based on these notifications.
• Search will need to reindex data based on this notifications, so system cannot live with missing even one update notification, because the data would be inconsistent in such case.
• In future the same notifications may be consumed by other modules (e.g. possible implementation of INN-Reach)

Action items

• Implement sending of Kafka messages (domain events) from inventory storage, when instance/holding/item is changed. There should separate topics for each type of entities. The id of the changed entity and difference (between old/new) should be provided in the body of the message.