SPIKE: [MODINREACH-10] Possible Methods to Trigger Actions in INN-Reach Module When a Patron Performs Certain Circulation Actions

Owned by Andrii Khodyka
Last updated: Apr 07, 2021 by Raman Auramau
3 min read

Context

Jira Story

MODINREACH-10 - Getting issue details... STATUS

Spike goals

Find the possible methods to trigger actions in inn-reach-module when Patron performs certain circulation actions.

Alternatives evaluated

Proposed solutions:

Basically, all circulation actions are events, to handle these events we could use two publish/subscribe mechanisms that are already used in FOLIO.

Common entities:

  • Event Publisher – Entity that publishes events/messages. Implemented on the side of the module that publishes the events (mod-circulation, for example).
  • Event Subscriber – Entity that subscribes to events published by the Event publisher. Implemented on inn-reach-module side.
  • Event/Message – Java object describes the event/message model. Used as an API and container for data that needs to be published/consumed on publisher/subscriber side.
  1. Apache Kafka

    The first solution is Apache Kafka. We could use it to subscribe inn-reach-module to get events.

    In this case, we need to implement the following:

    • Event publisher – Can be implemented using the native Kafka API.
    • Event subscriber – Can be implemented using native Kafka API or Spring Kafka API (since inn-reach-module is Spring based).
    • Kafka topic configuration – Spring based configuration with a topic name, partition numbers, replication factor.

    This approach is more reliable, faster, easier than the second one (mod-pubsub), since there are no intermediate services/layers and we interact directly with the Kafka cluster.

    This solution was discussed with Bohdan Suprun (Deactivated).

  2. Mod-PubSub

    The second solution is Mod-PubSub. It provides additional service/abstraction to publish/subscribe events.

    In this case. we need to implement the following:

    • Event publisher - Can be implemented using PubSubClient since most modules are Vert.x based.
    • Event subscriber – REST endpoint that will be called by Mod-PubSub to publish event. It can be implemented using Spring @RestController.
    • PubSub configuration – Spring configuration that registers inn-reach-module in the PubSub system. We can’t use PubSubClient to register inn-reach-module since PubSubClient using Vert.x entities and doesn't provide API for Spring, so it must be direct REST call with a list of events to publish/subscribe.

     This approach is less reliable, slower and more complex than the first one (Apache Kafka), since there is an intermediate service/layer (Mod-PubSub that uses Apache Kafka) and we interact with the Kafka cluster through Mod-PubSub API.

    Concerns:

    • Mod-PubSub can be a bottleneck in under heavy load
    • Mod-PubSub client doesn’t provide API for Spring based modules
    • Mod-PubSub is a potential single point of failure (SPOF)
    • Mod-PubSub is a potential source of errors, bugs (nobody is perfect)

    This solution was discussed with Oleksii Kuzminov

Summary

The final decision should be discussed with the team and Raman Auramau and based on:

  • Number of events / load factor
  • Project architecture / general practices
  • Other requirements?

Requirements and Notes

  • Guaranteed Delivery is mandatory - no changes should be lost
  • Is any change filtering required?
  • At-least-once vs. Exactly-once?
  • What about mod-audit - does it track changes from mod-circulation via mod-pubsub? If so, mod-circulation should already have an event publisher implementation with PubSubClient
  • Project architecture / general practices
    • AFAIK, there is no massive migration from mod-pubsub to Direct Kafka approach though Direct Kafka way has proven himself well in recent inventory search implementation
    • Besides mod-audit, at least one more module (for remote storage integration) tracks changes from mod-circulation - only 1 event type, implemented via pubsub (reach out to Firebird team if need more info)
  • How can required performance / load be estimated?
    • (warning) Is it possible to get some real statistics from mod-audit?
    • Actually, item circulation changes are being made manually by librarians (e.g., while scanning of barcodes etc.). So, even hundreds or thousands of librarians working with the same FOLIO instance will give relatively small change event flow.