Source Record Domain Eventing
- Olamide Kolawole
This document will illustrate a design to allow mod-source-record-storage(SRS) to emit domain events upon successful modification of source records.
Code Paths
In mod-source-record-storage, there are two avenues for source record mutations; a batch mutation path and a single mutation path.
The batch mutation path was necessary because batches of source records was sent to SRS for parsing and saving prior to any job profile processing. This flow for Data Import is being changed so that every source record is not saved. Individual API calls from a Data Import Processor like mod-inventory would mutate the source records. This means that the batch mutation path would probably be deprecated in the future. Nonetheless we should keep this consistent and ensure that batch mutations also emit events.
The single mutation path should emit source records as well.
Event Model
The representation below is very similar to RAML specification used by other FOLIO modules. The only differeence is the eventPayload property.
Property | Java Type | Required? |
|---|---|---|
id | String | Yes |
eventType | String | Yes |
eventMetadata | EventMetadata(can be derived from raml or manually defined in SRS) | Yes |
eventPayload | io.vertx.core.json.JsonObject | No |
Event Types
These are the major event types
SOURCE_RECORD_CREATED
SOURCE_RECORD_UPDATED
With SOURCE_RECORD_CREATED & SOURCE_RECORD_UPDATED, the full JSON? representation of the source record will be included in the eventPayload.
SOURCE_RECORD_DELETED
TBD if this should be tracked, if so we have to define what deletion of a source record means. With SOURCE_RECORD_DELETED, eventPayload will contain only the identifier of the source record.
Should we emit the full source record for deletes? This usually means retrieval before deletion. This could be helpful for rollback or auditing scenarios.
Event Headers
There will be special header on each message from Kafka to denote whether the source record is MARC, EDIFACT or something else. Consumers of this event stream can decide to discard messages without introspecting the message.
Header Name: folio.srs.recordType
Header Value: MARC_BIB or MARC_AUTHORITY or MARC_HOLDING or EDIFACT
Ensure FOLIO’s correlation Id is forwarded as well.
Event Topic
All domain events will be pushed into a Kafka topic named srs.source_records. It will be prepended with the tenantId or topic consolidation id and also the environment. A full topic name could be folio.diku.srs.source_records .
Conditional Eventing
There may be scenarios where the creation or update of a source record should not send an event message. This will be accomplished by a flag in the Vertx context. By default, domain event will always be sent but if the flag is present a domain event is not sent.