Flow (prior to changes made in Quesnelia)- JobDefinition (uuid, profile - defines job type: insert/update) for import created in mod-srm
- MARC file + JobDefinition ID are uploaded from WEB client to mod-data-import
- MARC records are packed into batches and put to Kafka queue DI_RAW_RECORDS_CHUNK_READ
- mod-srm reads batches from the queue, validates and passes to mod-srs via Kafka queue DI_RAW_RECORDS_CHUNK_PARSED. JobStarts on first chunk received.
- mod-srs stores records into PostgreSQL database and returns the result back via Kafka queue (broken records are also stored as 'error record') - DI_PARSED_RECORDS_CHUNK_SAVED
- mod-srm reads the profile and creates JSON payload (containing parsed MARC, profile, mapping parameters) for processing. exports it to an appropriate Kafka queue (one message per MARC entry) - DI_SRS_MARC_BIB_RECORD_CREATED
- mod-inventory reads message from DI_SRS_MARC_BIB_RECORD_CREATED. Tries to find entity according to match criteria (in mod-inventory-storage via OKAPI HTTP)
- If found: exports result to DI_INVENTORY_INSTANCE_MATCHED / DI_INVENTORY_HOLDING_MATCHED / DI_INVENTORY_ITEM_MATCHED
- mod-inventory receives match result and updates Instance/Holding/Item (according to action in profile) in mod-inventory-storage according to profile. And publishes result to applied topic (one per entity type) DI_INVENTORY_INSTANCE_UPDATED / DI_INVENTORY_HOLDING_UPDATED / DI_INVENTORY_ITEM_UPDATED
- mod-inventory reads message from prev. step (1 of 3 topics) and seeks for more actions (return to a.) If no more actions in profile: export to DI_COMPLETE
- If not found: DI_INVENTORY_INSTANCE_NOT_MATCHED / DI_INVENTORY_HOLDING_NOT_MATCHED / DI_INVENTORY_ITEM_NOT_MATCHED events are published
- mod-srm reads DI_INVENTORY_INSTANCE_NOT_MATCHED / DI_INVENTORY_HOLDING_NOT_MATCHED / DI_INVENTORY_ITEM_NOT_MATCHED events and updates job status accordingly with "NOT_MATCHED" status for record
Diagram (prior to changes made in Quesnelia) Source
|