Page Comparison

Flow description

1. JobDefinition (uuid, profile - defines job type: insert/update) for import created in mod-srm
2. MARC file + JobDefinition ID are uploaded from WEB client to mod-data-import (stored in memory, can be persisted. possible oom)
3. MARC records are packed into batches and put to Kafka queue
   
4. mod-srm reads batches from the queue, validates and passes to mod-srs via Kafka queue. JobStarts on first chunk received
5. mod-srs stores records (creates new generation for updates, has exclusion! Kateryna Senchenko ) into PostgreSQL database and returns the result back via Kafka queue (broken records are also stored as 'error record') -
6. mod-srm reads the profile and creates JSON payload (containing parsed MARC, profile, mapping parameters) for processing. exports it to Kafka queue (one message per MARC entry), (topic-1 Kateryna Senchenko)

FOR CREATE

1. mod-inventory reads the message, creates Instance. Stores (via OKAPI HTTP) in mod-inventory-storage. Exports message ('INVENTORY_INSTANCE_CREATED')
   
2. mod-srs reads message and updates according entry with instance id. Creates new message with updated payload ('DI_SRS_MARC_BIB_INSTANCE_HRID_SET')
3. mod-inventory updates Instance with updated fields (Kateryna Senchenko export or same thread?)
   
4. mod-inventory reads the message, creates Holdings. Stores (via OKAPI HTTP) in mod-inventory-storage. Exports message (Kateryna Senchenko)
5. mod-inventory reads the message, creates Items. Stores (via OKAPI HTTP) in mod-inventory-storage. Exports message (Kateryna Senchenko)
6. mod-inventory reads message and exports to 'DI_COMPLETED' (Kateryna Senchenko)
   

FOR UPDATE

1. mod-inventory / mod-srs reads the message , tries to match it according to profile rules. Exports matched records to Kafka queue one by one.from (topic: Kateryna Senchenko)
   

UPDATE:

1. mod-inventory reads message from 'topic-1'. Tries to find entity according to match criteria (in mod-inventory-storage via OKAPI HTTP)
   
2. If found: exports result to ('inventory instance matched': Kateryna Senchenko )
   1. mod-inventory receives match result and updates Instance/Holding/Item (according to action in profile) in mod-inventory-storage
   reads messages and stores matched entity to DB. exports the result to queue
   1. according to profile. And publishes result to applied topic (one per entity type) Kateryna Senchenko)
      
   2. mod-inventory reads
   the message, tries to match (cycle 7-9).
3. if nothing can be matched, exports result to kafka queue
4. mod-srm reads queue, updates the progress. updates job state.
   1. message from prev. step (1 of 3 topics) and seeks for more actions (return to a.)  If no more actions in profile: export to 'DI_COMPLETED'
5. If not found: follow 'NON-MATCHED' branch in profile (go to 'CREATE' or 'UPDATE' or export to 'DI_COMPLETED' for empty actions)

AFTER PROCESSING

1. mod-srm reads 'DI_COMPLETED'/'DI_ERROR' updates job progress