Description

Update entities, match by Instance/Holdings/Item

Flow (according to changes made in Quesnelia R1 2024)

1. [1-4] UploadDefinition is created for importing files in mod-data-import
2. [2-3] JobExecution is created in mod-srm
3. [5] MARC files are uploaded from WEB client to mod-data-import
4. [6] Uploaded file is saved chunk by chunk in mod-data-import
5. [8-9] User selects the Job Profile and initiates the processing of the uploaded file
6. [10-11] The job profile ID is set for JobExecution
7. [12-13] MARC file is split into batches of MARC records and put to Kafka queue DI_RAW_RECORDS_CHUNK_READ
8. [14-15] mod-srm reads batches from the DI_RAW_RECORDS_CHUNK_READ queue, parses MARC records, and saves the incoming records to the DB to provide them for data import logs. JobExecution is set to IN_PROGRESS status on first chunk received.
9. [16-17] mod-srm reads the profile and creates JSON payload (containing parsed MARC) for processing. Sends it to an appropriate Kafka queue (one message per MARC entry) - DI_MARC_FOR_UPDATE_RECEIVED
10. [18-19] mod-inventory reads the message and processes instance match profile - tries to find an instance in mod-inventory-storage using match criteria from the match profile. Processes instance search result and issues an appropriate event (DI_INVENTORY_INSTANCE_NOT_MATCHED if no Instance exists satisfying search criteria, or DI_INVENTORY_INSTANCE_MATCHED if such Instance is found)
    
11. [20] If instance is not found - mod-inventory sends DI_COMPLETED since no other profiles are placed for "non-matches" branch of the Instance Match Profile. The processing flow for the particular incoming record is completed. The sent event contains result event indication (DI_INVENTORY_INSTANCE_NOT_MATCHED) denoting instance matching result
    
    1. [21] mod-srm receives DI_COMPLETED and updates the import job progress
12. [22] If instance is found - mod-inventory sends DI_INVENTORY_INSTANCE_MATCHED
13. [23-25] mod-inventory reads the message and processes update instance action profile - maps instance according to mapping profile linked to update action profile and saves updated instance in mod-inventory-storage
14. [26-29] mod-inventory updates underlying MARC record in mod-srs via HTTP and sends DI_INVENTORY_INSTANCE_UPDATED
15. [30-31] mod-inventory reads the message and processes holdings match profile - searches holdings record in mod-inventory-storage using match criteria from the match profile. Processes holdings search result and issues an appropriate event (DI_INVENTORY_HOLDING_NOT_MATCHED if no holdings record is found, or DI_INVENTORY_HOLDING_MATCHED if holding found)
16. [32] If holding is not found - mod-inventory sends DI_COMPLETED since no other profiles are placed for "non-matches" branch of the Holdings Match Profile. The processing flow for the particular incoming record is completed. The sent event contains result event indication (DI_INVENTORY_HOLDING_NOT_MATCHED) denoting holdings matching result
    1. [33] mod-srm receives DI_COMPLETED and updates the import job progress
17. [34] If holding is found - mod-inventory sends DI_INVENTORY_HOLDING_MATCHED 
18. [35-38] mod-inventory reads the message and processes update holdings action profile - maps holding according to mapping profile linked to the update action profile, saves updated holding in mod-inventory-storage and sends DI_INVENTORY_HOLDING_UPDATED
19. [38-40] mod-inventory reads the message and processes item match profile - searches item in mod-inventory-storage using match criteria from the match profile. Processes item search result and issues an appropriate event (DI_INVENTORY_ITEM_NOT_MATCHED if no item is found, or DI_INVENTORY_ITEM_MATCHED if item found)
20. [41] If item is not found - mod-inventory sends DI_COMPLETED since no other profiles are placed for "non-matches" branch of the Item Match Profile. The processing flow for the particular incoming record is completed. The sent event contains result event indication (DI_INVENTORY_ITEM_NOT_MATCHED ) denoting item matching result
    1. [42] mod-srm receives DI_COMPLETED and updates the import job progress
21. [43] If item is found - mod-inventory sends DI_INVENTORY_ITEM_MATCHED
22. [43-46] mod-inventory reads the message and processes update item action profile - maps item according to mapping profile linked to the update action profile, saves updated item in mod-inventory-storage and sends DI_INVENTORY_ITEM_UPDATED
23. [49] In case of a successful item update mod-inventory sends DI_COMPLETED event
24. [50] mod-srm receives DI_COMPLETED event and marks JobExecution status as "COMMITTED" if all incoming record have been processed without errors, otherwise sets JobExecution status to "ERROR"

Diagram (updated with changes made in Quesnelia R1 2024)

Source

Comments

For not empty "non-match" branch in profile a new flow description needs to be created