Versions Compared

Version Old Version 9 New Version 10
Changes made by

Kateryna Senchenko

Kateryna Senchenko

Saved on

Key

  • This line was added.
  • This line was removed.
  • Formatting was changed.

...

1. Modify Incoming MARC Bib + Create Instance

Info
iconfalse
NB

...

In this case Create Instance action can be followed by Create Holdings and Create Item actions. They've been omitted for brevity. 

Flow (Updated with changes made in Quesnelia R1 2024)

...

2. Create Instance + Modify MARC Bib 

Info
iconfalse
NB: In this case Create Instance action can also be followed by Create Holdings and Create Item actions. They've been omitted for brevity. 

Flow

  1. [1-4] UploadDefinition is created for importing files in mod-data-import
  2. [2-3] JobExecution is created in mod-srm
  3. [5-7] MARC files are uploaded from WEB client to mod-data-import
  4. [6] Uploaded file is split into chunks in mod-data-import
  5. [8-9] User selects the JobProfile and initiates the processing of the uploaded files
  6. [10-11] Profile ID is set for JobExecution
  7. [12] MARC records are packed into batches and put to Kafka queue DI_RAW_RECORDS_CHUNK_READ
  8. [13-15] mod-srm reads batches from the queue, parses MARC records and saves the Incoming records in the DB. JobExecution is set to IN_PROGRESS status on first chunk received.
  9. [15-16] mod-srm reads the profile and creates JSON payload (containing parsed MARC) for processing. Exports it to an appropriate Kafka queue (one message per MARC entry) - DI_INCOMING_MARC_BIB_RECORD_PARSED
  10. [17-20] mod-inventory reads the message, MARC Bib value from '001' + '003' is moved to '035', after which the Instance is mapped and created  via HTTP in mod-inventory-storage. At this point Instance UUID and Instance HRID are generated. 
  11. [21-23] Instance HRID is set to '001' MARC_BIB field, Instance ID is set to '999 ff i' field and is saved in mod-srs via HTTP
  12. [26] mod-inventory issues DI_INVENTORY_INSTANCE_CREATED event 
  13. [27] mod-inventory modifies MARC_BIBLIOGRAPHIC, re-maps INSTANCE that are present in the payload (created as a result of the first action)
  14. [28-33] mod-inventory updates Instance in mod-inventory-storage and SRS MARC in mod-srm
  15. [36] mod-inventory issues DI_SRS_MARC_BIB_RECORD_MODIFIED event
  16. [37] mod-inventory reads message and exports to DI_COMPLETED
  17. [38] mod-srm receives DI_COMPLETED event, marks JobExecution as COMPLETED

Diagram

Source


3. Modify MARC Bib + Match and Update Instance

Image Added

Info
iconfalse
NB: In this case any Match can be used (including MARC-to-MARC Match), and any allowed actions placed under the Match.

Flow

  1. [1-4] UploadDefinition is created for importing files in mod-data-import
  2. [2-3] JobExecution is created in mod-srm
  3. [5-7] MARC files are uploaded from WEB client to mod-data-import
  4. [6] Uploaded file is split into chunks in mod-data-import
  5. [8-9] User selects the JobProfile and initiates the processing of the uploaded files
  6. [10-11] Profile ID is set for JobExecution
  7. [12] MARC records are packed into batches and put to Kafka queue DI_RAW_RECORDS_CHUNK_READ
  8. [13-15] mod-srm reads batches from the queue, parses MARC records and saves the Incoming records in the DB. JobExecution is set to IN_PROGRESS status on first chunk received.
  9. [15-16] mod-srm reads the profile and creates JSON payload (containing parsed MARC) for processing. Exports it to an appropriate Kafka queue (one message per MARC entry) - DI_INCOMING_MARC_BIB_RECORD_PARSED
  10. [17] mod-inventory processes Modify action on incoming MARC Bib and substitutes MARC_BIBLIOGRAPHIC value in the event payload (no existing entities are affected in this case)
  11. [20] mod-inventory issues DI_SRS_MARC_BIB_RECORD_MODIFIED event
  12. [21-22] mod-inventory reads the message, processes Match profile - reads Match criteria and constructs a CQL search query for mod-inventory-storage, sends an HTTP request to get the Instance
  13. [23] mod-inventory processes search result and issues an appropriate event (either DI_INVENTORY_INSTANCE_NOT_MATCHED if no Instance exists satisfying search criteria, or DI_INVENTORY_INSTANCE_MATCHED if such Instance is found)
  14. [24-26] In case Instance is not found Job completes for this particular Job Profile (no other actions are placed under a Non-Match branch of the Job Profile)
  15. [27-33] In case Instance is found - mod-inventory processes Update Instance action. Instance for an update is mapped from the modified version of incoming MARC Bib.
  16. [37] mod-inventory reads message and exports to DI_COMPLETED
  17. [38] mod-srm receives DI_COMPLETED event, marks JobExecution as COMPLETED

Diagram

Source

Image Added


4. Match and Update Instance + Modify MARC Bib

Image AddedImage Added

Info
iconfalse
NB: In this particular case Modify can also be placed either inside the Match block following other actions, or outside the Match block. Modify will be processed in the same way. 


Info
iconfalse
NB: Any Match profile and any allowed Actions can be used before the Modify action, however Modify at the end will be processed only if there was an explicit or implicit action on the Instance.

Flow

  1. [1-4] UploadDefinition is created for importing files in mod-data-import
  2. [2-3] JobExecution is created in mod-srm
  3. [5-7] MARC files are uploaded from WEB client to mod-data-import
  4. [6] Uploaded file is split into chunks in mod-data-import
  5. [8-9] User selects the JobProfile and initiates the processing of the uploaded files
  6. [10-11] Profile ID is set for JobExecution
  7. [12] MARC records are packed into batches and put to Kafka queue DI_RAW_RECORDS_CHUNK_READ
  8. [13-15] mod-srm reads batches from the queue, parses MARC records and saves the Incoming records in the DB. JobExecution is set to IN_PROGRESS status on first chunk received.
  9. [15-16] mod-srm reads the profile and creates JSON payload (containing parsed MARC) for processing. Exports it to an appropriate Kafka queue (one message per MARC entry) - DI_INCOMING_MARC_BIB_RECORD_PARSED
  10. [17-18] mod-inventory reads the message, processes Match profile - reads Match criteria and constructs a CQL search query for mod-inventory-storage, sends an HTTP request to get the Instance
  11. [19] mod-inventory processes search result and issues an appropriate event (either DI_INVENTORY_INSTANCE_NOT_MATCHED if no Instance exists satisfying search criteria, or DI_INVENTORY_INSTANCE_MATCHED if such Instance is found)
  12. [20-22] In case Instance is not found Job completes for this particular Job Profile (no other actions are placed under a Non-Match branch of the Job Profile)
  13. [23-29] In case Instance is found - mod-inventory processes Update Instance action. Instance for an update is mapped from the incoming MARC Bib and Instance is updated in mod-inventory-storage, SRS MARC is updated in mod-srs
  14. [33] mod-inventory processes Modify action, modifies MARC_BIBLIOGRAPHIC, re-maps INSTANCE that are present in the payload (found and placed in the context as a result of the Match)
  15. [34-39] mod-inventory updates Instance in mod-inventory-storage and SRS MARC in mod-srs
  16. [42] mod-inventory issues DI_SRS_MARC_BIB_RECORD_MODIFIED event
  17. [43] mod-inventory reads message and exports to DI_COMPLETED
  18. [44] mod-srm receives DI_COMPLETED event, marks JobExecution as COMPLETED

Diagram

Source

Image Added