3. Modify MARC_BIB
Description
Modify action was designed to be applied on the incoming MARC Bib record before mapping any other entities, to ensure such behavior it should be placed at the beginning of the Job Profile. However, Modify action can also be applied at the end of the Job Profile, following actions on MARC Bib/Instance, in that case Modify will affect existing record that was acted upon in the context of the Job. Following are the supported scenarios using Modify action.
1. Modify Incoming MARC Bib + Create Instance
Flow (Updated with changes made in Quesnelia R1 2024)
- [1-4] UploadDefinition is created for importing files in mod-data-import
- [2-3] JobExecution is created in mod-srm
- [5-7] MARC files are uploaded from WEB client to mod-data-import
- [6] Uploaded file is split into chunks in mod-data-import
- [8-9] User selects the JobProfile and initiates the processing of the uploaded files
- [10-11] Profile ID is set for JobExecution
- [12] MARC records are packed into batches and put to Kafka queue DI_RAW_RECORDS_CHUNK_READ
- [13-15] mod-srm reads batches from the queue, parses MARC records and saves the Incoming records in the DB. JobExecution is set to IN_PROGRESS status on first chunk received.
- [15-16] mod-srm reads the profile and creates JSON payload (containing parsed MARC) for processing. Exports it to an appropriate Kafka queue (one message per MARC entry) - DI_INCOMING_MARC_BIB_RECORD_PARSED
- [17] mod-inventory processes Modify action on incoming MARC Bib and substitutes MARC_BIBLIOGRAPHIC value in the event payload (no existing entities are affected in this case)
- [20] mod-inventory issues DI_SRS_MARC_BIB_RECORD_MODIFIED event
- [20-24] mod-inventory reads the message, MARC Bib value from '001' + '003' is moved to '035', after which the Instance is mapped and created via HTTP in mod-inventory-storage. At this point Instance UUID and Instance HRID are generated.
- [25-27] Instance HRID is set to '001' MARC_BIB field, Instance ID is set to '999 ff i' field and is saved in mod-srs via HTTP
- [30] mod-inventory issues DI_INVENTORY_INSTANCE_CREATED event
- [31] mod-inventory reads message and exports to DI_COMPLETED
- [32] mod-srm receives DI_COMPLETED event, marks JobExecution as COMPLETED
Diagram (Updated with changes made in Quesnelia R1 2024)
2. Create Instance + Modify MARC Bib
Flow
- [1-4] UploadDefinition is created for importing files in mod-data-import
- [2-3] JobExecution is created in mod-srm
- [5-7] MARC files are uploaded from WEB client to mod-data-import
- [6] Uploaded file is split into chunks in mod-data-import
- [8-9] User selects the JobProfile and initiates the processing of the uploaded files
- [10-11] Profile ID is set for JobExecution
- [12] MARC records are packed into batches and put to Kafka queue DI_RAW_RECORDS_CHUNK_READ
- [13-15] mod-srm reads batches from the queue, parses MARC records and saves the Incoming records in the DB. JobExecution is set to IN_PROGRESS status on first chunk received.
- [15-16] mod-srm reads the profile and creates JSON payload (containing parsed MARC) for processing. Exports it to an appropriate Kafka queue (one message per MARC entry) - DI_INCOMING_MARC_BIB_RECORD_PARSED
- [17-20] mod-inventory reads the message, MARC Bib value from '001' + '003' is moved to '035', after which the Instance is mapped and created via HTTP in mod-inventory-storage. At this point Instance UUID and Instance HRID are generated.
- [21-23] Instance HRID is set to '001' MARC_BIB field, Instance ID is set to '999 ff i' field and is saved in mod-srs via HTTP
- [26] mod-inventory issues DI_INVENTORY_INSTANCE_CREATED event
- [27] mod-inventory modifies MARC_BIBLIOGRAPHIC, re-maps INSTANCE that are present in the payload (created as a result of the first action)
- [28-33] mod-inventory updates Instance in mod-inventory-storage and SRS MARC in mod-srm
- [36] mod-inventory issues DI_SRS_MARC_BIB_RECORD_MODIFIED event
- [37] mod-inventory reads message and exports to DI_COMPLETED
- [38] mod-srm receives DI_COMPLETED event, marks JobExecution as COMPLETED
Diagram
3. Modify MARC Bib + Match and Update Instance
Flow
- [1-4] UploadDefinition is created for importing files in mod-data-import
- [2-3] JobExecution is created in mod-srm
- [5-7] MARC files are uploaded from WEB client to mod-data-import
- [6] Uploaded file is split into chunks in mod-data-import
- [8-9] User selects the JobProfile and initiates the processing of the uploaded files
- [10-11] Profile ID is set for JobExecution
- [12] MARC records are packed into batches and put to Kafka queue DI_RAW_RECORDS_CHUNK_READ
- [13-15] mod-srm reads batches from the queue, parses MARC records and saves the Incoming records in the DB. JobExecution is set to IN_PROGRESS status on first chunk received.
- [15-16] mod-srm reads the profile and creates JSON payload (containing parsed MARC) for processing. Exports it to an appropriate Kafka queue (one message per MARC entry) - DI_INCOMING_MARC_BIB_RECORD_PARSED
- [17] mod-inventory processes Modify action on incoming MARC Bib and substitutes MARC_BIBLIOGRAPHIC value in the event payload (no existing entities are affected in this case)
- [20] mod-inventory issues DI_SRS_MARC_BIB_RECORD_MODIFIED event
- [21-22] mod-inventory reads the message, processes Match profile - reads Match criteria and constructs a CQL search query for mod-inventory-storage, sends an HTTP request to get the Instance
- [23] mod-inventory processes search result and issues an appropriate event (either DI_INVENTORY_INSTANCE_NOT_MATCHED if no Instance exists satisfying search criteria, or DI_INVENTORY_INSTANCE_MATCHED if such Instance is found)
- [24-26] In case Instance is not found Job completes for this particular Job Profile (no other actions are placed under a Non-Match branch of the Job Profile)
- [27-33] In case Instance is found - mod-inventory processes Update Instance action. Instance for an update is mapped from the modified version of incoming MARC Bib.
- [37] mod-inventory reads message and exports to DI_COMPLETED
- [38] mod-srm receives DI_COMPLETED event, marks JobExecution as COMPLETED
Diagram
4. Match and Update Instance + Modify MARC Bib
Flow
- [1-4] UploadDefinition is created for importing files in mod-data-import
- [2-3] JobExecution is created in mod-srm
- [5-7] MARC files are uploaded from WEB client to mod-data-import
- [6] Uploaded file is split into chunks in mod-data-import
- [8-9] User selects the JobProfile and initiates the processing of the uploaded files
- [10-11] Profile ID is set for JobExecution
- [12] MARC records are packed into batches and put to Kafka queue DI_RAW_RECORDS_CHUNK_READ
- [13-15] mod-srm reads batches from the queue, parses MARC records and saves the Incoming records in the DB. JobExecution is set to IN_PROGRESS status on first chunk received.
- [15-16] mod-srm reads the profile and creates JSON payload (containing parsed MARC) for processing. Exports it to an appropriate Kafka queue (one message per MARC entry) - DI_INCOMING_MARC_BIB_RECORD_PARSED
- [17-18] mod-inventory reads the message, processes Match profile - reads Match criteria and constructs a CQL search query for mod-inventory-storage, sends an HTTP request to get the Instance
- [19] mod-inventory processes search result and issues an appropriate event (either DI_INVENTORY_INSTANCE_NOT_MATCHED if no Instance exists satisfying search criteria, or DI_INVENTORY_INSTANCE_MATCHED if such Instance is found)
- [20-22] In case Instance is not found Job completes for this particular Job Profile (no other actions are placed under a Non-Match branch of the Job Profile)
- [23-29] In case Instance is found - mod-inventory processes Update Instance action. Instance for an update is mapped from the incoming MARC Bib and Instance is updated in mod-inventory-storage, SRS MARC is updated in mod-srs
- [33] mod-inventory processes Modify action, modifies MARC_BIBLIOGRAPHIC, re-maps INSTANCE that are present in the payload (found and placed in the context as a result of the Match)
- [34-39] mod-inventory updates Instance in mod-inventory-storage and SRS MARC in mod-srs
- [42] mod-inventory issues DI_SRS_MARC_BIB_RECORD_MODIFIED event
- [43] mod-inventory reads message and exports to DI_COMPLETED
- [44] mod-srm receives DI_COMPLETED event, marks JobExecution as COMPLETED