MODQM-128 - Getting issue details... STATUS

Problem and current flow description

The current implementation of quickMARC update MARC record flow is described in the sequence diagram: Open Diagram

Short description

1) Upon receiving the request for an update of the MARC record in SRM, a new QM_RECORD_UPDATED event is formed and sent to the Kafka queue. Event payload contains the following data

• PARSED_RECORD_DTO - received ParsedRecordDto containing updated MARC record
• MAPPING_RULES - MARC-to-Instance default mapping rules
• MAPPING_PARAMS - mapping parameters for populating reference data
• SNAPSHOT_ID (optional) - new Snapshot id to which the updated MARC record will be linked

2) SRS is subscribed to QM_RECORD_UPDATED eventType. When SRS receives the event the following operations are performed:  

1. a new Snapshot is created 
2. a new Record is created with incremented generation value and state ACTUAL
3. state of the original Record is set to OLD

3) After the Record is successfully “updated” (records in SRS are not overwritten, new Record is being saved with the higher generation, and all the others are marked as OLD), a new QM_SRS_MARC_BIB_RECORD_UPDATED event is published to Kafka queue with payload:

• MAPPING_RULES (received from SRM) - MARC-to-Instance default mapping rules
• MAPPING_PARAMS (received from SRM)- mapping parameters for populating reference data
• MARC - updated SRS MARC Record

4) mod-inventory is subscribed to QM_SRS_MARC_BIB_RECORD_UPDATED event. When the event is received, an Instance entity is mapped from the MARC record using Mapper from the data-import-processing-core library. The original Instance is retrieved from the mod-inventory-storage, compared to the newly mapped one, and updated accordingly.

Current implementation problems

1. When UI fetches updated Inventory Instance there could be a case when it is still not updated.
2. It is not very fast, it could take 5 and more seconds to update all related records.
3. If some exceptional case happened in SRM, SRS, Inventory, or Inventory Storage happened the error message not recorded.

Possible solutions

Request Record update status (same as derive flow)

Changed flow is described in the sequence diagram: Open Diagram

Short description

1. Upon receiving the request for an update in quickMarc save to database Record ID and status "In progress"
2. Send a request to SRM to update the record.
3. Listen to QM_INVENTORY_INSTANCE_UPDATED and QM_ERROR Kafka topics and update status in the quickMARC database to "Completed" or "Failed" accordingly.
4. After UI received a response, it should repeatable call updated `GET /records/status` endpoint.
5. When received status is changed to "Completed" or "Failed" then redirect to the Instance record page.

Asynchronous quickMARC response with DeferredResult

Changed flow is described in the sequence diagram: Open Diagram

Short description

1. Create a new Kafka topic QM_COMPETED that SRM should send with the payload contains a Record ID, an indicator of a successful update, and an error message if failed update.
2. Wrap the result of the endpoint in DeferredResult and save it to cache with Record ID as a key.
3. Listen for QM_COMPETED in quickMARC. When received get DeferredResult from the cache and set the result that will depend on the indicator. If the update was successful then return 202 status, if not 400, and received message.

Synchronous Request-Reply using Apache Kafka

Changed flow is described in the sequence diagram: Open Diagram

Short description

1. Create a new Kafka topic QM_UPDATE_RECORD that quickMARC send to subscribed SRM with payload that contains ParsedRecordDto and reply topic header.
2. Create a new Kafka topic QM_COMPETED that SRM should send with the payload contains a Record ID, an indicator of a successful update, and an error message if failed update.
3. The quickMARC wait until reply received and then respond.

Improvements that could decrease time of processing update.

1. Decrease amount of HTTP request to mod-inventory-storage. Possible solutions:
   1. Update `PUT /instance-storage/instances/{instanceId}` endpoint to update Preceding/Succeeding titles together with Instance.
   2. Create `PUT /preceding-succeeding-titles` endpoint to replace old Preceding/Succeeding titles with new one.
2. Make processing in SRS and Inventory concurrent. Probable solution described at diagram: Open Diagram
3. Check implementation of producers in SRS and SRM. Currently they are created each time the event send and then closed. 

Selected for development

1. Asynchronous quickMARC response with DeferredResult
2. Make changes to producers in SRS and SRM not to be closed after sending event
3. Changes in instance update in mod-inventory