SPIKE: Update MARC authority records via Data import
Feature: 
Data Import Subgroup Requirements
Overview
Now the data-import is able to import Marc authority records, mod-inventory can generate Folio records, mod-search provides API for the search. In this story, we need to understand how to implement the Update for Marc authorities in data-import, what changes are needed to support the Search of the updated records via mod-search.
The Update is an action that is specified by the Action profile, so anytime the user needs to choose the Action profile if there is a need to update records. The Matching profile defines the types of incoming records and existing records, that need to be updated. Also, a Match profile contains the Match criteria, which specifies parameters for records comparison.
Reference page: Update SRS MARC Authority records via Data Import
Approach
MARC Authority update workflow in data-import
The user uploads the MARC file, chooses an Action profile & Matching profile on the UI application, starts the import process
Step№ | Module | Task | DI Event |
|---|---|---|---|
1 | mod-data-import | Sends a batch of records to Kafka queue | DI_RAW_RECORDS_CHUNK_READ |
2 | mod-source-record-manager | Receives records, parses to the JSON, sends it to Kafka queue | DI_RAW_RECORDS_CHUNK_PARSED |
3 | mod-source-record-storage | Receives the event from SRM, Stores Authority records into the database, and returns the result back via Kafka queue. The SRS now can match records only by 001 and 999 ff $s, this logic is hard to expand. Matching 010 $a or other tags requires implementing a spike | DI_PARSED_RECORDS_CHUNK_SAVED |
4 | mod-source-record-manager | Receives the event from SRS, creates JSON payload (containing parsed MARC, profile, mapping parameters) for further processing, and sends it to the queue | DI_SRS_MARC_AUTHORITY_RECORD_CREATED |
5 | mod-inventory | Receives the event from SRM, loads the Folio record that matches Marc tags, creates JSON payload with a matched Folio record, sends it to the queue. We have to implement AuthorityLoader on this step. The loader retrieves the Folio record from mod-inventory-storage that matches Marc tags 001, 010 $a, 999 ff $s , or use Folio record id | DI_INVENTORY_AUTHORITY_MATCHED |
6 | mod-inventory | Receives the event from itself. Updates the existing Folio record. We have to implement UpdateAuthorityEventHandler on this step. The handler updates the existing Folio record by the matched record taken from the event payload | DI_INVENTORY_AUTHORITY_RECORD_UPDATED DI_COMPLETED |
7 | mod-inventory-storage | Receives HTTP PUT request with the record that needs to be updated, updates the record in the database, puts record to Kafka topic | |
8 | mod-search | Receives a record from Kafka topic on an update, deletes the existing record matching by id, inserts the new record |
Field protection is not needed at the beginning
User stories
Module | Description | Jira | High-level rough est |
|---|---|---|---|
mod-source-record-storage | Spike to extend matching |
| 5 |
mod-data-import-converter-storage | Story to create default Action profile & Matching profiles | To be defined later | To be defined later |
data-import-processing-core | Story to add new event types for the Update | 1 | |
mod-inventory | Story to implement the AuthorityLoader |
| 3 |
mod-inventory | Story to implement the UpdateAuthorityEventHandler |
| 5 |
mod-source-record-manager | Log the non-matched records |
| 2 |
mod-source-record-storage | Set matched_id, external_id, and parsedRecord data |
| 5 |
mod-source-record-storage: Update matching criteria for authorities to support 010 field
ui-data-import: Update Matching profiles to enable Authorities
ui-plugin-find-import-profile: https://github.com/folio-org/ui-plugin-find-import-profile does it require some work ???