| Jira Legacy | ||||||||||
|---|---|---|---|---|---|---|---|---|---|---|
|
Feature:
| Jira Legacy | ||||||
|---|---|---|---|---|---|---|
|
Data Import Subgroup Requirements
Overview
Now the data-import is able to import Marc authority records, mod-inventory can generate Folio records, mod-search provides API for the search. In this story, we need to understand how to implement the Update for Marc authorities in data-import, what changes are needed to support the Search of the updated records via mod-search.
...
| Step№ | Module | Task | DI Event | ||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|
| 1 | mod-data-import | Sends a batch of records to Kafka queue | DI_RAW_RECORDS_CHUNK_READ | ||||||||
| 2 | mod-source-record-manager | Receives records, parses to the JSON, sends it to Kafka queue | DI_RAW_RECORDS_CHUNK_PARSED | ||||||||
| 3 | mod-source-record-storage | Receives the event from SRM, Stores Authority records into the database, and returns the result back via Kafka queue. The SRS now can match records only by 001 and 999 ff $s, this logic is hard to expand. Matching 010 $a or other tags requires implementing a spike | DI_PARSED_RECORDS_CHUNK_SAVED | ||||||||
| 4 | mod-source-record-manager | Receives the event from SRS, creates JSON payload (containing parsed MARC, profile, mapping parameters) for further processing, and sends it to the queue | DI_SRS_MARC_AUTHORITY_RECORD_CREATED | ||||||||
| 5 | mod-inventory | Receives the event from SRM, loads the Folio record that matches Marc tags, creates JSON payload with a matched Folio record, sends it to the queue. We have to implement AuthorityLoader on this step. The loader retrieves the Folio record from mod-inventory-storage that matches Marc tags 001, 010 $a, 999 ff $s , or use Folio record id | DI_INVENTORY_AUTHORITY_MATCHED DI_INVENTORY_AUTHORITY_NOT_MATCHED | ||||||||
| 6 | mod-inventory | Receives the event from itself. Updates the existing Folio record. We have to implement UpdateAuthorityEventHandler on this step. The handler updates the existing Folio record by the matched record taken from the event payload | DI_INVENTORY_AUTHORITY_RECORD_UPDATED DI_COMPLETED | ||||||||
| 7 | mod-inventory-storage | Receives HTTP PUT request with the record that needs to be updated, updates the record in the database, puts record to Kafka topic | |||||||||
| 8 | mod-search | Receives a record from Kafka topic on an update, deletes the existing record matching by id, inserts the new record |
...
| Module | Description | Jira | High-level rough est | ||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|
| mod-source-record-storage | Spike to extend matching |
| 5 | ||||||||
| mod-data-import-converter-storage | Story to create default Action profile & Matching profiles | To be defined later | To be defined later | ||||||||
| data-import-processing-core | Story to add new event types for the Update |
| 1 | ||||||||
| mod-inventory | Story to implement the AuthorityLoader |
| 3 | ||||||||
| mod-inventory | Story to implement the UpdateAuthorityEventHandler |
| 5 | ||||||||
| mod-source-record-manager | Log the non-matched records update DataImportJournalConsumersVerticle and JournalParam |
| 2 | ||||||||
| mod-source-record-storage | Set matched_id, external_id, and parsedRecord data |
| 5 |
mod-source-record-storage: Update matching criteria for authorities to update the 'generation' field (If not enough to use only matchId)
mod-source-record-storage: Update externalId, parsedContent, externalId when the record is matched (Get externalId from the existing record and Set it to the incoming auth record), update Parsed content
support 010 field
ui-data-import: Update Matching profiles to enable Authorities
ui-plugin-find-import-profile: https://github.com/folio-org/ui-plugin-find-import-profile does it require some work ???