/
MODQM-68 Spike: Support duplication of a MARC bib record

MODQM-68 Spike: Support duplication of a MARC bib record

Jul 06, 2021

In Progress

Goal: 

Investigate the existing solution and provide a design for duplication of existing MARC record.

What do we currently have:

'mod-quick-marc' is used as a proxy between User and 'mod-source-record-manager' (which inside use 'mod-source-record-storage' and 'mod-inventory') and can work only with existing records.

Lest have a look on 'get-by-id' flow

From the picture above we can see that 'mod-quick-marc' and 'mod-source-record-manager' operate with QuickMarcJson ↔ ParsedRecordDto models. Communication takes place thought

"source-manager-parsed-records" interface of 'mod-source-record-manager'

 

"source-manager-parsed-records"

GET

/change-manager/parsedRecords

PUT

/change-manager/parsedRecords/{id}

 

On the other side Folio has an ability to create a new Record inside 'mod-source-record-storage' and Instance in 'mod-inventory' using 'mod-data-import' module.

'mod-source-record-manager' currently supports two ways to import MARC records : event-driven and REST approaches. As module will be moving to event-driven model we will focus on it.

 

 

Some parts of the process such as post-process-after-instance-creation, Journal updates, JobExecution status updates and initial call of import are omitted in the digram to simplify process. From the diagram it is clearly understandable that process of creation Records and Instances is quite complex and we can use some parts of that flow.  

'mod-source-record-manager' works with RawRecordDto model which then converted into Record and then start the flow of creation.

Record model has several required fields to be filled:

Field name

Description

Field name

Description

snapshotId

a UUID of the executionJob

matchedId

a UUID of a record

recordType

MARC

rawRecord

raw content of the record

So, to be able to use flow for importing we need to provide a Record for 'mod-source-record-manager'.

Concerns

  • 'mod-quick-marc' works with ParsedRecordDto but 'mod-source-manager' needs to have Record model.

One of the required field of Record is 'rawRecord'. So, there is need to convert ParsedRecordDto to RawRecord or get the existing entity from 'mod-source-record-storage' module.

  • Cloning process will not create any JobExecution, so there is no id that might be used as 'snapshotId' for Record model.

Next required field of Record is 'snapshotId'. 'mod-source-record-storage' may contain a STUB_SNAPSHOT entry that might be used. If not - Snapshot  should be created 

URL

Note

URL

Note

GET /source-storage/snapshots/{jobExecutionId}

where {jobExecutionId} = "00000000-0000-0000-0000-000000000000"

POST /source-storage/snapshots

sample body
{
"jobExecutionId": "00000000-0000-0000-0000-000000000000",
"status": "COMMITTED",
"processingStartedDate" : "<some_date>"

}

Another (simplified) solution:

There is a simple solution exist to delegate whole work of creating a new Record and Instance based on 'mod-source-record-manager' at README file.

According to it the clone record flow will be simplified to:

Based on the diagram we can divide flow to 3 stages:

  1. prepare entry on 'mod-quick-marc' side   

  2. delegate creation of a new Instance and Record to 'mod-source-record-manager'

  3. get user notified about new record

 

1.Prepare entry on 'mod-quick-marc' side

Module 'mod-quick-marc' has to support a new endpoint to accept a QuickMarcJson payload. Here is the endpoint definition: 

Method

URL

Permissions

Payload

Method

URL

Permissions

Payload

POST

/records-editor/records

endpoint permission - 

"records-editor.records.item.clone"

module permissions -

"source-manager-job-executions",
"source-manager-records"

QuickMarcJson

 

2.Delegate creation of a new Instance and Record to 'mod-source-record-manager'

'mod-source-record-manager' provides endpoints to initialize import process. On this step we completely rely on execution of following three endpoints:

 

Method

URL

Body

Checked

 

Method

URL

Body

Checked

1

POST

/change-manager/jobExecutions

{ "files": [], "sourceType": "ONLINE", "jobProfileInfo": { "id": "c8f98545-898c-4f48-a494-3ab6736a3243", "name": "Default job profile", "dataType": "MARC" }, "userId": "952d9764-c0a9-5d81-9c2e-cd93c2600990" }

2

POST

/change-manager/jobExecutions/{jobExecutionId}/records?defaultMapping=true

{
   "recordsMetadata": {
   "last": false,
   "counter": 1,
   "total": 1,
   "contentType":"MARC_JSON"
},
"initialRecords": [{
   "record": "{\"leader\": \"00648cam a2200193 a 4500\",\r\n \"fields\": [\r\n {\r\n \"008\": \"960521s1972\\\\\\\\\\\\\\\\se\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\0\\\\\\\\\\\\swe\\\\\\\\\"\r\n },\r\n {\r\n \"041\": {\r\n \"ind1\": \"1\",\r\n \"ind2\": \"\\\\\",\r\n \"subfields\": [\r\n {\r\n \"a\": \"swe\"\r\n }\r\n ]\r\n }\r\n },\r\n {\r\n \"096\": {\r\n \"ind1\": \"\\\\\",\r\n \"ind2\": \"\\\\\",\r\n \"subfields\": [\r\n {\r\n \"y\": \"Z\"\r\n },\r\n {\r\n \"b\": \"TAp Chalmers tekniska h\u00F6gskola. Inst. f\u00F6r byggnadsstatik. Skrift. 1972:4\"\r\n },\r\n {\r\n \"s\": \"g\"\r\n }\r\n ]\r\n }\r\n },\r\n {\r\n \"100\": {\r\n \"ind1\": \"1\",\r\n \"ind2\": \"\\\\\",\r\n \"subfields\": [\r\n {\r\n \"a\": \"Sahlin, Sven\"\r\n }\r\n ]\r\n }\r\n },\r\n {\r\n \"245\": {\r\n \"ind1\": \"0\",\r\n \"ind2\": \"0\",\r\n \"subfields\": [\r\n {\r\n \"a\": \"P\u00E5lslagning\"\r\n }\r\n ]\r\n }\r\n },\r\n {\r\n \"260\": {\r\n \"ind1\": \"\\\\\",\r\n \"ind2\": \"\\\\\",\r\n \"subfields\": [\r\n {\r\n \"c\": \"1972\"\r\n }\r\n ]\r\n }\r\n },\r\n {\r\n \"300\": {\r\n \"ind1\": \"\\\\\",\r\n \"ind2\": \"\\\\\",\r\n \"subfields\": [\r\n {\r\n \"a\": \"19 bl.\"\r\n }\r\n ]\r\n }\r\n },\r\n {\r\n \"440\": {\r\n \"ind1\": \"\\\\\",\r\n \"ind2\": \"\\\\\",\r\n \"subfields\": [\r\n {\r\n \"a\": \"Skrift, Chalmers tekniska h\u00F6gskola, Institutionen f\u00F6r byggnadsstatik\"\r\n },\r\n {\r\n \"x\": \"9903909802 ;\"\r\n },\r\n {\r\n \"v\": \"72:4\"\r\n }\r\n ]\r\n }\r\n },\r\n {\r\n \"907\": {\r\n \"ind1\": \"\\\\\",\r\n \"ind2\": \"\\\\\",\r\n \"subfields\": [\r\n {\r\n \"a\": \".b11154585\"\r\n },\r\n {\r\n \"b\": \"hbib \"\r\n },\r\n {\r\n \"c\": \"s\"\r\n }\r\n ]\r\n }\r\n },\r\n {\r\n \"902\": {\r\n \"ind1\": \"\\\\\",\r\n \"ind2\": \"\\\\\",\r\n \"subfields\": [\r\n {\r\n \"a\": \"190206\"\r\n }\r\n ]\r\n }\r\n },\r\n {\r\n \"998\": {\r\n \"ind1\": \"\\\\\",\r\n \"ind2\": \"\\\\\",\r\n \"subfields\": [\r\n {\r\n \"b\": \"0\"\r\n },\r\n {\r\n \"c\": \"990511\"\r\n },\r\n {\r\n \"d\": \"m\"\r\n },\r\n {\r\n \"e\": \"b \"\r\n },\r\n {\r\n \"f\": \"s\"\r\n },\r\n {\r\n \"g\": \"0\"\r\n }\r\n ]\r\n }\r\n },\r\n {\r\n \"909\": {\r\n \"ind1\": \"0\",\r\n \"ind2\": \"0\",\r\n \"subfields\": [\r\n {\r\n \"a\": \"m\"\r\n },\r\n {\r\n \"c\": \"a\"\r\n },\r\n {\r\n \"d\": \"b\"\r\n }\r\n ]\r\n }\r\n },\r\n {\r\n \"945\": {\r\n \"ind1\": \"\\\\\",\r\n \"ind2\": \"\\\\\",\r\n \"subfields\": [\r\n {\r\n \"l\": \"hbib3\"\r\n },\r\n {\r\n \"a\": \"TAp Chalmers tekniska h\u00F6gskola.Inst. f\u00F6r byggnadsstatik. Skrift 72:4\"\r\n }\r\n ]\r\n }\r\n }\r\n ]\r\n }"
   }]
}

 

 

 

 3

POST

/change-manager/jobExecutions/{jobExecutionId}/records?defaultMapping=true

{ "recordsMetadata": { "last": true, "counter": 3, "total": 3, "contentType":"MARC_RAW" }, "initialRecords": [] }

 

3.Get user notified about new record

To provide an ability for User to be notified about the creation of the Instance 'mod-quick-marc' should, one the one hand 

  • provide an endpoint to get record status information

and on the other hand -

  • be able to consume an event from 'mod-source-record-manager' that process of Instance and Record creation has beed finished.

For this being said, 'mod-quick-marc' has to publish an endpoint with following description:

 

Method

URL

Permissions

Method

URL

Permissions

GET 

/records-editor/records/status?qmRecordId=<id>

"records-editor.records.status.item.get"

with the sample of the payload

Sample payload

Notes

Sample payload

Notes

{
 "qmRecordId": "sample_id",
   "instanceId": "00000000-0000-0000-0000-000000000001",
   "status": "NEW",
   "jobExecutionId": "00000000-0000-0000-0000-000000000000",
   "metadata": {
        "createdDate": "2020-01-27T12:01:03.696+0000",
        "createdByUserId": "00000000-0000-0000-0000-000000000002",
        "updatedDate": "2020-01-27T12:10:25.887+0000",
        "updatedByUserId": "00000000-0000-0000-0000-000000000002"
    }
}

Proposed values for the status:

  • NEW 

  • IN_PROGRESS

  • CREATED

  • ERROR

'mod-quick-marc' has to 

Stories needed:

 

 

 

 

'mod-quick-marc' create endpoint

https://folio-org.atlassian.net/browse/MODQM-77

'mod-quick-marc' to 'mod-source-record-manager' integration

https://folio-org.atlassian.net/browse/MODQM-80

user notification about created entries

https://folio-org.atlassian.net/browse/MODQM-78https://folio-org.atlassian.net/browse/MODQM-79

Additional information:

  • 'mod-quick-marc' get by id schema -

  • Duplicate (simplified) solution -