[UIIN-204] Create element for link to source record in blob storage, part 1 Created: 27/Jun/18  Updated: 03/Nov/23  Resolved: 25/Sep/18

Status: Closed
Project: ui-inventory
Components: None
Affects versions: None
Fix versions: None

Type: Story Priority: P3
Reporter: Charlotte Whitt Assignee: Niels Erik Nielsen
Resolution: Done Votes: 0
Labels: inventory, metadatamanagement
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original estimate: Not Specified

Attachments: PNG File NEW Inventory - Storage - MARCCat (1).png     PNG File Skärmavbild 2018-05-25 kl. 10.00.10.png     PNG File Skärmavbild 2018-09-06 kl. 14.41.42.png     PNG File Skärmavbild 2018-09-06 kl. 17.42.13.png     PNG File Skärmavbild 2018-09-25 kl. 11.43.31.png     PNG File Skärmavbild 2018-09-25 kl. 11.44.24.png     PNG File V1 Inventory and MARCCat Integration Ideas (4).png    
Issue links:
Cloners
clones UXPROD-125 Define link to source record Closed
Defines
defines UXPROD-1577 Make Inventory Instance rely on SRS M... Closed
Relates
relates to MODINVSTOR-143 provide sample data for source marc s... Closed
relates to UIIN-93 Display of rich source record stored ... Closed
relates to UIIN-203 Implement storage of batch loaded MAR... Closed
relates to UXPROD-126 Link an existing metadata record to a... Closed
Sprint:
Development Team: Other dev
Tester Assignee: Charlotte Whitt
Analysis Estimator: Charlotte Whitt
Back End Estimator: Niels Erik Nielsen
Front End Estimator: Niels Erik Nielsen

 Description   

Purpose: Create element in Inventory, Instance record for link to given source record in blob storage (link to source record = the original (rich) native metadata format which have been used to create the FOLIO Instance record).

In the instance record the element Metadata source is implemented (in alpha). This element is auto-populated with source of the Generic bib storage record and its format, if applicable (e.g. "FOLIO", "MARC" or "EPKB").

We need to create a new element to store the identifier for the record in the blob storage - e.g. `sourceRecordId`. This element will store e.g. the MARC blob ID. This will be populated by the system.

  1. Scenario:
    • Log in to FOLIO Snapshot stable, go to Inventory
    • Go to any given of the 11 test instance records ( MODINVSTOR-143 Closed )
    • Click on the button 'View source record'
    • Then the user will get the original (rich) native metadata format stored in the Generic Bib Storage
  2. Scenario: Moved to UIIN-326 Closed
    • If the given instance record is converted from a MARC record (imported as part of the library's migration of data to FOLIO, imported using the MARC Batch loader, or created and maintained in the MARCcat app) it will in the Instance record, the Administrative data accordion show the data element 'Metadata source' labeled MARC
  3. Scenario: Moved to UIIN-326 Closed
    • When the given instance record in accordion named Administrative data has 'Metadata source' labeled MARC (or other source types as of post-V1) then this is system generated data, and not edible by the user.


 Comments   
Comment by Marc Johnson [ 06/Sep/18 ]

Charlotte Whitt Thanks for the screenshot.

Cate Boerema Charlotte Whitt Jakub Skoczen Niels Erik Nielsen

I've had a discussion with Wayne Schneider and done some investigation. I believe there are a few contributing factors to the observed behaviour.

It looks like this field was put in as a placeholder (so it may well be that this issue was never worked on directly, the first scenario was likely covered by a different change).

My understanding of the current situation

We only have 11 instance records in our hosted environments at the moment (from Julian, see MODINVSTOR-143 Closed )

The UI presents TBA for all records (see code below, in case my interpretation is incorrect)

There are two properties in the API for source information - source which is required (and can be set to anything we want, typically Sample or MODS Ingest at the moment) and sourceRecordFormat which is set to MARC-JSON automatically when a source record is created for an instance, otherwise it is blank.

What could we do to resolve?

What do we want the Metadata source field to show when there is a source record available (which would be display for the 11 records we have at the moment)?

Scenario two suggests we want that to be MARC.

What do we want the Metadata source field to show when there isn’t a source record (which would display for every other record at the moment)?

To complete this work, we likely want to replace that placeholder with some logic based upon one of the source properties in the instance record.

Which one do you want to use?

If we use the sourceRecordFormat then we need to translate MARC-JSON to MARC in the UI

If we use source then we can set this to be MARC for the 11 records we have. As it is required, we would need something to put in this property. It could be blank for the moment?

Later on, we may want to deprecate the source property or change the way it is used.

Additional Notes

Display of TBA in the interface

<KeyValue label={formatMsg({ id: 'ui-inventory.metadataSource' })} value={formatMsg({ id: 'ui-inventory.tba' })} />

Representation of the Age of TV heroes instance record

{
  "@context" : "http://folio-snapshot-355.aws.indexdata.com:9130/inventory/instances/context",
  "id" : "319b768b-9eaf-489c-b4d6-62878e818324",
  "source" : "Local: MODS",
  "title" : "Age of TV heroes",
  "alternativeTitles" : [ ],
  "series" : [ ],
  "identifiers" : [ {
    "identifierTypeId" : "8261054f-be78-422d-bd51-4ed9f33c3422",
    "value" : "16307972"
  }, {
    "identifierTypeId" : "8261054f-be78-422d-bd51-4ed9f33c3422",
    "value" : "9781605490106"
  }, {
    "identifierTypeId" : "8261054f-be78-422d-bd51-4ed9f33c3422",
    "value" : "1605490105"
  }, {
    "identifierTypeId" : "8261054f-be78-422d-bd51-4ed9f33c3422",
    "value" : "2010280610"
  }, {
    "identifierTypeId" : "8261054f-be78-422d-bd51-4ed9f33c3422",
    "value" : "ocn213447472"
  } ],
  "contributors" : [ {
    "contributorNameTypeId" : "2b94c631-fca9-4892-a730-03ee529ffe2a",
    "name" : "Hofius, Jason.",
    "contributorTypeId" : "",
    "contributorTypeText" : ""
  }, {
    "contributorNameTypeId" : "2b94c631-fca9-4892-a730-03ee529ffe2a",
    "name" : "Khoury, George (George A.)",
    "contributorTypeId" : "",
    "contributorTypeText" : ""
  }, {
    "contributorNameTypeId" : "2b94c631-fca9-4892-a730-03ee529ffe2a",
    "name" : "Ross, Alex",
    "contributorTypeId" : "",
    "contributorTypeText" : ""
  } ],
  "subjects" : [ ],
  "classifications" : [ ],
  "publication" : [ ],
  "urls" : [ ],
  "instanceTypeId" : "6312d172-f0cf-40f6-b27d-9fa8feaf332f",
  "physicalDescriptions" : [ ],
  "languages" : [ ],
  "notes" : [ ],
  "metadata" : {
    "createdDate" : "2018-09-05T03:39:56.611+0000",
    "createdByUserId" : "67b700fb-e1a9-5ed6-aae6-35b6a4df7b00",
    "updatedDate" : "2018-09-05T03:39:56.611+0000",
    "updatedByUserId" : "67b700fb-e1a9-5ed6-aae6-35b6a4df7b00"
  },
  "links" : {
    "self" : "http://folio-snapshot-355.aws.indexdata.com:9130/inventory/instances/319b768b-9eaf-489c-b4d6-62878e818324"
  }
}

Example Instance Record with Source Record

{
  "@context" : "http://folio-snapshot-355.aws.indexdata.com:9130/inventory/instances/context",
  "id" : "8be05cf5-fb4f-4752-8094-8e179d08fb99",
  "source" : "Sample",
  "title" : "Anglo-Saxon manuscripts in microfiche facsimile Volume 25 Corpus Christi College, Cambridge II, MSS 12, 144, 162, 178, 188, 198, 265, 285, 322, 326, 449 microform A. N. Doane (editor and director), Matthew T. Hussey (associate editor), Phillip Pulsiano (founding editor)",
  "alternativeTitles" : [ ],
  "series" : [ "Medieval and Renaissance Texts and Studies volume 497", "volume 497" ],
  "identifiers" : [ {
    "identifierTypeId" : "5d164f4b-0b15-4e42-ae75-cfcf85318ad9",
    "value" : "880391235"
  }, {
    "identifierTypeId" : "8261054f-be78-422d-bd51-4ed9f33c3422",
    "value" : "9780866989732"
  }, {
    "identifierTypeId" : "8261054f-be78-422d-bd51-4ed9f33c3422",
    "value" : "0866989730"
  }, {
    "identifierTypeId" : "8261054f-be78-422d-bd51-4ed9f33c3422",
    "value" : "9780866985529"
  }, {
    "identifierTypeId" : "8261054f-be78-422d-bd51-4ed9f33c3422",
    "value" : "0866985522"
  }, {
    "identifierTypeId" : "7e591197-f335-4afb-bc6d-a6d76ca3bace",
    "value" : "(OCoLC)962073864"
  }, {
    "identifierTypeId" : "7e591197-f335-4afb-bc6d-a6d76ca3bace",
    "value" : "(OCoLC)ocn962073864"
  }, {
    "identifierTypeId" : "7e591197-f335-4afb-bc6d-a6d76ca3bace",
    "value" : "(OCoLC)962073864"
  }, {
    "identifierTypeId" : "7e591197-f335-4afb-bc6d-a6d76ca3bace",
    "value" : "(DE-599)GBV880391235"
  } ],
  "contributors" : [ {
    "contributorNameTypeId" : "2b94c631-fca9-4892-a730-03ee529ffe2a",
    "name" : "Lucas, Peter J",
    "contributorTypeId" : null,
    "contributorTypeText" : null
  }, {
    "contributorNameTypeId" : "2e48e713-17f3-4c13-a9f8-23845bb210aa",
    "name" : "Arizona Center for Medieval and Renaissance Studies",
    "contributorTypeId" : null,
    "contributorTypeText" : null
  } ],
  "subjects" : [ ],
  "classifications" : [ ],
  "publication" : [ {
    "publisher" : "ACMRS, Arizona Center for Medieval and Renaissance Studies",
    "place" : "Tempe, Arizona",
    "dateOfPublication" : "2016"
  } ],
  "urls" : [ ],
  "instanceTypeId" : "6312d172-f0cf-40f6-b27d-9fa8feaf332f",
  "physicalDescriptions" : [ "69 Mikrofiches 1 Begleitbuch (XII, 167 Seiten)" ],
  "languages" : [ "eng", "ang", "lat" ],
  "notes" : [ "Titel und Angaben zu beteiligter Person vom Begleitheft", "In English with segments in Anglo-Saxon and Latin" ],
  "sourceRecordFormat" : "MARC-JSON",
  "metadata" : {
    "createdDate" : "2018-09-05T03:39:15.564+0000",
    "createdByUserId" : "67b700fb-e1a9-5ed6-aae6-35b6a4df7b00",
    "updatedDate" : "2018-09-05T03:39:15.564+0000",
    "updatedByUserId" : "67b700fb-e1a9-5ed6-aae6-35b6a4df7b00"
  },
  "links" : {
    "self" : "http://folio-snapshot-355.aws.indexdata.com:9130/inventory/instances/8be05cf5-fb4f-4752-8094-8e179d08fb99"
  }
}
Comment by Charlotte Whitt [ 06/Sep/18 ]

Hi Marc Johnson - when we in a 'real' live situation migrate MARC records, then they'd all be stored just like the 11 test records - and then all viewable in Inventory, and in the Inventory storage as MARC-records.

The remaining test records (which are not the special 11 ones), of course are a bit intermediary records. In a real live situation and all was loaded correctly, then these records should be loaded like the 11 with the native MARC records in the Inventory storage.

I would prefer that all bibliographic records were stored the same way as the 11 ones, and labelled with Metadata source value MARC. BUT I can also live with the TBA label, if this is the easiest thing, and is what you Marc Johnson Niels Erik Nielsen prefer - for now.

Comment by Marc Johnson [ 06/Sep/18 ]

Charlotte Whitt I agree there are a few issues here.

It is my understanding that this field isn't presenting the right information for the 11 instances we do have (in that it says TBA for all records). Is that the case?

I think we could potentially resolve this for the Q3 release fairy easily just by checking a property behind the scenes. Niels Erik Nielsen what do you think?

If we do that work, I also think we can put whatever we want in the `Metadata source` field for records that don't have source records yet.

The 11 records we have are static and were converted through a different process by Julian Ladisch. I think there is an intention for all of our instances in the reference environments (folio-testing and folio-snapshot) to be imported from MARC records eventually.

Moving our environments over to loading instances is something Wayne Schneider brought up yesterday, maybe he can expand upon the plan for this?

when we in a 'real' live situation migrate MARC records, then they'd all be stored just like the 11 test records - and then all viewable in Inventory, and in the Inventory storage as MARC-records.

Are you saying that we don't believe we will have any instance records without source records in FOLIO production systems, and so don't have to cater for this situation?

Comment by Charlotte Whitt [ 06/Sep/18 ]

Hi Marc Johnson - oh you're right - the 11 records also says TBA as value in the 'Metadata source'. I thought they would have been associated with 'MARC'.

Comment by Charlotte Whitt [ 06/Sep/18 ]

> Are you saying that we don't believe we will have any instance records without source records in FOLIO production systems, and so don't have to cater for this situation?

no if we migrate MARC records into the FOLIO system, then they will have 'MARC' in the metadata source.

If the Instance record is created in FOLIO, and it has no underlying MARC record, and it is edited in Inventory (not MARCcat), then it should not have 'MARC' in the metadata source, but 'FOLIO' - to the best of my knowledge.

Out of scope for this story, must be to spec. when does the metadata source change I'll write a story for this, which will be relevant, when we have the situation, that the record can be edited in MARCcat. Right now that app is not yet there, but we have this conceptual idea:

Comment by Wayne Schneider [ 06/Sep/18 ]

Moving our environments over to loading instances is something Wayne Schneider brought up yesterday, maybe he can expand upon the plan for this?

The intent is to load instance records in our environments using the translation rules developed in concert with the MM SIG (using mod-data-loader, which also has a feature to create a MARC-in-JSON source record). So my thought was that the reference environment would have both the sample instances hand-coded in the mod-inventory-storage repository (with and without source records), and instances generated by a MARC load using mod-data-loader (all with source records). The instances generated by the MODS import of mod-inventory would go away.

Comment by Marc Johnson [ 06/Sep/18 ]

Charlotte Whitt Thanks for the clarification. I totally agree, imported records that have a MARC source record should present MARC in that field (and don't presently).

Should we use FOLIO as the default rather than blank for instances which don't have MARC source records?

I agree, I think the topic of how this works with MARCCat is indeed a bigger and important subject, and one we likely need to come back to (Wayne Schneider Niels Erik Nielsen and I have talked a little about this in the past and on Slack today).

Comment by Marc Johnson [ 07/Sep/18 ]

Given that it appears no work has been conducted on this, can it be put back into Open or Draft?

Comment by Julian Ladisch [ 07/Sep/18 ]

I don't think that we need a sourceRecordId field in the instance record.
The instance record and the source blob record (eg. MARC record) have the same UUID.
There is an example in the attached diagram NEW Inventory - Storage - MARCCat (1).png showing how it is currently implemented using the same UUID.
Think of it as if the source blob is an optional field of the instance record - then it is clear that they have the same UUID. Only for performance reasons we split off the source blob from the instance record.

Comment by Charlotte Whitt [ 11/Sep/18 ]

Niels Erik Nielsen [10:34]
"TBA" is hard-coded in UI. The 10-20 sample records in mod-inventory-storage has "source"="sample" or "MARC21" - so that's what they would say if I printed that field instead of the hard-coding. Should I print that field? Should I change the content of "source" for these 10-20 records to something other than "sample" or "MARC21"? Looking into random records in folio-testing, some has "source" set to "Local: MODS". Bridget Jones is "sample" up there too.

Niels Erik Nielsen and Charlotte Whitt agreed upon:

  • sample records (Waynes conversion with no MARC blob) - metadata source: FOLIO
  • the 27 test json-records some with View source records in MARC - metadata source: MARC
  • a brand new record created in Inventory - metadata source: FOLIO
Comment by Charlotte Whitt [ 25/Sep/18 ]

Hi Niels Erik Nielsen and Wayne Schneider - this issue say's it's blocked, but in the Q3 release http://folio-q3.aws.indexdata.com/ I can verify:
Metadata source: MARC in

Metadata source: Local: MODS in

The later would preferably to be labelled: metadata source: FOLIO - but I guess this is only a minor detail from the load of records, and has nothing to do with the implemented functionality.

Please confirm. Thanks!

Comment by Charlotte Whitt [ 25/Sep/18 ]

Niels Erik Nielsen Wayne Schneider - I know that you both are extremely busy with other q3 things so I'll go ahead and close this issue, and create a new one, where I attach the one blocking this one: MODINVSTOR-162 Closed .

Manual test - see my comment above.

Generated at Fri Feb 09 00:11:55 UTC 2024 using Jira 1001.0.0-SNAPSHOT#100246-sha1:7a5c50119eb0633d306e14180817ddef5e80c75d.