Metadata Record Export (UXPROD-652)

[UXPROD-1397] Generating a MARC bibliographic record from an Inventory Instance record Created: 06/Dec/18  Updated: 13/Oct/20  Resolved: 12/Jun/20

Status: Closed
Project: UX Product
Components: None
Affects versions: None
Fix versions: Q2 2020
Parent: Metadata Record Export

Type: New Feature Priority: P1
Reporter: Charlotte Whitt Assignee: Magda Zacharska
Resolution: Done Votes: 0
Labels: Lehigh(round_ii+mvp), SRS, cap-mvp, data-import, export, inventory, marc-bib, marccat, metadatamanagement, po-mvp, q1-2020-spillover, q4-2019-at-risk, q4-2019-spillover, split
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original estimate: Not Specified

Attachments: PDF File Instance2BibRecordNoSRS.pdf     PDF File Inventory2MARC.pdf     PNG File screenshot-1.png    
Issue links:
Blocks
blocks ZF-8 Create FOLIO-to-MARCXML translation s... Open
blocks MODOAIPMH-83 POC - Retrieve MARC records generated... Closed
Defines
defines UXPROD-652 Metadata Record Export In Progress
is defined by MDEXP-4 Generate Simplified MARC bib record ... Closed
is defined by MDEXP-6 Create Inventory Instance to MARC Bib... Closed
is defined by MDEXP-8 Export instance records in MARC based... Closed
is defined by MDEXP-21 SPIKE: evaluate the marc4j API for wr... Closed
is defined by MDEXP-22 SPIKE: mapping rules for inventory to... Closed
is defined by MDEXP-37 RecordLoaderService: retrieve Invento... Closed
is defined by MDEXP-59 Create PoC for the MappingProcessor Closed
is defined by MDEXP-7 Modify Inventory Instance to MARC Bib... Closed
is defined by MDEXP-84 Spike: Feature-level design for gener... Closed
is defined by MDEXP-93 Generate MARC bib record containing 0... Closed
is defined by MDEXP-97 Generate MARC bib record - Identifier... Closed
is defined by MDEXP-98 Generate MARC bib - Leader Closed
is defined by MDEXP-100 Generate basic MARC bib record on the... Closed
is defined by MDEXP-102 Generate MARC bib record - 655 $a/nat... Closed
is defined by MDEXP-103 Generate MARC bib record - Contributo... Closed
is defined by MDEXP-122 Generate MARC bib record - 005 field Closed
is defined by MDEXP-123 Generate MARC bib record - 264/publi... Closed
is defined by MDEXP-125 Generate MARC bib record - 490$a/ser... Closed
is defined by MDEXP-126 Generate MARC bib record - 300 field Closed
is defined by MDEXP-136 Generate MARC bib record - 008 fields Closed
is defined by MDEXP-169 Tenant level mapping rules Closed
Gantt End to Start
has to be done before UXPROD-2404 OAI-PMH: Harvest Inventory for invent... Closed
Relates
relates to UXPROD-665 Import Inventory Instances and MARCca... Closed
relates to UXPROD-1595 Connect SRS and MARCcat - Folijet wor... Closed
relates to UXPROD-1806 NFR: Data Import Pub-Sub (Event Drive... Closed
relates to UXPROD-2218 Connect SRS and MARCcat - AtCult work... Closed
relates to MODSOURCE-38 Add "Suppress Discovery" field to the... Closed
relates to UXPROD-149 Create Instance records from a MARC b... Closed
relates to UXPROD-2459 Generating MARC bibliographic record ... Closed
relates to UXPROD-2224 Export Inventory Instances to MARCXML... Open
relates to UXPROD-141 Export Inventory Instances with under... Closed
relates to UXPROD-1447 Instance record with an underlying MA... Closed
relates to UXPROD-1577 Make Inventory Instance rely on SRS M... Closed
relates to UXPROD-1805 SRS MARC-Inventory Instance relations... Closed
relates to UXPROD-2075 MARC Bib Field Mapping Profiles Detai... Closed
relates to UXPROD-2078 SRS MARC-Inventory Instance relations... Closed
relates to UXPROD-1380 CRUD for MARC Field Mapping Profiles,... Closed
relates to UXPROD-2154 Metadata Export - Job Profile Closed
relates to UXPROD-2315 Export Inventory Instances to other n... Closed
relates to ZF-4 Support returning MARC records Closed
Epic Link: Metadata Record Export
Analysis Estimate: Small < 3 days
Analysis Estimator: Charlotte Whitt
Front-End Confidence factor: Medium
Back End Estimate: XXL < 30 days
Back End Estimator: Igor Gorchakov
Estimation Notes and Assumptions: Change in scope because of other team dependencies.
Development Team: Concorde
PO Rank: 125
PO Ranking Note: Increasing PO rank because without ability of generating MARC records on the fly, multiple teams are blocked. This functionality is also required for Round II libraries that don't have all of their instance data in SRS
Rank: Chalmers (Impl Aut 2019): R5
Rank: Chicago (MVP Sum 2020): R1
Rank: Cornell (Full Sum 2021): R5
Rank: Duke (Full Sum 2021): R5
Rank: 5Colleges (Full Jul 2021): R1
Rank: FLO (MVP Sum 2020): R1
Rank: GBV (MVP Sum 2020): R5
Rank: hbz (TBD): R5
Rank: Hungary (MVP End 2020): R1
Rank: Lehigh (MVP Summer 2020): R1
Rank: Leipzig (Full TBD): R4
Rank: Leipzig (ERM Aut 2019): R5
Rank: MO State (MVP June 2020): R1
Rank: TAMU (MVP Jan 2021): R1
Rank: U of AL (MVP Oct 2020): R5

 Description   

While exporting inventory instances into MARC Bib record we will encounter cases when the instance doesn't have an underlying MARC record in SRS. This will happen for the instances that were entered directly in FOLIO and not imported by data-import app for example. For those instances we will need to generate a simplified MARC record that could be included into export file.

The scope for this feature is not exporting the record but generating it so it is accessible to the export when needed. The generated MARC Bib record will be short lived - will not be stored in SRS and will be created each time when inventory instance needs to be represented in MARC Bib record.

The generated record will be base on the mapping provided by Metadata Management SIG: Concorde - Instance to MARC 2019 tab in https://docs.google.com/spreadsheets/d/11lGBiPoetHuC3u-onVVLN4Mj5KtVHqJaQe4RqCxgGzo/edit#gid=1166940623
The spreadsheet is a base for mapping rules that are referenced each time MARC bib record is generated. The mapping rules are library specific and each library can have just one mapping rules set (stored in rules.json file).

This feature covers generating simplified MARC bib record for following inventory instance fields:

  • HRID
  • Mode of Issuance
  • Identifiers (LCCN, ISBN, ISSN, System Control Number, GPO Item Number)
  • Contributors (Personal, Corporate and Meeting Names)
  • Title
  • Instance Type
  • Place of publication
  • Publisher
  • Publication date
  • Physical descriptions
  • Series
  • Subjects
  • Nature of Content
  • Instance UUID
  • Date Created
  • Date Updated


 Comments   
Comment by Cate Boerema (Inactive) [ 03/Jan/19 ]

Per Charlotte Whitt, Wayne has been approved to do this work so I am adding Q1 milestone back. Need to assign to Core: Platform team when we have that set up in JIRA (Wayne is on that team).

While this is ranked as NOT NEEDED by Chalmers, it is actually required for other features they said they did need (Batch Import).

Comment by Sebastian Hammer [ 03/Jan/19 ]

It'd be worth aligning these efforts.

Comment by Cate Boerema (Inactive) [ 08/Jan/19 ]

It was previously thought that Wayne was available to do this work, but it turns out that is not the case. Removing from the Core team Q1 2019 backlog until capacity has been made available.

Comment by Ann-Marie Breaux (Inactive) [ 02/Apr/19 ]

Theodor Tolstoy (One-Group.se) For Chalmers Inventory Instances that DO NOT have an underlying MARC in SRS, what is the plan to get them into EDS? Will there be a separate feed from Libris for them? Will Chalmers want to create a MARC on-the-fly from the FOLIO Instance and send that out via OAI-PMH? If the latter, that is related to this feature, and we'll need to have an understanding of how soon it will be needed. Right now Chalmers has this marked as not needed, so just want to confirm.

Note that this is entirely separate from the conversation about Chalmers' MARC-SRS records and OAI-PMH for them.

cc: Hkaplanian Craig McNally

Comment by Theodor Tolstoy (One-Group.se) [ 03/Apr/19 ]

Yes, there will be a separate feed directly from Libris to EDS. So Chalmers does not want to create MARC-records on the fly from Inventory instances.

Slightly related is the need for defining sets/setSpecs in the OAI-PMH server so that only the relevant (SRS) records are being exported once we get there.

HkaplanianCraig McNallyAnn-Marie Breaux

Comment by Ann-Marie Breaux (Inactive) [ 03/Apr/19 ]

Thanks Theodor Tolstoy (One-Group.se) Since SRS only contains MARC records, then I don't think we need to define sets for what is harvested from SRS via OAI-PMH. Once the 28K problem records are loaded into Libris, will you want to remove them from SRS and only have the Instances in FOLIO (with no underlying SRS MARC?)

Comment by Theodor Tolstoy (One-Group.se) [ 05/Apr/19 ]

Thank you Ann-Marie Breaux.

What i meant was, that I just want to make sure that some time along the way, the feed will not suddenly start spitting out Inventory Instances together with the SRS records.

Comment by Cate Boerema (Inactive) [ 06/Aug/19 ]

Ann-Marie Breaux and Magda Zacharska I see this is assigned to Folijet. Is there a specific reason this needs to be assigned to Folijet? The Cap planing group had been viewing the Export epic as unassigned but I am now noticing 2 of the approximately 6 features are assigned to Folijet.

I am going to assign the other features to Core: Functional for now because that makes it easier for us to look across all the work that needs to be distributed. If there isn't a specific reason why this needs to belong with Folijet, can you please assign to Core: Functional?

Comment by Marc Johnson [ 06/Aug/19 ]

Cate Boerema Ann-Marie Breaux Magda Zacharska Jakub Skoczen

Is there a specific reason this needs to be assigned to Folijet? The Cap planing group had been viewing the Export epic as unassigned but I am now noticing 2 of the approximately 6 features are assigned to Folijet.

I don't know what the scope of the export epic is, so this might be way off.

As I understand it, there is the capability to export batches of FOLIO records as JSON and other capabilities for exporting records to specific third party formats like MARC.

I imagine this is assigned to Folijet because they are responsible for data import and source record management (referred to in the description). That is the place in FOLIO that currently maps MARC to an instance, meaning they are the team with experience of MARC mapping and so will likely be more aware of challenges for mapping an instance to MARC (whether it came from MARC or not).

Comment by Ann-Marie Breaux (Inactive) [ 06/Aug/19 ]

Hi Cate Boerema and Marc Johnson This is part of a 4-step process to connecting SRS, Inventory, and MARCcat for Bibs/Instances. There's a similar 4-step process for Holdings. When I created the 8 Features, I assigned them all to Folijet, even though Part 3 of each was actually export instead of import. Since Export didn't have a home (at the time) and since they related to other synchronizing work I was doing, I kept them for Folijet instead of orphaning them.

Now that Export has a PO and a home and these 2 features are linked to export, they can definitely move to whichever dev team will be handling export. When it comes time to actually do them, it would be good if I could talk over the details with Magda Zacharska since we're just finishing a second round of MARC-to-Instance mapping, and it is quite involved. Instance-to-MARC will be less involved, but there's still stuff to be mindful of.

Comment by Marc Johnson [ 06/Aug/19 ]

Cate Boerema Jakub Skoczen Magda Zacharska Ann-Marie Breaux

Now that Export has a PO and a home and these 2 features are linked to export

I'm out of the loop, who is the PO for export and where is it's home?

Comment by Ann-Marie Breaux (Inactive) [ 07/Aug/19 ]

Hi Marc Johnson

Magda Zacharska is the PO, and home is wherever the Cap Planning team says.

Comment by Marc Johnson [ 07/Aug/19 ]

Ann-Marie Breaux Thanks.

Apologies if I'm being slow. Does home is wherever the Cap Planning team says mean that this has been decided or it hasn't?

Comment by Ann-Marie Breaux (Inactive) [ 07/Aug/19 ]

Hi Marc Johnson I don't know. I don't think it has been decided. My understanding is that Folijet and Core-fxn are both overloaded.

Comment by Cate Boerema (Inactive) [ 07/Aug/19 ]

Marc Johnson cap planning team is still working on figuring out where to assign this (and other features)

Comment by Cate Boerema (Inactive) [ 07/Aug/19 ]

We are assigning pretty much everything to Core: Functional by default and then spreading it around from there so don't be alarmed by the fact that stuff is being assigned to CF. In fact, I am going to assign this one to CF so it's in the list with all the other stuff that needs to be spread around.

Comment by Ann-Marie Breaux (Inactive) [ 05/Dec/19 ]

Hi Magda Zacharska Once we can generate MARC records for Inventory Instances, we may need to add output of those to the OAI-PMH functionality. I looked for a Jira that covers that work, but couldn't find one. Do you know if one exists, or if not, could you perhaps create one? Thank you!

Comment by Magda Zacharska [ 05/Dec/19 ]

Hi Ann-Marie Breaux - here we go: MODOAIPMH-83 Closed

Generated at Fri Feb 09 00:15:16 UTC 2024 using Jira 1001.0.0-SNAPSHOT#100246-sha1:7a5c50119eb0633d306e14180817ddef5e80c75d.