Batch Importer (Bib/Acq) (UXPROD-47)

[UXPROD-2741] Import of MARC Bibs to create/update multiple holdings and items: BE work Created: 09/Oct/20  Updated: 08/Nov/23  Resolved: 08/Nov/23

Status: Closed
Project: UX Product
Components: None
Affects versions: None
Fix versions: Poppy (R2 2023)
Parent: Batch Importer (Bib/Acq)

Type: New Feature Priority: P2
Reporter: Ann-Marie Breaux (Inactive) Assignee: Ann-Marie Breaux (Inactive)
Resolution: Done Votes: 0
Labels: LC-priority1, consortia-ebsco, data-import, loc, metadatamanagement, po-mvp
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original estimate: Not Specified

Issue links:
Defines
defines UXPROD-47 Batch Importer (Bib/Acq) Analysis Complete
is defined by FAT-7383 Update Karate tests to support Multip... Closed
is defined by MODDICORE-90 Change Mapper to allow mapping of mul... Closed
is defined by MODDICORE-317 Add support for matching multiple ent... Closed
is defined by MODDICORE-336 Change Mapper to allow mapping of mul... Closed
is defined by MODDICORE-366 Remove distinct by permanentLocationI... Closed
is defined by UIDATIMP-1443 Improve mapping profile for Holdings ... Closed
is defined by FAT-4834 Add Karate tests for creating multipl... Closed
is defined by MODDATAIMP-801 Test and merge PRs to allow create/up... Closed
is defined by MODDICORE-71 SPIKE: Create multiple Holdings and I... Closed
is defined by MODDICORE-340 Improve mapper for Multiple Holdings Closed
is defined by MODDICORE-348 Improve mapping mechanism to handle U... Closed
is defined by MODINV-785 Update Create Holdings handler to sup... Closed
is defined by MODINV-786 Update Create Item handler to support... Closed
is defined by MODINV-801 Change Update Holdings handler to sup... Closed
is defined by MODINV-802 Change Update Item handler to support... Closed
is defined by MODORDERS-881 Populate locations and material type ... Closed
is defined by MODSOURMAN-965 Adjust progress tracking mechanism wi... Closed
is defined by MODSOURMAN-971 Adjust journal records population to ... Closed
is defined by MODSOURMAN-975 Adjust DI log DTO to allow displaying... Closed
is defined by MODSOURMAN-993 Create journal_records for non matche... Closed
is defined by UIDATIMP-1486 UI Test and merge PRs to allow create... Closed
Gantt End to End
has to be finished together with UXPROD-4079 DI Log updates for multiple holdings/... Closed
Relates
relates to UXPROD-4124 NFR: UI end-to-end current tests for ... Closed
relates to UXPROD-2338 Remaining Job and Match profile refin... Closed
relates to UXPROD-3782 Import of MARC Bibs to create MARC Bi... Draft
Requires
requires UXPROD-3729 Planning for Import of MARC Bibs to c... Closed
Release: Poppy (R2 2023)
Epic Link: Batch Importer (Bib/Acq)
Analysis Estimate: Small < 3 days
Analysis Estimator: Ann-Marie Breaux (Inactive)
Front End Estimate: Small < 3 days
Front-End Confidence factor: 50%
Back End Estimate: XXL < 30 days
Back-End Confidence factor: 80%
Development Team: Folijet
Kiwi Planning Points (DO NOT CHANGE): 40
PO Rank: 112
Rank: Chalmers (Impl Aut 2019): R5
Rank: Chicago (MVP Sum 2020): R1
Rank: Cornell (Full Sum 2021): R2
Rank: Duke (Full Sum 2021): R1
Rank: 5Colleges (Full Jul 2021): R2
Rank: FLO (MVP Sum 2020): R1
Rank: GBV (MVP Sum 2020): R4
Rank: Lehigh (MVP Summer 2020): R1
Rank: MO State (MVP June 2020): R2
Rank: TAMU (MVP Jan 2021): R2
Rank: U of AL (MVP Oct 2020): R2

 Description   

Split from UXPROD-2338 Closed /Q3 2020

This feature allows for multiple holdings and/or items to be created or edited based on data supplied in a single MARC record

Background:

Purpose: To create/update multiple holdings and items on the same instance, from the same imported MARC record

Important for

  • 5 Colleges Consortium
  • Spokane Public
  • Chicago
  • MARMOT Consortium

As a staff person working with data import

I want to update multiple holdings and their associated items when importing a single MARC record
So that I can use all of the holdings/items data on the MARC record properly

Scenarios

  1. Scenario 1
    • Given a MARC record with multiple holdings and item data in it
    • When importing to update holdings and items
    • Then update all relevant holdings on the same instance and update all relevant items on the appropriate holdings
    • Include any holdings and items data in the same MARC field, different subfields
    • If multiple item records are to be updated on the same holdings, repeat the holdings data in repeated MARC fields, with each item's data in a separate MARC field. Item data/subfields for multiple item records cannot be included in the same MARC field
    • See example MARC record below
  2. Scenario 2
    • Ensure this story passes Accessibility testing
  3. Scenario 3
    • Ensure automated testing coverage for this new code is 80% plus and confirm with screenshot

See attached MARC file; test load to update instance, holdings, and item to ensure the logic is working correctly.

Matching notes:

  • Match for holdings could be based on
    • Holdings HRID/UUID
    • Instance HRID/UUID plus secondary match on Holdings HRID/UUID or Location code
  • Match for item could be based on
    • Item HRID, UUID, or barcode
    • Holdings HRID/UUID plus secondary match on Item copy number

Sample record with multiple holdings/item data:

=LDR 01262nam a2200301Ia 4500
=001 ocm54341618\
=003 OCoLC
=005 20070103101904.0
=008 010330s1798\\\\enkaf\\\\\\\\\000\0\eng\d
=035
$a(Sirsi) a551407
=035
$a(Sirsi) o54341618
=049
$aDRUM
=040
$aCUD$beng$cCUD$dDRU$dMvI
=090
$aBS2095$b.S33 1798
=130 0\$aBible.$pNew Testament$lEnglish.$sScarlett.$f1798.
=245 12$aA translation of the New Testament from the original Greek /$chumbly attempted by Nathaniel Scarlett, assisted by men of piety & literature ; with notes.
=260
$aLondon :$bPrinted by T. Gillet; and sold by Nathaniel Scarlett, No. 349, near Exeter 'Change, Strand; also F. & C. Rivington, St. Paul's Church Yard,$c1798.
=300
$axi, 483, vi p., [1] folded leaf of plates :$bill. ;$c19 cm.
=500
$aEngraved t.p.
=500
$aIncludes Observations on some terms used in this translation: vi p. at end.
=510 3\$aDarlow-Moule-Herbert 1433
=700 1\$aScarlett, Nathaniel,$d1753-1802.
=740 02$aObservations on some terms used in this translation.
=945
$a34678234678246423786427$b1$hKU/CC/DI/M <===== $a = item barcode, $b = copy number, $h = holdings permanent location
=945
$a34678234678246423786428$b2$hKU/CC/DI/M
=945
$a34678234678246423786429$b1$hKU/CC/DI/A

If this record were updating holdings and items, it would do so as follows:

  • Update 1 instance, with the data in the main part of the MARC bib record
  • Update 2 holdings, 1 for permanent location KU/CC.DI/M and 1 for permanent location KU/CC/DI/A, using holdings mapping data from 945$h and the defaults in the holdings mapping profile
  • Update 3 item records, 2 linked to the holdings for KU/CC/DI/M and 1 linked to the holdings for KU/CC/DI/A, using item mapping data from 945$a, 945$b, and the defaults in the item mapping profile

NOTE: These 9xx fields and subfields are just examples. Real fields/subfields would be mapped in the field mapping profile. The key is that related holdings and item data must be in the same field, different subfields, and that item data for different items must be in separate MARC fields, not concatenated in the same MARC field.

Steps:

  • data-import-processing-core library should be changed to allow for mapping of multiple entities from a single supplied MARC Bib record
  • mod-inventory Create Holdings and Create Items should be changed to process multiple entities at one stage and fill in relevant information about created/failed entities in the event payload context
  • mod-source-record-manager should contain changes in Journaling and Progress tracking functionality to fill in information about multiple entities and mark record as errored in case at least one of the entities failed to be created
  • mod-orders should be changed to fill in Locations and Material type in POLine based on multiple entities present in the DI_ORDER_CREATED_READY_FOR_POST_PROCESSING event payload  


 Comments   
Comment by Jacquie Samples [ 10/Aug/22 ]

Does this issue include creation of SRS MARC holdings?  That would be very useful.  If not, can we fold it in somehow?

Comment by Ann-Marie Breaux (Inactive) [ 10/Aug/22 ]

Hi Jacquie Samples Do you mean creation of SRS MARC Holdings via imported SRS MARC Bibs?

Comment by Jacquie Samples [ 15/Aug/22 ]

Hi Ann-Marie Breaux,

Yes, that is what I meant. Thanks for the clarifying question.

Comment by Ann-Marie Breaux (Inactive) [ 15/Aug/22 ]

Hi Jacquie Samples That's not currently an aspect of this feature. I can add a new feature to cover that. Would you have a MARC file that I could attach to the new feature as an example of a MARC Bib file that you would like to create MARC Holdings from? And what field(s) from that MARC Bib would be used to create the MARC Holdings? I created Feature UXPROD-3782 Draft for this potential work.

Comment by Jacquie Samples [ 15/Aug/22 ]

Hi Ann-Marie Breaux I will get you a file to use as an example tomorrow.  Do you want it in an email?   I did see UXPROD-3782 Draft , thanks for creating it!

Comment by Ann-Marie Breaux (Inactive) [ 04/Jan/23 ]

Previously status was Analysis complete, but these stories will need to be reviewed and likely rewritten in the context of UXPROD-3729 Closed and https://folio-org.atlassian.net/wiki/pages/viewpage.action?pageId=1397449 - so I changed the status to DRAFT

Comment by Ann-Marie Breaux (Inactive) [ 25/Jan/23 ]

Hi Kateryna Senchenko When creating stories for this feature, please review the draft stories that Oleksii K created. They may need rewriting or replacing. We especially need to review the UI ones, since I don't think we are currently planning any UI changes (or at least not like the ones outlined here)

Generated at Fri Feb 09 00:26:28 UTC 2024 using Jira 1001.0.0-SNAPSHOT#100246-sha1:7a5c50119eb0633d306e14180817ddef5e80c75d.