Skip to:

  1. Skip to Jira Navigation
  2. Skip to Side Navigation
  3. Skip to Main Content

Consume SRS Domain events

Description

mod-linked-data should consume a SourceRecordDomainEvent (Kafka message), sending counterpart is https://folio-org.atlassian.net/browse/MODSOURCE-752.

Acceptance criteria, technical:

Marc bib:

  1. If incoming event contains Instance which doesn’t exist by ID in mod-linked-data → it should be created with source MARC;

  2. If incoming event contains Instance which exists by ID and has source = MARC in mod-linked-data → it should be updated;

  3. If incoming event contains Instance which exists by ID or INVENTORY_ID and has source = LINKED_DATA in mod-linked-data → it should not be updated;

  4. If incoming event contains Instance which exists by INVENTORY_ID under different ID and has source = MARC in mod-linked-data → it should be replaced;

Authority:

  1. If incoming event contains Authority which doesn’t exist by ID in mod-linked-data → it should be created

  2. If incoming event contains Authority which exists by ID in mod-linked-data → it should be updated

Acceptance criteria, QA:

Create and update operations should work for 3 channels:

  1. QuickMarc

  2. Data import

  3. Authority app

Meaning that LDE should reflect all the changes (limited to supported MARC fields).

Environment

None

Potential Workaround

None

Attachments

1
  • 15 Aug 2024, 05:58 PM

continues

is defined by

Checklist

hide

Activity

Show:

Pavel BobylevAugust 19, 2024 at 12:52 PM

Team decision is to test this task as a part of MODSOURCE-752, added NoQA label.

Punnoose Kutty Jacob PullolickalAugust 15, 2024 at 5:58 PM
Edited

Problem #2

Also, I found that Data import of MARC record is not working.

Steps:

  • Import the attached MARC file using data import

  • Check in inventory. You will see that inventory instances are created.

  • Check in linked data. None of the instances are available in linked data.

Loading file...

 

Reason:

I got the kafka event sent from SRS & found that it is not formatted correctly. See the event attached. Check the data in “eventPayload" section & compare it with the “eventPayload" section in my previous comment (which contains the event when instance is created through Quick MARC).

{ "id":"a2db25fe-09b8-4acb-a251-1b6a2078e991", "eventType":"SOURCE_RECORD_CREATED", "eventMetadata":{ "tenantId":"fs09000000", "eventTTL":1, "publishedBy":"mod-source-record-storage-5.9.0-SNAPSHOT" }, "eventPayload":"02000nkd a22004093a 4500001000900000005001700009007001500026007000700041008004100048906004500089955001700134010001700151037005400168040002500222050002600247100004700273245021700320260001600537300004700553336003300600337002600633338003600659500008000695541008100775580009000856524011800946540004201064653004301106655005001149662005201199773007901251856008601330852012801416985001401544985001201558991002001570\u001E21037859\u001E20190626100055.0\u001Ecr |||||||||||\u001Eki c||\u001E180131s2018 xxunnn| |||||u||||kneng \u001E \u001Fa0\u001Fbcbc\u001Fcorignew\u001Fdu\u001Fencip\u001Ff20\u001Fgy-printpho\u001E \u001Faqw26 2019-04\u001E \u001Fa 2018700254\u001E \u001FaLC-DIG-highsm-52135\u001FbDLC\u001Fc(original digital file)\u001E \u001FaDLC\u001Fbeng\u001FcDLC\u001Fedcrmg\u001E00\u001FaLC-DIG-highsm-\u001Fb52135\u001E1 \u001FaHighsmith, Carol M.,\u001Fd1946-\u001Fephotographer.\u001E10\u001FaVeterans Memorial Park, which opened in 2008 in Auburn, New York, honors all veterans form Cayuga County who served in the United States military from the Revolutionary War through current deployments\u001Fh[graphic].\u001E \u001Fc2018-07-19.\u001E \u001Fa1 photograph :\u001Fbdigital, tiff file, color.\u001E \u001Fastill image\u001Fbsti\u001F2rdacontent\u001E \u001Facomputer\u001Fbc\u001F2rdamedia\u001E \u001Faonline resource\u001Fbcr\u001F2rdacarrier\u001E \u001FaTitle, date and keywords based on information provided by the photographer.\u001E \u001FcPurchase;\u001FaCarol M. Highsmith Photography, Inc.;\u001Fd2018;\u001Fe(DLC/PP-2018:052-2)\u001E \u001FaForms part of Carol M. Highsmith's America Project in the Carol M. Highsmith Archive.\u001E8 \u001FaCredit line: Photographs in the Carol M. Highsmith Archive, Library of Congress, Prints and Photographs Division.\u001E \u001FaNo known restrictions on publication.\u001E \u001FaAmerica\u001FaAuburn Veterans Memorial Park\u001E 7\u001FaDigital photographs\u001FxColor\u001Fy2010-2020.\u001F2gmgpc\u001E \u001FaUnited States\u001FbNew York (State)\u001FdAuburn.\u001F2local\u001E0 \u001FaHighsmith, Carol M., 1946-\u001FtCarol M. Highsmith Archive. \u001Fw(DLC) 00650024\u001E41\u001F3original digital file\u001Fdhighsm\u001Ff52135\u001Fqp\u001Fuhttps://hdl.loc.gov/loc.pnp/highsm.52135\\u001E \u001FaLibrary of Congress\u001FbPrints and Photographs Division\u001FeWashington, D.C. 20540 USA\u001Fndcu\u001Fuhttps://hdl.loc.gov/loc.pnp/pp.print\\u001E \u001Fapp/highsm\u001E \u001Fapp/diof\u001E \u001Fbc-P&P\u001Fm(ONLINE)\u001E\u001D" }

Punnoose Kutty Jacob PullolickalAugust 15, 2024 at 3:31 PM
Edited

Hi

Problem #1
Karate tests are failing after this change. Reason - When SRS emits SOURCE_RECORD_CREATED event when an instance is created via quick marc, 999 $i and $s are not present in the event (see a sample event I pulled from Kafkadrop below). As a result, the SRS ID and inventory ID are not persisted in the linked data DB. So, when you make a change to that instance in linked data editor & save, the change is not sent back to inventory

Steps to reproduce this manually:

  1. In inventory, derive a new MARC bib record.

  2. Navigate to linked data editor

  3. Optional - Open dev tools of the browser.

  4. Search for the new instance. The new instance will be shown in search results.

  5. Open the instance for editing (Optional - Look at the GET /resource/{id} API response in dev tools. You can see that “inventoryId” and “srsId” are missing in “instanceMetadata” section of the response [or “folioMetaData” if Andrei’s changes are deployed])

  6. Now, make some changes in the instance in Linked data editor (say, update the title) and save

  7. Navigate back to Inventory

Expected result: The change made in step #6 is reflected in inventory. Also, the source of the record is changed to LINKED_DATA

Actual result: Inventory still show the old instance. [Problem]



Sample event from Kafka (http://int-pub-us-west-2.int.folio.ebsco.com:7102/topic/qld.ALL.srs.source_records)

{ "id":"77f3831c-0dd6-4482-a026-82a51a74a3f8", "eventType":"SOURCE_RECORD_CREATED", "eventMetadata":{ "tenantId":"fs09000000", "eventTTL":1, "publishedBy":"mod-source-record-storage-5.9.0-SNAPSHOT" }, "eventPayload":"{\"leader\":\"04513nam a2200829 i 4500\",\"fields\":[{\"008\":\"240815s1981 ic f 001 0 u\"},{\"010\":{\"subfields\":[{\"a\":\" 80021016 \"}],\"ind1\":\" \",\"ind2\":\" \"}},{\"020\":{\"subfields\":[{\"a\":\"123456789123456\"},{\"q\":\"ISBN\"}],\"ind1\":\" \",\"ind2\":\" \"}},{\"024\":{\"subfields\":[{\"a\":\"EAN\"},{\"d\":\"EAN test2\"}],\"ind1\":\"1\",\"ind2\":\"3\"}},{\"040\":{\"subfields\":[{\"a\":\"DLC\"},{\"c\":\"DLC\"},{\"d\":\"ICU\"}],\"ind1\":\" \",\"ind2\":\" \"}},{\"041\":{\"subfields\":[{\"a\":\"eng\"},{\"a\":\"spa\"}],\"ind1\":\"0\",\"ind2\":\" \"}},{\"046\":{\"subfields\":[{\"k\":\"12/09/1990\"},{\"l\":\"2056\"}],\"ind1\":\" \",\"ind2\":\" \"}},{\"050\":{\"subfields\":[{\"a\":\"PC4112\"},{\"b\":\".N38 1981\"}],\"ind1\":\"0\",\"ind2\":\" \"}},{\"082\":{\"subfields\":[{\"a\":\"468.2/421 class_number\"},{\"2\":\"19\"}],\"ind1\":\"0\",\"ind2\":\" \"}},{\"100\":{\"subfields\":[{\"a\":\"Neale-Silva, Eduardo\"},{\"0\":\"Family type\"}],\"ind1\":\"3\",\"ind2\":\" \"}},{\"110\":{\"subfields\":[{\"a\":\"name test\"},{\"0\":\"number test\"},{\"e\":\"lyricist.\"}],\"ind1\":\" \",\"ind2\":\" \"}},{\"110\":{\"subfields\":[{\"a\":\"name test\"},{\"0\":\"number test\"},{\"e\":\"editor.\"}],\"ind1\":\" \",\"ind2\":\" \"}},{\"111\":{\"subfields\":[{\"a\":\"meeting name test\"},{\"0\":\"meeting number test\"},{\"j\":\"author\"}],\"ind1\":\" \",\"ind2\":\" \"}},{\"245\":{\"subfields\":[{\"a\":\"New Marc derived record\"}],\"ind1\":\"1\",\"ind2\":\"0\"}},{\"250\":{\"subfields\":[{\"a\":\"2d ed.\"}],\"ind1\":\" \",\"ind2\":\" \"}},{\"260\":{\"subfields\":[{\"a\":\"Glenview, Ill. :\"},{\"b\":\"Scott, Foresman, test\"},{\"c\":\"c1981.\"}],\"ind1\":\"0\",\"ind2\":\" \"}},{\"261\":{\"subfields\":[{\"d\":\"2023\"}],\"ind1\":\" \",\"ind2\":\" \"}},{\"263\":{\"subfields\":[{\"a\":\"2023\"}],\"ind1\":\" \",\"ind2\":\" \"}},{\"264\":{\"subfields\":[{\"a\":\"Alaska\"},{\"b\":\"name $ 2023\"}],\"ind1\":\"2\",\"ind2\":\"0\"}},{\"300\":{\"subfields\":[{\"a\":\"467 p., [8] leaves of plates : a\"},{\"b\":\"ill., some col. ; b\"},{\"c\":\"25 cm.\"}],\"ind1\":\" \",\"ind2\":\" \"}},{\"310\":{\"subfields\":[{\"a\":\"publication frequency\"},{\"b\":\"publication frequency_b\"}],\"ind1\":\" \",\"ind2\":\" \"}},{\"334\":{\"subfields\":[{\"a\":\"multipart monograph\"}],\"ind1\":\" \",\"ind2\":\" \"}},{\"336\":{\"subfields\":[{\"a\":\"text\"},{\"b\":\"txt\"},{\"2\":\"336\"}],\"ind1\":\" \",\"ind2\":\" \"}},{\"337\":{\"subfields\":[{\"a\":\"projected\"},{\"b\":\"g\"},{\"2\":\"337 source\"}],\"ind1\":\" \",\"ind2\":\" \"}},{\"338\":{\"subfields\":[{\"a\":\"card\"},{\"b\":\"no\"},{\"2\":\"338 source1\"}],\"ind1\":\" \",\"ind2\":\" \"}},{\"362\":{\"subfields\":[{\"a\":\"dates of publication note\"},{\"z\":\"dopn_z\"}],\"ind1\":\" \",\"ind2\":\" \"}},{\"500\":{\"subfields\":[{\"a\":\"English or Spanish note.\"}],\"ind1\":\" \",\"ind2\":\" \"}},{\"500\":{\"subfields\":[{\"a\":\"Includes indexes.\"}],\"ind1\":\" \",\"ind2\":\" \"}},{\"501\":{\"subfields\":[{\"a\":\"with Note field\"}],\"ind1\":\" \",\"ind2\":\" \"}},{\"504\":{\"subfields\":[{\"a\":\"bibliography\"},{\"b\":\"note\"}],\"ind1\":\" \",\"ind2\":\" \"}},{\"505\":{\"subfields\":[{\"a\":\"table of contents\"}],\"ind1\":\" \",\"ind2\":\" \"}},{\"506\":{\"subfields\":[{\"a\":\"access note\"},{\"b\":\"access note_b\"},{\"c\":\"access note_c\"},{\"d\":\"access note_d\"},{\"e\":\"access note_e\"},{\"f\":\"access note_f\"},{\"g\":\"access note_ g\"},{\"q\":\"access note_q\"}],\"ind1\":\" \",\"ind2\":\" \"}},{\"508\":{\"subfields\":[{\"a\":\"credits Note field\"}],\"ind1\":\" \",\"ind2\":\" \"}},{\"511\":{\"subfields\":[{\"a\":\"participant Note\"}],\"ind1\":\" \",\"ind2\":\" \"}},{\"513\":{\"subfields\":[{\"a\":\"type of report_a\"},{\"b\":\"type of report_b\"}],\"ind1\":\" \",\"ind2\":\" \"}},{\"515\":{\"subfields\":[{\"a\":\"issuance Note field\"}],\"ind1\":\" \",\"ind2\":\" \"}},{\"516\":{\"subfields\":[{\"a\":\"computer Data Note\"}],\"ind1\":\" \",\"ind2\":\" \"}},{\"520\":{\"subfields\":[{\"a\":\"summary test\"}],\"ind1\":\" \",\"ind2\":\" \"}},{\"524\":{\"subfields\":[{\"a\":\"citation Coverage test\"}],\"ind1\":\" \",\"ind2\":\" \"}},{\"530\":{\"subfields\":[{\"a\":\"additional Physical Form test\"}],\"ind1\":\" \",\"ind2\":\" \"}},{\"532\":{\"subfields\":[{\"a\":\"accessibility Note test\"}],\"ind1\":\" \",\"ind2\":\" \"}},{\"533\":{\"subfields\":[{\"a\":\"reproduction note_a\"},{\"b\":\"repro note_b\"},{\"c\":\"repro note_c\"},{\"d\":\"repro note_d\"},{\"e\":\"repro note_e\"},{\"f\":\"repro note_f\"},{\"n\":\"repro note_n\"}],\"ind1\":\" \",\"ind2\":\" \"}},{\"534\":{\"subfields\":[{\"a\":\"original Version Note_a\"},{\"b\":\"ovn_b\"},{\"c\":\"ovn_c\"},{\"e\":\"ovn_e\"},{\"f\":\"ovn_f\"},{\"k\":\"ovn_k\"},{\"m\":\"ovn_m\"},{\"n\":\"ovn_n\"},{\"t\":\"ovn_t\"}],\"ind1\":\" \",\"ind2\":\" \"}},{\"535\":{\"subfields\":[{\"a\":\"location of originals duplicates\"},{\"b\":\"lofd_b\"},{\"c\":\"lofd_c\"},{\"d\":\"lofd_d\"},{\"g\":\"lofd_g\"}],\"ind1\":\" \",\"ind2\":\" \"}},{\"536\":{\"subfields\":[{\"a\":\"funding information\"},{\"b\":\"funding info_b\"},{\"c\":\"fund info_c\"},{\"d\":\"fund info_d\"},{\"e\":\"fund info_e\"},{\"f\":\"fund info_f\"},{\"g\":\"fund info_g\"},{\"h\":\"fund info_h\"}],\"ind1\":\" \",\"ind2\":\" \"}},{\"538\":{\"subfields\":[{\"a\":\"system details\"},{\"i\":\"system details_i\"},{\"u\":\"system details_u\"}],\"ind1\":\" \",\"ind2\":\" \"}},{\"542\":{\"subfields\":[{\"a\":\"info relating to copyright status\"},{\"b\":\"b\"},{\"c\":\"c\"},{\"d\":\"d\"},{\"e\":\"e\"},{\"f\":\"f\"},{\"g\":\"g\"},{\"h\":\"h\"},{\"i\":\"i\"},{\"j\":\"j\"},{\"k\":\"k\"},{\"l\":\"l\"},{\"m\":\"m\"},{\"n\":\"n\"},{\"o\":\"o\"},{\"p\":\"p\"},{\"q\":\"q\"},{\"r\":\"r\"},{\"s\":\"s\"}],\"ind1\":\" \",\"ind2\":\" \"}},{\"544\":{\"subfields\":[{\"a\":\"related parts\"},{\"b\":\"related parts_b\"},{\"c\":\"related parts_c\"},{\"d\":\"related parts_d\"},{\"e\":\"related parts_e\"},{\"n\":\"related parts_n\"}],\"ind1\":\" \",\"ind2\":\" \"}},{\"546\":{\"subfields\":[{\"a\":\"Language note\"}],\"ind1\":\" \",\"ind2\":\" \"}},{\"547\":{\"subfields\":[{\"a\":\"former Title Note\"}],\"ind1\":\" \",\"ind2\":\" \"}},{\"550\":{\"subfields\":[{\"a\":\"issuing Body\"}],\"ind1\":\" \",\"ind2\":\" \"}},{\"552\":{\"subfields\":[{\"a\":\"entity and attribute information\"},{\"b\":\"e&a info_b\"},{\"c\":\"e&a info_c\"},{\"d\":\"e&a info_d\"},{\"e\":\"e&a info_e\"},{\"f\":\"e&a info_f\"},{\"g\":\"e&a info_g\"},{\"h\":\"e&a info_h\"},{\"i\":\"e&a info_i\"},{\"j\":\"e&a info_j\"},{\"k\":\"e&a info_k\"},{\"l\":\"e&a info_l\"},{\"m\":\"e&a info_m\"},{\"n\":\"e&a info_n\"},{\"o\":\"e&a info_o\"},{\"p\":\"e&a info_p\"},{\"u\":\"e&a info_u\"},{\"z\":\"e&a info_z\"}],\"ind1\":\" \",\"ind2\":\" \"}},{\"555\":{\"subfields\":[{\"a\":\"location of other archival material\"},{\"b\":\"locofother_b\"},{\"c\":\"locofother_c\"},{\"d\":\"locofother_d\"}],\"ind1\":\" \",\"ind2\":\" \"}},{\"556\":{\"subfields\":[{\"a\":\"information About Documentation test\"}],\"ind1\":\" \",\"ind2\":\" \"}},{\"581\":{\"subfields\":[{\"a\":\"additional physical form\"},{\"z\":\"add phys form_z\"}],\"ind1\":\" \",\"ind2\":\" \"}},{\"585\":{\"subfields\":[{\"a\":\"exhibitions Note\"}],\"ind1\":\" \",\"ind2\":\" \"}},{\"588\":{\"subfields\":[{\"a\":\"description Source Note\"}],\"ind1\":\" \",\"ind2\":\" \"}},{\"600\":{\"subfields\":[{\"a\":\"name test tt family\"},{\"b\":\"numeration\"},{\"c\":\"titles\"},{\"d\":\"date 2024\"},{\"j\":\"attribution\"},{\"q\":\"Alternative name tt\"},{\"u\":\"affiliation\"},{\"v\":\"formSubdivision\"},{\"x\":\"generalSubdivision\"},{\"y\":\"chronologicalSubdivision\"},{\"z\":\"geographicSubdivision\"},{\"2\":\"source\"},{\"3\":\"materialsSpecified\"},{\"e\":\"relator_term\"},{\"4\":\"relator_code\"},{\"0\":\"authorityLink\"},{\"1\":\"equivalent\"},{\"6\":\"linkage\"},{\"7\":\"controlField\"},{\"8\":\"fieldLink\"}],\"ind1\":\"3\",\"ind2\":\" \"}},{\"650\":{\"subfields\":[{\"a\":\"Spanish language.\"},{\"b\":\"geog cover\"},{\"c\":\"Event location\"},{\"d\":\"2024\"},{\"g\":\"misc info\"},{\"2\":\"source\"},{\"0\":\"http://id.worldcat.org/fast/fst01128292\\"},{\"x\":\"General subd\"},{\"y\":\"chron subd\"},{\"3\":\"Specified material\"},{\"e\":\"Relator term\"},{\"4\":\"Relator code\"},{\"1\":\"Equivalent\"},{\"6\":\"Linkage\"},{\"7\":\"Control field\"},{\"8\":\"Field link\"}],\"ind1\":\" \",\"ind2\":\" \"}},{\"651\":{\"subfields\":[{\"a\":\"name tt\"},{\"g\":\"Misc\"},{\"v\":\"Form subdivision\"},{\"x\":\"General subdivision\"},{\"y\":\"chronologicalSubdivision\"},{\"z\":\"geographicSubdivision\"},{\"2\":\"source\"},{\"3\":\"materialsSpecified\"},{\"e\":\"relator_term\"},{\"4\":\"relator_code\"},{\"0\":\"authorityLink\"},{\"1\":\"equivalent\"},{\"6\":\"linkage\"},{\"7\":\"controlField\"},{\"8\":\"fieldLink\"}],\"ind1\":\" \",\"ind2\":\" \"}},{\"655\":{\"subfields\":[{\"a\":\"Textbooks name\"},{\"b\":\"geographicCoverage\"},{\"v\":\"for English speakers. formSubdivision\"},{\"x\":\"generalSubdivision\"},{\"y\":\"chronologicalSubdivision\"},{\"z\":\"geographicSubdivision\"},{\"2\":\"fast source\"},{\"3\":\"materialsSpecified\"},{\"e\":\"relator_term\"},{\"4\":\"relator_code\"},{\"0\":\"http://id.worldcat.org/fast/fst01423865\\"},{\"1\":\"equivalent\"},{\"6\":\"linkage\"},{\"7\":\"controlField\"},{\"8\":\"fieldLink\"}],\"ind1\":\" \",\"ind2\":\"7\"}},{\"700\":{\"subfields\":[{\"a\":\"Nicholas, Robert L. test\"},{\"e\":\"editor09\"},{\"0\":\"contr number\"},{\"a\":\"rt\"}],\"ind1\":\"3\",\"ind2\":\" \"}},{\"710\":{\"subfields\":[{\"a\":\"org contr\"},{\"0\":\"org contr nmbr\"},{\"4\":\"anl1\"}],\"ind1\":\" \",\"ind2\":\" \"}},{\"711\":{\"subfields\":[{\"a\":\"cntr meeting name\"},{\"0\":\"cntr meeting number\"},{\"j\":\"photographer.,\"}],\"ind1\":\" \",\"ind2\":\" \"}},{\"753\":{\"subfields\":[{\"a\":\"system details access note\"},{\"b\":\"sdan_b\"},{\"c\":\"sdan_c\"}],\"ind1\":\" \",\"ind2\":\" \"}},{\"856\":{\"subfields\":[{\"u\":\"test@tt.com\"},{\"z\":\"note test tt\"}],\"ind1\":\" \",\"ind2\":\" \"}},{\"005\":\"20240815150718.8\"}]}" }
Done

Details

Assignee

Reporter

Labels

Priority

Story Points

Sprint

Development Team

Citation

TestRail: Cases

Open TestRail: Cases

TestRail: Runs

Open TestRail: Runs

Created August 1, 2024 at 1:48 PM
Updated November 21, 2024 at 10:42 PM
Resolved August 19, 2024 at 12:52 PM
TestRail: Cases
TestRail: Runs

Flag notifications