Versions Compared

Old Version 6

changes.mady.by.user Christie Thomas

Saved on

compared with

New Version Current

changes.mady.by.user Ann-Marie Breaux

Saved on

Key

  • This line was added.
  • This line was removed.
  • Formatting was changed.

Recordings are posted Here (2022+) and Here (pre-2022)                   Slack channel for Q&A, discussion between meetings

Requirements details Here                                                                    Additional discussion topics in Subgroup parking lot


Attendees: Ann-Marie Breaux (Deactivated) Jennifer Eustis Monica Arnold Lynne Fors Christie Thomas Taylor Smith Jeanette Kalchik Jamie Jesanis (Unlicensed) Jenn Colt 

Morning Glory

Agenda topics:

  • MM SIG presentation on last week on Data Import and Morning Glory/Nolana
  • Deleting import logs via API:
    • Available as of Morning Glory
    • There was a Slack question recently about being able to delete Import logs via API as of Morning Glory. We have added some documentation in the Data Import tips and tricks. Please see this page: 4-Data Import Logs, in the last section: Deleting Import Logs via API.
    • Questions:
      • The log data can be queried and compiled via the LDP; what happens with that when 1) the log is marked for deletion (via the UI or API) and then 2) completely deleted in the next 24-hour sweep?
      • Can the log info still be queried via API when it has been marked for deletion but not yet deleted?
      • Jenn: Cornell started playing with log info via API, but stopped until they go live on Lotus
      • NOTE: This has nothing to do with the instance "mark for deletion" discussion
  • MARC Modifications and Field protections
    • MG bug:
      Jira Legacy
      serverSystem JiraJIRA
      serverId01505d01-b853-3c2e-90f1-ee9b165564fc
      keyMODDICORE-248
    • Modify scenarios – describe real use cases – homeworkcases, for  use  in  developer  analysis  and  test  cases
      • https://bugfest-mg.int.aws.folio.org/settings/data-import/job-profiles/view/673f7ba3-27d2-4bf4-935f-f358f46583fe?sort=name
      • At beginning of job
        • Single record imports: remove junk fields from OCLC, clean up ocm/ocn/on fields, remove 029, some 9xx fields, add some data to allow for creating holdings/items (Jennifer E)
        • Assign a collection ID to a vendor file; correct some MARC errors via DI instead of MarcEdit (Jenn C)
        • MARC Bib for print record and want to add a URL; match on HRID, add the URL, update the record with this collection name - edge case; then update action for now (Jeanette)
      • At end of job
        • Remove 9xx fields after data has been harvested for use in holdings or item (Jennifer)
        • Remove 856s after it has been mapped into a holdings or item (Jennifer)
      • After match
        • Match to instance, after no match for instance, then create instance, then modify to add 856 proxy for school A, then create holdings for school A (Jennifer E)
        • Then another job to match to instance that was just created, and after match, modify to add proxy for school B, then create holdings for school B, then match to same instance, modify for school C, create holdings for school C (Jennifer E) - right now all of this is in separate jobs, after modifications in MarcEdit, then removing 856 from Bib at the end, so 6 total jobs; would be great to reduce to 2, and then perhaps 1 eventually
        • After
        match?
        • match for a particular school, edit 856 to assign correct proxy, then update holdings (Jennifer)
        • Bulk edits - find location of X and change to Y (Christie) - not a modification scenario, if using something other than location for match
        • Match on location and then delete the location from 9xx data after it was used for the match (Christie) - can do at the end of job
        • Existing SRS MARC Bib with JSTOR  (Lisa) - talk about when we go into details for MARC Updates
      • At beginning and end of job
        • Yes
      • At beginning, after match, and at end of job?
        • Potentially yes
    • Any scenario where a job would have a Modify action without a Create or Update?
      • Example: https://bugfest-mg.int.aws.folio.org/settings/data-import/job-profiles/view/5564537e-9bda-426f-a755-de5ae16d7ebd?sort=name
        • Maybe in a bulk edit type scenario, where you are wanting to identify records and then change them, without exporting and re-importing, but we have the current export/reimport workaround, and potentially bulk-edit without exporting in the future
        • What would be the expected outcome for a job set up like this? Should we allow modify actions without a subsequent create or update for SRS MARC Bib and/or Instance? NO - would break the current vision of data import (see 
        • Confirm that you can have Modify and Update MARC Bib actions in the same job profile, under the same match; otherwise add bug (2 actions on the same kind of record in a row) (Jenn C)
        • And start making a list of invalid job profile structures so that we can begin to block them ro make them useable
          • Modify with no Create or Update MARC Bib or Authority
          • Match on Instance ID, then Modify/Update MARC Bib (currently have to match on MARC ID before a MARC action)
        • Also causes weird log summary results when same record is visited twice (multi instead of created or updated); multi should only show when multiple (holdings or items) are created/updated with multiple actions, which cannot currently happen via DI
          • Make sure there is a bug for the current weird log results
          • Revisit what the log info should look like when multiple holdings/items acted upon via a single incoming MARC Bib
    • Would it be valid to say that field protections and overriding field protections only invoke when there is one of these Update actions in a job profile?
      • Update MARC Bib (protect and override)
      • Update Instance (protect only, if override need an Update MARC Bib action)
      • Update MARC Authority (protect only, in previous discussion, decided that overrides are not needed for MARC Authority, only Bibs)
      • Not for update Holdings, Item actions (no MARC record underneath)
      • Not for update MARC Holdings action (field protections not currently allowed in FOLIO, per previous discussion)
    • Misc
      • Make sure there are Jiras for on-demand cleanup scripts to 
        • Identify and remove HRIDs stored as 035s (which then cleans up the related instance identifiers derived from the 035)
        • Identify and clean up 035s with various versions of the OCLC prefix (end up with one 035 with only the (OCoLC) prefix (which then cleans up the related instance identifiers derived from the 035)
  • Lotus HF3 Hot fixes for Data Import:
    • View list here
    • Jira Legacy
      serverSystem JiraJIRA
      serverId01505d01-b853-3c2e-90f1-ee9b165564fc
      keyMODSOURCE-509
      : working except one small issue; when an incoming MARC Bib has an HRID in 001, an 035 will be created; documenting as known issue in Lotus; attempting to correct in Morning Glory
    • Jira Legacy
      serverSystem JiraJIRA
      serverId01505d01-b853-3c2e-90f1-ee9b165564fc
      keyMODDICORE-272
      : working but would be good to have additional testing
    • Jira Legacy
      serverSystem JiraJIRA
      serverId01505d01-b853-3c2e-90f1-ee9b165564fc
      keyMODSOURMAN-841
      : working but would be good to have additional testing
  • Morning Glory Bug fixes for Data Import: 
    • View complete list here
    • View unclosed bugs here
      • Some issues have different solutions for Lotus HF versus MG Bugfix versus Nolana, e.g.
        Jira Legacy
        serverSystem JiraJIRA
        serverId01505d01-b853-3c2e-90f1-ee9b165564fc
        keyMODSOURCE-509
        versus
        Jira Legacy
        serverSystem JiraJIRA
        serverId01505d01-b853-3c2e-90f1-ee9b165564fc
        keyMODSOURCE-528
        , or
        Jira Legacy
        serverSystem JiraJIRA
        serverId01505d01-b853-3c2e-90f1-ee9b165564fc
        keyMODSOURMAN-841
        versus
        Jira Legacy
        serverSystem JiraJIRA
        serverId01505d01-b853-3c2e-90f1-ee9b165564fc
        keyMODSOURMAN-843
        ; fixed Lotus first, now MG in progress
      • Jira Legacy
        serverSystem JiraJIRA
        serverId01505d01-b853-3c2e-90f1-ee9b165564fc
        keyMODDICORE-275
        : tested on Snapshot, and seems to be working properly; would be good to have someone else test it on Snapshot and confirm, before we release it to MG BF
      • Jira Legacy
        serverSystem JiraJIRA
        serverId01505d01-b853-3c2e-90f1-ee9b165564fc
        keyMODSOURMAN-840
        : finishing the fix; files where some records have 999 ff's and some don't were giving us the most trouble
      • Jira Legacy
        serverSystem JiraJIRA
        serverId01505d01-b853-3c2e-90f1-ee9b165564fc
        keyMODINV-709
        :
        • Separate profiles for 1) match by POL and update Inventory records and 2) match by VRN and update Inventory records work properly
        • Aiming to fix in Nolana; use separate profiles as a workaround for MG

...

  • 17 August 2022
    • Order field mapping profile questions
    • Jira Legacy
      serverSystem JiraJIRA
      serverId01505d01-b853-3c2e-90f1-ee9b165564fc
      keyUIDATIMP-294
      • Go over POL limit override, Create separate POs based on MARC value 
      • POL Subscription fields: included as part of one-time order in Orders app; include in field mapping?
      • Overall screen - any questions?
  • 24 August 2022
    • Combined meeting with quickMARC Subgroup, to go over MARC Authority and MARC Bibliographic headings linking (only manually for now, not via DI)

Chat:

From Christie Thomas (she/her) to Everyone 01:40 PM
I think it would be an update and then the 856 would be the only field field impacted, correct?

From Jenn Colt to Everyone 01:40 PM
it's update, but update has repeatable field problems

From Christie Thomas (she/her) to Everyone 01:41 PM
Right, Jenn. We have not attempted to use  update in that what.
in that way ..

From Jenn Colt to Everyone 01:41 PM
i tried, it was not good
lol

From Christie Thomas (she/her) to Everyone 01:52 PM
it does not work.
that sounds like the job that lisa mccoll was describing earlier.

From jeanette kalchik to Everyone 01:59 PM
Thank you. Still trying to get some of the logistics of what can do and cannot do.

From Jennifer Eustis (she/her) to Everyone 02:00 PM
I think it gets stuck with the ones I tested

From Jenn Colt to Everyone 02:01 PM
have to go…

From Christie Thomas (she/her) to Everyone 02:02 PM
yeah, the log information is really confusing.

From jeanette kalchik to Everyone 02:08 PM
Thank you so much!