2024-5-22 Data Import Subgroup meeting

Owned by Ryan Taylor
Last updated: May 22, 2024 by Corrie Hutchinson (Unlicensed)
4 min read

Recordings are posted Here (2022+) and Here (pre-2022)                   Slack channel for Q&A, discussion between meetings

Requirements details Here                                                                    Additional discussion topics in Subgroup parking lot


Attendees: Corrie Hutchinson (Unlicensed) Jennifer Eustis Aaron Neslin Taylor Smith Tess Amram, Whitney Christopher Mary Aycock Ryan Taylor Robert Pleshar  Kimberly Wiljanen Autumn Faulkner Yael Hod Sara Colglazier Lucas Mak Christie Thomas Lisa Lorenzo Jeanette Kalchik  Natalie K.

Notetaker: Corrie Hutchinson (Unlicensed)

Links:

Agenda: 

TopicWhoMeeting NotesRelated JiraDecisions and Actions

Announcements:


Ramsons UAT planned for next week : 035 normalization

  • Latest code is in snapshot


Discuss: Issue of multiple SRS with State "ACTUAL" after Update jobCorrie Hutchinson (Unlicensed), Christie Thomas

See Slack discussion here: https://folio-project.slack.com/archives/CA39M62BZ/p1716239777567769

Overview : a Data Import job is resulting in multiple SRS records with state = 'ACTUAL' associated with an instance record 

  • Chicago is noticing the issue on Poppy CSP3 ; MSU seeing it on CSP4 as well
    • Chicago not using file splitting
    • Has been happening in Chicago since they upgraded to Poppy 
    • MSU's production environment is still on Orchid ; dry run environment is on Poppy CSP4 and just started seeing this issue 
  • Issue stops the record from being editable using quickMARC, but you can still 'View source' of the most recent SRS record
  • Chicago sees it only on loads larger than 1000 ; MSU seeing it on loads smaller than 1000
  • Chicago not seeing this issue with SRI
  • DI log not showing these records as errors
  • MSU did a load (yesterday) of 3300 about almost half of the records had this issue
  • Issue seeing multiple ACTUALs isn't sadly a new issue
    • Maybe this is just getting worse?
    • Discussion on why this is popping up now?
    • Usually this has been related to performance issues in SRS
  • file-splitting is enabled on bugfest-poppy and at MSU 
  • Without Metadb, a library can find this problem by trying to export records using the UUIDs ; duplicate records will create an error in the export

  • Ryan : take back to team to investigate how to run tests to look for this
  • Ryan : to see if there are existing duplicates in bugfest-poppy already with this issue
  • DI Subgroup members : try to reproduce in bugfest-poppy
  • Corrie : submit FOLIO Jira ticket & post on Slack
Review/Discuss: UXPROD-4303: Set instance/bib record for deletion

Review and discuss feedback from initial testing of new 'Set record for deletion' action for Instances.

What is the expectation of being able to update a record that has been set to delete? In testing, it is possible to update an instance record that has been marked for deletion.


See notes from DI Lab sessions here and spreadsheet:  

  • "Set to delete an instance with source=FOLIO and no dependencies (requests, circulation, etc)"
    • Row 2 in the spreadsheet, 'Inv. app Action...' tab
    • Recommended to have a new marker for 'DELETE' to separate the instances from those just suppressed
    • A new 'DELETE' marker would allow querying via facets, the Lists app, Data Export, etc.  
    • Related ticket: if the LDR is set to delete, can logic be implemented that causes other related changes in FOLIO
    • Interaction and new marker would help with bulk edit of records
  • How does DI behave when the matched record for an update is marked for deletion? 
    • Row 26in the spreadsheet, 'Inv. app Action...' tab
    • Expected results? Responses : 
      • Update record, add an admin note, leave it as 'DELETE'
      • Add instructions to the job profile to designate how DI should behave
      • Once a record is marked for 'DELETE', it isn't part of the collection and should be ignored during an update
      • Need flexibility to unmark for 'DELETE' in some cases, leave as 'DELETE' in others
    • Consensus : users need to be notified when this happens in the log, with an admin note, etc.
    • Expectations vary based on current workflows employed to deal with deletions, subscription e-resources, anticipated time to keep 'soft deleted' records in FOLIO


Previous discussion notes:

 Click here to expand...
  • Ryan has reviewed the spreadsheet of feedback and shared some already with the dev team.  
  • MODSOURCE-756 : newly submitted ticket 
    • Folijet is reviewing the ticket and options for addressing it
    • Hoping to address with a simple BE update
  • In some instances, neither 'suppress' or 'delete' covers all needs.  
  • Advocation for a separate section in the 'Actions' button for the deletion actions.
  • Lots of open questions concerning dependencies (outlined on the spreadsheet).  Important for future phases of this project (not part of Phase 1).
    • Note : ensure that item status is accounted for in reviewing dependencies ; i.e. items on loan 
  • Should the default behavior be that the user is expected to delete holdings and items before the instance? i.e. dependencies aren't as important
    • Use case could be that holdings and items are marked for deletion separately; last step is to mark instance for deletion and clean all of them out at that point
    • Discussion on various internal procedures for withdrawing and deleting records
    • Practices in place in FOLIO to address the current inability to delete records leads some libraries to prefer the idea of deleting instances, holdings, and items in one step.
    • Practices can vary between libraries based on type of material: physical vs. electronic
  • No consensus on when to use a 'hard' vs. 'soft' delete.
  • Discussion on ability and need to delete the SRS & Instance separately. 
 UXPROD-4303 - Getting issue details... STATUS
NoteRyan

Next meeting is June 12th (Ryan out all next week and June 5)



Upcoming meetings/agenda topics:


Chat: