Skip to end of banner
Go to start of banner

2022-11-28 - Data Migration Subgroup Agenda and Notes

Skip to end of metadata
Go to start of metadata

You are viewing an old version of this page. View the current version.

Compare with Current View Page History

Version 1 Current »

Date

Attendees

Darsi Rueda jpnelson Carol Sterenberg Ian Walls Tod Olson Susanne Schuster 


Meeting Link

Link to Recordings

Discussion items

Time

Item

Who

Notes


Intros

newbie Susanne Schuster (BSZ)

regulars are Darsi/Jeremy (Stanford), Carol (Texas State), Ian (Bywater), Tod (Chicago), Jeff (Duke), Ingolf (hbz)


Migration status reportsDarsi/Stanford

Ian working on his migration toolkit: general purpose set of systems for migration, pieces assembled in a meaningful way. Allows user to upload data, operator to analyze the data, patterns/trends, build scratch mapping spaces to do data processing to test things out. Has capability to define workflows to upload data to source. They probably will take files from db spreadsheets, or sometimes databases themselves, and push into Folio.  Can use apis or go direct to database. 

Also useful for post-processing cleanup, bulk edits



Darsi/Jeremy

Stanford wanted to do a refresh of most data, used table truncate to blow away inventory, and disable commands on modules (which blew away data from related modules/databases too). Table truncation probably best next time.


Post-migration automation with our migration tools

Prefect being used to identify records (LDP query), dump records (Data Export job), update outside of Folio, reload records (Data Import job)

ARLEF discussion on Automation and Data Import from Friday Nov 18 12:30pm Eastern, Jenn of Cornell leading


Advice for new data migration
Darsi: load default reference data
Tod: use UUIDs from old system in the new (for data)

Action items

  •  
  • No labels