2023-09-18 - Data Migration Subgroup Agenda and Notes

Darsi (Stanford)


Ian (Bywater)

We’re live!  All went well, aside from Airflow server running of resources, so we slowed inventory migration a bit, still finished on time.  DB crashed during loan migration (ran out of connections, too many parallel loan processes)

St. Thomas Law Library migrated. No acq data. Small enough that inventory was done with DI and direct to storage loading. Created holdings/items with api commands.  No fines! Challenges were with setting up regular patron import. Keeping it up to date with central IT complicated; IT has different “group” for the Law folks but not separated correctly to match desired loan policies.

Ian didn’t attend any Data Migration sessions.

Workflows SIG is getting kicked off. Use cases include data migration. Need ability to move data around in configurable ways. How can we have a unified approach to make that happen.

Workflows = steps needed to get something done. Includes automating steps. Triggers can include manual start, cron, events.  Running workflows at scale. Need ability to get lists of things to do stuff to, so reporting. Or lists module.

Dashboards presentation - dashboard could become part of workflows approach.

Ian’s thoughts: Attendance is low. No other library is planning a non-vendor-supported migration. Could put the group on pause until there’s a need for this? Just do on slack? Check in on Implementers/SysOps.

Darsi’s thoughts: Small group, no sites (aside from Ingolf’s sites) doing their own data migration, everyone coming on board working with a vendor. Data migration subject could be covered in Implementer’s group meetings. Technical aspects could be a topic at occasional SysOps meetings?  And slack.

Next step: Darsi talk with Ingolf, as he convenes SysOps SIG, and this is subgroup of SysOps.