2019-02-11 - Data Migration Subgroup Agenda and Notes

Owned by Dale Arntson
Last updated: Nov 20, 2020 by jroot

Date

at 11 EST

Discussion items

TimeItemWhoNotes
5WelcomeDale

Welcome and request for someone to take notes.

15Face-to-Face meetingvariousWe need to come to a consensus about whether the Data Migration group is going to skip the Face-to-Face meeting in early Summer. From last week, it seems like we were leaning in the direction of continuing our work on-line. I want to make sure that people agree with that.

Notes
Next week Patty will discuss her meetings on the data loader. Also, Matina will provide a preliminary report on discussions with the ERM subgroup on migration of ERM data. Discussion of Face to Face meeting. We should take the opportunity ot discuss the data loader with developers. Patty will be discussing with Kate about resource allocation. Some of our members will be going to Face to Face, and they can represent the interests of the group in discussion with the developers on the data loader. Members seemed to concur that that was a reasonable approach to the Face to Face. We don't have to meet there as a group, and that we can do our work on-line. Uchi suggested that we could do a session on inventory loading on-line. Ingolf said that he had tried to load some inventory data, and that he could not do it because he did not know how to manage the UUIDs. Dale said that he was managing his own UUIDs in his inventory imports direct to the database. Theodore said that he thought that was appropriate for loading whether going through the APIs or going direct to db because the reference data was coming from outside Folio, and institutions may want to use their own data in the reference tables. That is what he was doing in some cases. Dale said the Chicago was also using custom reference data. Theodore advised Ingolf to use default UUIDs if he hadn't yet settled on what his reference data should be. That way he could do his loading now and worry about establishing his reference data later. Ingolf asked how people were doing their mapping from Marc to Json. Dale said that he had written a python script to do that mapping. Sharon asked if Mod Data loader could be used for that, and Wayne said that that was very old and would not be updated. Ann H. said that she was using the Mod data loader, despite its limitations. Wayne said t. hat some people are using Katmandu to map marc to json inventory. That work was done by EBSCO. and is available on gitub under inventory sample data. Dale said that he could make his code for doing that transformation available to the group. Wayne that that that would be quite valuable. Theodore said that he could also share his code that does that transformation from Marc to inventory. Charlotte said that it would be nice to have a data migration session about how to use these tools. Ann-Marie said that her developers had been assuming that electronic resources had to have a holdings record, which turns out to not be true. They can be handled only in the instance record if desired. So they were trying to rethink how that should be coded. Among other things, that was implications for requiring location in orders, among others. Charlotte said that different libraries will want to represent e-holdings differently. They could do it just at the instance level, or at the instance/holdings level. Ann-Marie said that they have some redesigning to do to make that possible.

Link to Acquisitions Interface Fields
Link to FOLIO Record Data Elements

Action Items

  • Tod Olson Pull list of data elements and descriptions from JSON schema, focus on purchase-order and po_line schemas, send to Beltaine.
  • Tod Olson Contact User Management SIG (Maura Byrne) to find out more about Affiliation data upload plans
  • Dale Arnston will contact Khalilah and Ann-Marie attend an upcoming Data Migration subgroup meeting to discuss questions about the User-Import Endpoint tool and the Data Import tool
  • Tod Olson and Patty Wanninger to draft a Data Loaders Requirements document to for Data Migration group to discuss in 2 weeks (Dec 10)

  • Patty creates a Slack channel for Voyager resource access. She creates #migration-voyager, #migration-aleph, ... Slack channels.

  • Dale Arnston will talk with Patty about plans for discussing Data Loader user stories in a future meeting

  • Dale Arnston will follow up with Paula Sullenger and Jesse Koenneke to determine when the Data Loader Proposal can be presented to Product Council