2019-08-05 - Data Migration Subgroup Agenda and Notes

*There will not be a meeting on Monday 8-12-19

Date

05 Aug 2019 at 11 EST

Attendees

Link to meeting: https://zoom.us/j/204980147

Discussion items

Time	Item	Who	Notes
5	Welcome and assign note taker.	Dale	Welcome and request for someone to take notes, preferably, someone who hasn't done it in awhile. Michelle will take notes.
20	Early implementers requirements	Tod	Report from Tod on whether any early implementers need to do a gap load in their migration to Folio: Early Implementer meeting notes with discussion details – see "Survey about migration scenarios - "Big Bang" vs. "Trickle" Migration plans" General discussion about: 2 phase migration...e.g. bib records first...and frozen, are API deletes really needed (for migration)? ...is gap loading needed? The 'MVP' may require a larger downtime during migration in the spirit of a true MVP. Acceptable since it is an MVP? Mention of a requirement for sync. between a union catalog and FOLIO and union catalog will be integrated as a knowledge base (Ingolf) – similar to what Chalmers is doing. There may be some potential for re-use of the Chalmers functionality. Chalmers use case is exchanging 'inventory' records.
35	Data Loader	Jon	Jon may give us a brief update on improvements to the data loader that he showed us previously. Improvements to the general data loader (the updated version is on github - Jon will post a link in slack): -save -validate -force -all -- will print out all of the validation errors.... -save flag with the new -whatif flag will just validate data and won't save to file Jon is working on a different data loader that: -Loads data directly from relational database sub-tables (like email, phones addresses - for organizations) are loaded into hash tables ...then the main loop builds everything up into an object graph currently working for smaller objects like organizations - the goal is to generate templates - he wants to get the code to the point where all you have to do is replace the default SQL queries If there is interest he'll demo it in a few weeks
N/A	Bulk API review	We did not discuss this today →	Continued discussion on bulk API requirements. Some jiras of relevance: How to design batch API (General recommendations), UXPROD-1826 - APIs for batch uploads and downloads Draft FOLIO-2050 - SPIKE Design batch create / update API endpoint standard Draft UXPROD-1826 - APIs for batch uploads and downloads Draft MODINVSTOR-295 - streaming POST holdings Open MODINVSTOR-296 - streaming POST items Open And for some background and discussion from other sites (cited by Anatolii Starkov): https://evertpot.com/http/207-multi-status https://apihandyman.io/api-design-tips-and-tricks-getting-creating-updating-or-deleting-multiple-resources-in-one-api-call/ https://medium.com/paypal-engineering/batch-an-api-to-bundle-multiple-paypal-rest-operations-6af6006e002 https://developers.google.com/gmail/api/guides/batch
			**No meeting next week (8/12). The next meeting will be held on Monday 8-19

Link to Acquisitions Interface Fields
Link to FOLIO Record Data Elements (contains links to specific spreadsheets, but most of them are not up to date.)

Action Items

Request for migration plans of Lehigh, TAMU and Cornell to be added to this document. (Michelle Suranofsky Christopher Creswell )