2019-07-08 - Data Migration Subgroup Agenda and Notes

0

New meeting link

Dale

Please note. We will be meeting using a new Zoom link going forward: https://zoom.us/j/204980147 The old one expired and is no longer available.

5

Welcome and assign note taker.

Dale

Welcome and request for someone to take notes, preferably, someone who hasn't done it in awhile. (@Wayne Schneider)

40

Demonstration on how to load marc records

Ann-Marie and Folijet staff

Hopefully, we will have a demonstration and discussion from the folks at Folijet on how to load marc data into Folio using the new CLI interface for data imports.

Notes:

Ann-Marie: disclaimer – the UI is not designed for data migration. The API is designed strictly to support the UI, and so may not meet data migration requirements.

@Kateryna Senchenko provided links to documentation and a demo: https://github.com/folio-org/mod-source-record-manager/blob/master/README.md#data-import-workflow

• The module to interact with to load MARC and automatically create instances with links between source record storage and inventory is mod-source-record-manager. It is possible to interact directly directly with the source record storage and inventory storage APIs to link existing records by changing the source property in the instance and adding a 999 with the UUID of the instance to the MARC record.

• Changes to the default mapping only affect inventory records going forward, they do not retrospectively re-map existing inventory records.

• Some changes to source record storage appear to have affected mod-oai-pmh; @Theodor Tolstoy (One-Group.se)will file a bug.

• Some API documentation is also in the Wiki: SourceRecordStorage tools for test data populating

New for FOLIO Q2.2:

• View source link in Inventory app now hooked up

• Significantly updated default mapping (currently being debugged)

To come: Q3 will see an implementation of pub/sub, applied to the data import application. See https://folio-org.atlassian.net/browse/UXPROD-1806.  RFC for pub/sub: https://github.com/folio-org/rfcs/pull/2.

10

Discussion

Tabled for lack of time

various

If any time is left, we will continue to discuss bulk API requirements for migration.

Some jiras of relevance: https://folio-org.atlassian.net/browse/FOLIO-1932., https://folio-org.atlassian.net/browse/FOLIO-2050, https://folio-org.atlassian.net/browse/UXPROD-1826, https://folio-org.atlassian.net/browse/MODINVSTOR-295, https://folio-org.atlassian.net/browse/MODINVSTOR-296

And for some background and discussion from other sites (cited by Anatolii Starkov):

https://evertpot.com/http/207-multi-status
https://apihandyman.io/api-design-tips-and-tricks-getting-creating-updating-or-deleting-multiple-resources-in-one-api-call/
https://medium.com/paypal-engineering/batch-an-api-to-bundle-multiple-paypal-rest-operations-6af6006e002
https://developers.google.com/gmail/api/guides/batch

This is a complex topic, and we will do well to get oriented  in this session, and discuss what is possible. We can continue with requirements and stories future sessions.

Future agenda items

Next week @Jeremy Huffwill provide a demo of workflow-based migration. @patty.wanningeralso emphasized the importance of documenting requirements for bulk data APIs – so please update the issues referenced above.