2019-09-23 Data Migration Subgroup Agenda and Notes

40

Ian's update:

3 Jiras regarding the issues

https://folio-org.atlassian.net/browse/FOLIO-2050 (API standard)

https://folio-org.atlassian.net/browse/UXPROD-1826 (Import)

https://folio-org.atlassian.net/browse/UXPROD-1928 (Export)

Jakub's/Wayne's update:

RMB does support streaming of records.

Matter of rolling out to those modules built with RMB.

Mod-Inventory is the large one to tackle.

Batch API endpoint standard not complete

            -Synchronous

            -Case for asynchronous possibly to be implemented in the business logic modules

No partial success of failure support with these two models

What exists within the platform now:

Platform provides building blocks, does not enforce a specific API design.

Provides support for streaming data in and out of database.

Platform team will roll out in Q4 streaming downloads for IDENTs/Holdings – use-case mainly for reporting.

Platform currently provides streaming uploads – up to developer to consume stream of data.

What doesn't exist within the platform now:

No Postgres client batch loading support (still needs to be addressed).

Platform team looking at new Postgres driver to allow for batch updates.

We would like to see a business logic module that supports streaming – so there doesn’t have to be a work around.

Dale would like to see going to the storage API directly. Wayne thinks this is a work-around and that it could be good-enough for migration, but not really where we want to see the system in a year.

Going forward:

For Q4 - No full blown batch support, but at least support for mod-inventory-storage. The idea is to get production-ready code in.

Most feel batch loading is more important than streaming.

The platform team needs to ensure the spike that’s out there now aligns with the expectation of the SIG. Spike needs to conclude before work is done. Start small, simplify the design.

Update on in-place migration from Folio version to Folio version:

Focus has been on verifying that the support is provided by the platform. PoC has been done, guidelines have been written as a result. The plan for the roll-out is Q4 for each of the modules to have automatic migration support.

Expectation is if you run Q3.2 that you can in-place upgrade to Q4.

Skip meeting for next week, and wait to hear about institutions loading their data in Q3.2