2018-04-09 - Data Migration Subgroup Notes

 Matt described his composite user spreadsheet (tab CompositeUserElements on link under item) Matt worked from the JSON composite user object which describes what kind of property and data type each element is. It also describes required data and expresses some dependencies. The JSON composite user object includes permissions; Anne Highsmith questioned whether permissions would be migrated. Charlotte said that permissions are not yet fully scaled out. Tod thought a tab with permissions would be for comparison rather than for porting the data. Matt said permissions don't map to user properties just not in the user data object. Dale remarked that we are trying to flatten a hierarchical data structure. Going from relational to semi-structured is hard to represent. e,g, the patron table in Aleph is connected to a separate address table. So showing one element to one element is misleading.

Patty remarked that the UI might have limitations on phone number and address, though Wayne said the back end is not limited; we used JSON schema to describe the data structure, i.e., data is in X format, some fields are required, but JSON itself doesn't impose limits (And Charlotte checked and currently the UI is allowing more than two addresses so there you go.)

Matt said that related to PHONE - as a JSON object, it doesn't care, but the UI specifies only two types of phones, "Phone" and "Mobile Phone."

Matt clarified that the JSON Object Composite User has 5 properties and and permissions is one of the properties. Wayne explained from the data structure perspective, there are a few *interfaces* in play:

`users`: provides users (what Matt called "user data"), groups, address types, and proxies. Implemented in the `mod-users` module.
`login`: provides credentials. Implemented in the `mod-login` module.
`permissions`: provides permissions. Implemented in the `mod-permissions` module

The interface `users-bl` (implemented in `mod-users-bl`) pulls these things together into a composite user.

The link to the https://github.com/folio-org/mod-users-bl was added to Matt's spreadsheet.

Christie Thomas from the MM SIG will discuss a new  composite inventory spreadsheet she started with Dale Arntson on Inventory data, on the model of the composity user spreadsheet.

Christie Thomas

Christie displayed the spreadsheet she and Dale worked on for mapping FOLIO to OLE fields She began with the fields from the MM SIG's Marc mapping document (https://docs.google.com/spreadsheets/d/1kdYx63J0KoqR3-LUHuPAzERgj8WE0OQ08rzuCaJaHWs/edit#gid=952741439)

.In red, she noted fields that do not need to be migrated. In purple, she noted fields that may be stored elsewhwere and may not need to be stored in inventory ultimately. OLE unfortunately has live data in a number of places, which causes problems and we would prefer not to repeat in FOLIO.

If the data is in FOLIO with no corresponding OLE data, the OLE field is blank.

Christie is wondering about processes like "fast adds" which get flagged for review, info about donors, bookplate info.

Great to get these tables made for all sources so gap analysis can take place at WOLFcon.

Anne has started for Voyager, Ingolf will provide his and Theodor will contribute Sierra.

Then we can highlight missing data and refer back to SIGS.

Charlotte pointed out a discuss post on this topic. https://discuss.folio.org/t/bib-records-required-in-inventory-to-create-orders/1737/5

Charlotte said the inventory module is having elements added, so there is "some urgency." Want to implement by summer.

Sharon mentioned interest in the part of the reporting SIG to be sure the fields they need to report on are there.

A discussion followed on building a separate module for migrating inventory data, separate from bulk loading. Rather than the API we already have. This module could wrangle the UUID.

Christie asked where the MARC data resides, and Wayne said some design of the cataloging app and inventory app will include a MARCstore of some kind. Could be in Cat mod and then the Inventory view would be read-only.

Team members will continue to fill out the inventory sheet so the gap analysis can be discussed at WOLFcon.

For next week we will confirm that we are done with the User.