2018-09-17 - Data Migration Subgroup Agenda and Notes

The category we have called historical data does not obviously fall within the purview any particular Folio group. We will hopefully get an update on how the SIGs interpret this category of data, as well as advance the discussion, already begun, within our own group.

The category we have called historical data does not obviously fall within the purview any particular Folio group. We will hopefully get an update on how the SIGs interpret this category of data, as well as advance the discussion, already begun, within our own group.

Sharon reports on the survey she sent out to the various sigs. Some members said that these were questions that applied to individual institutions and not functional questions. The consortia sig did not need historical data in Folio as long as it was available elsewhere. Most of the issues should be handled by reporting. Patron data will have to be anonymized. MM sig was confused by task of defining historical data. Sharon counciled that each school will have to define what they regard as historical data. They were concerned about how to format historical data to allow for access. For the RA sig, Cheryl said that they had not yet completed it. Said that they need to keep things like loan counters, and maybe also item information on loans, but they did not know how to draw the boundry between identifiable data and historical data. All said that keeping the loan history would be useful. RM sig said that order and invoice records need to migrate in certain year ranges. Dale commented that Chicago was concerned that segregating out historial data may create workflow issues if that data has to be accessed in the same operations as folio data. The RM sig had trouble answering the question about maintaining the integrity of historical data. The Reporting sig will be the recipient of historical data in Sharon's view. The sig does not have answers, but know it will have an impact. No one is defining histoical data. It will be done institution by institution. Sharon thinks that these issues should be passed on to the first implementers group. Sharon wonders if migrating historical data should become part of the mission of the Data Migration subgroup. Ingolf said that if the data will be handled by the data migration loader, it will fall within our purview. If it will be loaded into the data warehouse, the data will be handled by the Reporting sig. Sharon asked about anonymizing data, whether that falls within our purview. Chery felt it did not, because that data was different for each institution, and each data set had its own problems to resolve. Dale said that the question of what do with historical data was part of the larger question of what to do with their data to put it into a form that could be loaded into Folio. Culling was part of the process of transforming that data into a form that folio could accept. Sharon reports that Nacib was concerned just to get the data formats for the data warehouse working without first delving into the issue of how to house historical data. Dale said that the issue of how to handle historical data was not just an initial migration issue because folio data itself over time is going to age and be subject to some rules about when it becomes historical has to be handled in a different fashion. Sharon did not feel that the data warehouse could support legacy data formats at least initially because of all the other work that has precedence over that. Sharon says that this should be of immediate concern to first implementers. Sharon said that she could help them to think about it. Sharon will try to answer the questions that people have emailed to them so far.

We will continue the discussion, initiated last week, on the Vendor data model and the gap analysis spreadsheet, which is here.

Uschi raised question of how to handle vendor types. Dennis said that they have supplied to different vendor types in a dropdown list. Uschi added more types which are not in Dennis' list. Uschi's system can create custom vendor types. Folio cannot do this now because the functionality is tied to hard-coded types. Dennis will take this issue back to his group. Uschi uses this for searching and reporting. Dennis wonders if tags could handle this case. Uschi thought that tags were for individual institutions and not for general folio functionality. Dale wondered if these type categories could be be assigned to folio functionality through a level of indirection. Uschi: different vendors and contacts are assigned to different faculty libraries. How do you associate different contacts with different library units and specific order with those units? Different libraries have different claiming periods as well.Dennis suggested that this discussion could be continued in the small group.

More discussion next week about vendors