2018-04-09 Reporting SIG Notes

Previous Notetaker: Karen Newbery

Today's Notetaker: Anne Highsmith

KN: Holly is stepping in to help with the project manager role for the time being.

Please review tags spreadsheet and provide feedback.

Review Topics for Future Reporting SIG Meetings and send Sharon an email or add your ideas. 

Need a convener for 4/23/18 – Focus on Loans Reports with Emma Boettcher, Loans PO

Jeremy Huff (TAMU Dev) and Frances Webb (Cornell Dev) will join this meeting to discuss JIRA ticket OKAPI-570 , which has been entered into the Folio Issue Tracker to capture recent discussions about concerns related to the approach taken in JIRA tickets UX-PROD-331 and UX-PROD-332. These are development tickets for building the technical infrastructure to support Folio Data Lake environments. OKAPI-570 details concerns and offers alternative proposals for the implementation of reporting features residing in OKAPI’s core, dereferencing at the time of report generation, the impact of schema mutation on report creation, and implementing message queues as an internal component of reporting and/or OKAPI core.

-WolfCon: We have requested slots at WolfCon to discuss this in more depth with developers. What are the most important questions to address during this session?

Notes from the discussion on 20180409:

• Jeremy Huff reviewed the points covered in JIRA ticket OKAPI-570, which he created. Jeremy felt that the bulk of the concerns expressed in JIRA ticket OKAPI-570 were resolved during a discussion that he and other TAMU developers had with Vince Bareau in response to the creation of the ticket. He reviewed the initial concerns and VBareau's response to each
• The one concern that remained unresolved to some extent was the question of when and how de-referencing should be done. Sharon suggested that this would be a good topic to include in the Data Confidence session that the Reporting SIG has requested for WOLFCon. Jeremy will type up some ideas he has on this issue to contribute to that discussion.
• TOlson pointed out that there are 2 kinds of reporting under the SIG's remit: transactional reporting and data confidence reporting. He said the latter type might not belong in the data lake, especially for an institution running its own system. Someone in a hosting environment would have to solve the data confidence reporting issue in a different way.
• IKuss asked why personal information would be put into the data lake at all; participants agreed that there are definitely data that would never be dereferenced so that appropriate privacy can be maintained. Ingolf also asked if it would be possible to have more than 1 data lake; Jeremy replied that it would be a matter of implementation, but that theoretically it is entirely possible to have more than 1 data lake.
• Sharon talked about how the SIG could go forward from here and take next steps at WOLFCon. Possible topics include:
  • Discussion of the "data river", the "module" within FOLIO that would act as the outbound filter, making it possible to write data to the data lake without severely impacting Okapi
  • Bring in devs from different functional areas about how the schema(s) need to be defined/standardized to support reporting. This point generated a lot of discussion about schemas, how tightly they should be defined; whether and how they should be versioned, etc. Frances Webb held that if the json schema were accepted as a requirement, it would simply be as another piece of metadata that is used to describe the module and that it doesn't require that a module be written in a particular way or that the schema even be an integral part of the module.
  • How to deal with private data
  • The unresolved issue of when to de-reference and how much de-referencing should be done at the point of the transaction raises a point for discussion at the Data Confidence session: how can we examine this issue of de-referencing with an eye toward maintaining acceptable performance in the operational system.

JHuff will do some more documenting, to indicate where VBareau's responses have resolved the points raised in JIRA ticket OKAPI-570 and where there are still areas of discussion.

*** Topic saved for next meeting

Review current plans for WolfCon and brainstorm ways Reporting SIG can make best use of the conference time and resources.

WolfCon organizers are working to make conference sessions available by ZOOM so all project participants may join WolfCon sessions remotely.

KN:Open in Google sheets to view the entire schedule.From 2-4 on Tuesday (Track 3) the Reporting SIG has time with the developers.

Mike highly recommends that we have a SIG meeting during WOLFcon to discuss how central reporting is to our work.

Sharon thinks that it would be good for the Reporting SIG to meet with other SIGS regarding in-app reporting. Will request a 2 hour block. External reporting - meet with developers to discuss data needs in the data lake, data elements needed in reports. Please let Sharon know if there are more topics we should consider so we can make the most of WOLFcon.

WolfCon Topic Proposals: https://docs.google.com/spreadsheets/d/17JmVl-XUaALDYtyqEzPGGYyHAbH8JQM63XPdl9ZxKFU/edit?usp=sharing
Current WolfCon Schedule (View Only): https://docs.google.com/spreadsheets/d/1jmjRAKBXJN2i1qPLOjdbeqi5xndTPyKzrDKuB8MeeBQ/edit?usp=sharing

*** Topic saved for next meeting

We will discuss strategies for translating reporting requirements into reporting deliverables

-identifying data elements needed for each report

-working with SIGs to identify candidates for in app reports

-example: Emma Boettcher, Loans PO, circ reports on loans

KN: See Reporting SIG master spreadsheet: https://docs.google.com/spreadsheets/d/1svUM74Dkg4KvTXLzKZK_2k_SxeukX-87NnYf8CaTrYQ/edit#gid=312878932

We've identified lots of good information - how do we move forward? For within-app, suggestion that someone takes a report and the data elements that are needed, work with appropriate SIG to make sure the data points are available to create the report. For example, Acq_resource tab/ID251 work with Acquisitions to see if the data points are available, then work with PO for Acq and PO for Reporting to create user story supporting need for the requested data points.

For Data Warehouse effort, we have other reports that are going to be cross-application, by focusing on parallel in-app and Data Warehouse, we will understand more.

Proposal: have one or more contact person attached to each report in inventory list and then having the contact person meet with the involved SIGs.

Thoughts: Sounds similar to what Data Migration group is working on. Perhaps there's overlap with what we're trying to achieve. Organize reporting SIGs contacts per application? At least have a point person, and perhaps hold separate meetings to discuss reporting with the appropriate application SIG. Perhaps ask people who filled out the reporting needs on our spreadsheet to provide 1-2 contacts for a point people for that report from the application SIGs.

It's important that we bring in a broad range of voices to discuss in-app and cross-app reporting. This proposal gives us a foundation to do just that.

Sharon will help with documentation so we can proceed. Example of cross-app and cross-system ID403 resource_access tab (Circulation, Cataloging and Ares). For those that are cross-system, make sure we have documented data points.

Concerns about amount of work that needs to be done - how to organize this work? Sharon will try to start organization and coordination. We're at a point where we need to start working with the SIGs more closely to be sure we have the data elements we need. We can support this process at WOLFcon.

The Resource Access Loans PO has already contacted Sharon to ask about loan reports.

Add Report Requirements Contacts in Reporting SIG Master Spreadsheet

-contacts need to be reachable, if not part of Folio project, please add person's email address