| Product Council SIG check-in | Jesse & Owen | At WOLFcon, PC decided to revitalize connections to SIGs. Starting with a check-in - what are you currently working on, what's going well, what are your challenges? (notes from other check-ins) - Can you share the purpose of your SIG: functional area/cross cutting/affinity group, etc?
- What areas does your SIG have responsibility for?
- see statement of scope
- there have been discussions of in-app reports; the Reporting SIG doesn't consider ourselves responsible for in-app reports, but that might be something there could be more discussion about
- we have been involved in sharing initial requirements for in-app reports and do try to avoid duplicating effort with in-app reports
- LDP app exists, though, so maybe that is similar to in-app reporting
- we are open to being part of more discussions about in-app reporting, would be happy to consult/offer suggestions
- What do you feel like your SIGs purpose is currently? Has it changed over time?
- at first we thought we were here to roll out ready reports for FOLIO users; realized that different institutions use FOLIO differently enough that it is hard to create canned reports; transitioning from canned reports to more training about how to use FOLIO data and design their own custom reports
- in the beginning, started with spreadsheet of reports we need, but wasn't clear who would develop them, so we ended up having to develop them ourselves; but then it turned out we only will provide SQL; the project has also changed from data lake to now the library data platform
- are doing a lot more onboarding and training for people coming onto the project; have many new members. but throughout we have been advocates, especially around reporting being core to FOLIO and having equitable access to reporting
- Do you have product owners who regularly work with the SIG? Do you feel that your requirements are being successfully developed into FOLIO?
- Charlotte - the PO for LDP query builder app, among other things; reporting working groups might communicate with POs from other SIGs
- requirements successfully developed? mixed. Reporting is seen as ad hoc, so that is an issue. There is a tension between what is developed by the vendor and what is open source, what is and is not core. Confused about what about reporting is part of core.
- Nassib is our Product Owner for the LDP project. He works with the SIG to hear what features are needed and develop them. We do have the LDP app integrated into FOLIO release schedule, but it is independently developed and the LDP platform is also independently developed. It's always been a struggle to get reporting part of FOLIO development. There is competition among vendors that has made it hard for institutions to receive equitable access to reporting. We're clear about what we want to develop but don't always have the resources or hook to get the development resources.
- In ERM we have good connect to Owen Stephens, who helped a lot with the data schema. He will also be collaborating on future projects. We invited him via Slack to our meetings, and it's a good way to gather information about different areas in different working groups, inviting POs to subgroup meetings. Can ask them directly.
- How do you feel about the resources allocated in the development areas that are focused on?
- probably answered; it stems from the confusion about where reporting fits in. We don't have a FOLIO development team, and if we did and reporting was core, that would be great. But we don't.
- All development is funded by Index Data.
- Never expected SIG membership would have to do as much development as it is. SIG members have had to pick up the slack, and it's been tiring. We have persevered because we believe in the need for a reporting product. Many people have worked hard to make this a reality. But it would be great if we had dedicated development resources from FOLIO core.
- What are some of the challenges facing your SIG/subgroup? Are there ways in which you feel the SIG has been unable to accomplish its mission?
- There is misinformation about security and performance of reporting solutions that is a challenge
- Through reporting, we find data inconsistencies and inaccuracies; that's normal, but communicating those to the appropriate person is hard; something can be named one way in the UI and a different way in the JSON; would be great to have help navigating those waters and making sure the problems are brought to the right people
- PC might be in a good position to help us find out what people need in terms of hosting support and working with vendors to find ways to provide equitable access
- Hearing in advance about data model and business logic changes that might impact reporting and getting a chance to discuss
- Often there are changes that go into production and aren't even discussed with the SIGs directly involved, they just get changed by developers; sometimes they aren't even described in release notes
- Finding out about functional changes late has had an impact on Cornell; there are project management solutions for this - change control processes are commonplace in PM; would like to see FOLIO do that because things do break when things get changed without notice
- Problem is that we have some libraries that are going live now, and they are using LDP or Metadb software, and the software contains derived tables which are built on our queries; if the data model changes, our queries break, and then the LDP/Metadb overnight updates break; need advanced notice so that we can update reports so they match changes in the FOLIO release
- This group does not see some of the operations side of vendor/hosting issues; the way information is streamed turns out to be very expensive in AWS; there are assumptions about why platforms may or may not be available, but some of the concerns about hosting may be practical concerns about how to move data between platforms and how much money it actually takes; it is a complicated question; I personally worry about injecting ourselves in the discussions
- But if people are getting different information, maybe some mediation or transparency around those issues
- If you work on areas of development, do you feel like there are areas of development not resourced at all or significantly under-resourced?
- LDP platform development from Index Data is under-resourced, would like to find a solution with the FOLIO community
- What has been successful with the SIG? What things contributed to that success?
- The SIG has accumulated a lot of knowledge about the data models in FOLIO; that may be the most broad view of the data scape, and that applies across whatever reporting tools one adopts; so maybe the group could be more of a broad resource on reporting than it has been; we've focused on development for the LDP, but it's based on knowledge about how the data fit together, and the group is thus very useful for sharing insight on the FOLIO data
- There are many ways the SIG has been successful; volunteer efforts from many different institutions have contributed enormously to our success; the openness of the LDP software development - developer listening to what the SIG needs and developing in response to that - has provided users with solutions they needed. The availability of report templates and expertise for institutions who are getting started so they have something they can depend on.
- training and skill building opportunities
- examples: availability of ETL for MARC data; also the way that the SIG has worked with the rest of the project to meet data privacy requirements for European institutions
- What could the Product Council do to help the SIG?
- advocate for (production-ready) reporting functionality to be core to FOLIO and provide resources for its development
- any help with any of the other challenges
- the in-app reports are just for that app; what we need for operations is cross-app reporting, and FOLIO doesn't really do that. Bulk Edit app has been proposed to do that, but it has a totally different purpose, not meant to do that. LDP is the only cross-app reporting solution, so what should be core is a cross-app reporting solution. Also, accessing the SRS, which is currently only accessible through the API or an LDP product. There is no app for SRS.
- Do you feel like the Roadmap accurately represents the work of your SIG (both desired and already accounted for)?
- all types of reporting are listing for all times, and that seems accurate
- could probably clarify the subdivisions of reporting on the roadmap, and might also want to make a distinction between SIG member efforts and independent development teams
- How can we improve our communications with the SIG? Does the current monthly reporting work? Would have a dedicated PC meeting for your SIG be helpful?
- yes, probably would like a dedicated meeting; SIG members probably don't get much info about PC, though, since it comes through convener and she is bad about that
- Is the SIG comfortable communicating with the PC and have a sense of when a PC conversation would be helpful?
- seems like conveners have been able to raise issues as needed
- What functions should the PC be accomplishing and how would the SIGs like to have those need met?
- I think because we're knee-deep in reporting, we're not on top of governance and how the different teams work together to solve problems; having any of those groups work with us on challenges would be helpful (PC or otherwise); even just helping us understand how to work with the governance teams
- maybe an annual report, a summary of what the different teams are up to?
|
| Updates and Query Demonstrations from Various Reporting Related Groups and Efforts | Community & Coordination, Reporting Subgroup Leads | Project updates Reporting development is using small subgroups to address priorities and complete work on report queries. Each week, these groups will share reports/queries with the Reporting SIG. Reporting development team leads are encouraged to enter a summary of their work group activities below. RA/UM Working Group
MM Working Group - Meetings are 1st Tuesday of the month, 12-1pm ET via zoom using the usual FOLIO password. Our lab sessions are open to everyone. Please bring your questions, examples, and comments about reporting and metadata. MM notes
- We reviewed the consistency of our LDP and metaDB derived tables which is looking good.
ERM Working Group - FOLIO Data Model Training (in progress)
- ERM Query Development Status
- Open Access
- Report requests are being collected in the OA SIG Link (in progress)
- The test environment from the SIG Reporting will get the app OA (in progress)
- Test data will imported
- Meetings are bi-weekly on tuesdays 11am ET alternating with RM Working Group
- Next meeting will be at 6th, Dec
- Contact Stefan Dombek if you would like to get a calendar invitation
RM Working Group
Reporting SIG Documentation Subgroup - Morning Glory documentation is live on https://docs.folio.org/docs/
- Nolana documentation is in review
- Orchid documentation will be in progress soon, and plans are underway to include beta-level documentation for Metadb
- Additional Context
- The Reporting SIG has representation on the Documentation Working Group, which is building end-user documentation for https://docs.folio.org/docs/ (mostly linking to existing documentation over on GitHub)
External Statistics Working Group - no updates currently
- new organizational/tracking scheme for JIRA, with pointers to queries in folio-analytics repository
- New organizational structure for External Statistics reports
- external statistics reports (e.g., ACRL) typically require running queries from different functional reporting areas
- these reports will be captured in JIRA under one UXPROD-XXXX report cluster issue, then the descriptions will point to each of the queries required to run them on the folio-analytics repository
- institutions will need to rank each of these 8 new UXPROD-XXXX report cluster issues
- each reporting development team will take responsibility for the queries in their area for the external statistics clusters
Product Council
For all recent work on FOLIO Reporting SQL development:
|