2022-08-22 Reporting SIG Meeting notes









Arthur AguileraUniversity of Colorado, Boulder
Linda MillerCornell University

Sharon BeltaineCornell University
Nassib NassarIndex Data

Erin BlockUniversity of Colorado, Boulder
Elena O'MalleyEmerson

Nancy BolducCornell UniversityXTod OlsonUniversity of Chicago

Shannon BurkeTexas A&M

Suzette CanedaStanford University
Jean PajerekCornell University

Lloyd ChittendenMarmotXMichael PatrickThe University of Alabama
XTim DannayMount Holyoke College
Eric PenningtonTexas A&M

Axel DoerrerUniversity Mainz
Scott PerryUniversity of Chicago

Shelley DoljackStanford UniversityXNatalya PikulikCornell University
XStefan DombekLeipzig UniversityXVandana ShahCornell University

Jennifer EustisU. Massachusetts Amherst / Five College
Linnea ShiehStanford University
       XLisa F.Texas A&M

Alissa HafeleStanford University
Clare SpitzerStanford University

Jeanette KalchikStanford University
Amelia SuttonU. Massachusetts

Kevin KishimotoStanford University
Simona TabacaruTexas A&M
XIngolf Kusshbz
Huey-Ning Tan

Stanford University

Joanne LearyCornell University
Vitus TangStanford University

Eliana LimaFenway Library Organization
Irina TrapidoStanford University

Alexander LaoStanford University
Kevin WalkerThe University of Alabama

Eric LuhrsLehigh UniversityXAngela ZossDuke University

Discussion Items




Attendance & NotesAngela

Attendance & Notes

  • Today's attendance-taker: Linda (or substitute)
  • Today's note-takers:  Team Leads for project updates

Announcements /


Reporting Documentation Subgroup needs new leader

  • No need to have extensive experience with the SIG or the FOLIO reporting systems at this time! This is a great way to learn about everything!
  • Please reach out to Axel Dörrer or Sharon Beltaine if you might be interested.
  • This team may also evolve to take on some training tasks, so we are looking for at least a couple of new people

Updates to Reporting SIG Wiki

WOLFCon 2022

How to find our latest recordings

(Always) Recruiting New Query Developers

  • The Reporting SIG is always on the look-out for new query developers. Please let us know if you are interested in doing query development or if there are others at your institution who might be a good fit.

Discussion on possible changes to FOLIO Analytics documentation, report queriesAll
  • In the reporting development meetings, we talked a bit about developing more of a template or recipe model for report queries
  • Goal would be to help people get up and running with reporting quickly, provide tips and code snippets, but not spend a lot of time writing queries that aren't getting used
  • Another model might be an FAQ - start with a specific set of questions people want to answer, then share examples of how to do that
  • What other models might we consider? Any good examples you know about? Are there pros and cons to different approaches?
  • Discussion
    • ERM has a plan to create a tutorial for beginners in ERM; would like to collect information into one presentation (Google Slides); we have a lot already but want to bring it all together. It should be a good first step to reference in our tutorials, so beginners can read the information for each area of reporting.
    • Also plan to transform the ERM group content to have more lab time in the meetings; create new areas, eUsage and Open Access; would also like to talk about specific problems user have
    • The code snippets and tips will be useful for all. Also an FAQ for commonly asked questions.
    • Maybe we do need some kind of "Quickstart" on the repo, and then encourage people to go through the full training presentation after that
    • What is especially tricky for newcomers?
      • Explaining what a derived table is
      • JSON scripting
      • concept of ids in the tables that don't have a name (e.g., id in loans table is the loan_id); other systems, they're all labelled
      • cross-app connections in ERM
      • how tables connect - maybe a comprehensive table of the joins between tables
    • LDLite is very complex to begin with, even to install (easy on Unix, but Windows much harder) - maybe need better documentation about installation, or a walkthrough
Finishing the Reporting SIG vision: mission statementAll

After all of our visioning work, let's summarize it with a short mission statement. What really rises to the top as our primary mission?

  • writing SQL reports
  • conceptualizing what reporting should look like
  • training as people continue to join the group
  • sharing application information (e.g., DBeaver)
  • sharing installation experiences
  • documentation
  • FOLIO Reporting SIG works to develop ERM-related sample report queries and derived table queries for the folio-analytics Git repository and discuss all related topics around reporting. Documentation, FAQ ...
  • problem solving (SQL, tools, etc.)
  • offer a space for technical and non-technical people to talk about reporting
Updates and Query Demonstrations from Various Reporting Related Groups and EffortsCommunity & Coordination, Reporting Subgroup Leads

Project updates

Reporting development is using small subgroups to address priorities and complete work on report queries.  Each week, these groups will share reports/queries with the Reporting SIG.  Reporting development team leads are encouraged to enter a summary of their work group activities below.

RA/UM Working Group

MM Working Group

  • Meetings are 1st Tuesday of the month, 12-1pm ET via zoom using the usual FOLIO password. Our lab sessions are open to everyone. Please bring your questions, examples, and comments about reporting and metadata.
  • We are still looking for reviewers. Please contact MM reports working group.
  • Our next meeting will be focused on working with marc srs and folio data in particular for use cases related to Stanford reports.

ERM Working Group

  • ERM Prototype and Query Development Status
    • folio-analytics version 1.5:
      • add comments to columns in all derived tables
      • We will revise the directory structure for report queries within erm on GitHub (metadb) to be consistent with the repository rules.
  • eUsage
    • Goal: a derived table.
    • The development requires time and expertise from SME.
      • We're going to take it in phases:
        1. Get test data and enter it into our test environment. started
        2. Analyze data structure.
        3. Invite SME and have data explained.
        4. Creation of derived tables
  • Topics for the future
    • Open Access
  • Documentation / tutorials / lab sessions
    • There are slides. They will merged and improved. We would like one presentation for beginners (onboarding). This slides will uploaded into the directory for all presentations.
    • In planning: In addition to the documentation, tutorials will be created.
    • Group transformation: In addition to the specific developments, meetings will have become more of a lab session, working through specific problems.
  • Meetings are bi-weekly on tuesdays 11am ET alternating with RM Working Group
    • Next meeting will be at 23th, Aug together with RM in one meeting
    • Contact Stefan Dombek if you would like to get a calendar invitation

RM Working Group

Reporting SIG Documentation Subgroup

  • Honeysuckle documentation is live on https://docs.folio.org/docs/
  • Iris documentation is in progress, due December 15
  • Additional Context
    • The Reporting SIG has representation on the Documentation Working Group, which is building end-user documentation for https://docs.folio.org/docs/ (mostly linking to existing documentation over on GitHub)

External Statistics Working Group

  • no updates currently
  • new organizational/tracking scheme for JIRA, with pointers to queries in folio-analytics repository
  • New organizational structure for External Statistics reports
    • external statistics reports (e.g., ACRL) typically require running queries from different functional reporting areas
    • these reports will be captured in JIRA under one UXPROD-XXXX report cluster issue, then the descriptions will point to each of the queries required to run them on the folio-analytics repository
    • institutions will need to rank each of these 8 new UXPROD-XXXX report cluster issues
    • each reporting development team will take responsibility for the queries in their area for the external statistics clusters

Product Council

For all recent work on FOLIO Reporting SQL development:

Topics for Future MeetingsAll
  • How to deal with External Stats reports?
    • maybe subteam leads check in about that
    • probably wait until after Metadb conversion is more complete
  • ask for presenters: hosting experiences from implementers
  • Annual Reporting Goals
    • (in progress) Support the transition from LDP to Metadb (e.g., update derived table and report queries, update documentation, outreach, new training)
    • (ready to start) Developing training/onboarding for new SIG members/report users (esp. FOLIO-specific data model and transformation stuff)
    • (in progress) Develop a FOLIO Reporting Vision and Strategy
      • secondary goal: come up with language that can be used to explain to our institutions the importance of the work and of devoting resources
    • Improve communication between SIG and developers of apps so we hear about data model changes in advance
    • continued advocacy on part of the SIG to governance groups
    • (ready to start) Review JIRA issues, clean up, revisit strategy for JIRA
  • Additional data model training
  • Regular review of Milestones
  • Exploring new recruitment/onboarding strategies (e.g., buddy system)

Review and update Topics for Future Reporting SIG Meetings 

  • A test Action Item (Ingolf)