2022-09-01 Reporting SIG Meeting notes









xArthur AguileraUniversity of Colorado, Boulder
Eric LuhrsLehigh University

Sharon BeltaineCornell UniversityxLinda MillerCornell University

Erin BlockUniversity of Colorado, Boulder
Nassib NassarIndex Data

Nancy BolducCornell University
Elena O'MalleyEmerson

Shannon BurkeTexas A&M
Tod OlsonUniversity of Chicago

Suzette CanedaStanford UniversityxJean PajerekCornell University

Lloyd ChittendenMarmot
Michael PatrickThe University of Alabama

Tim DannayMount Holyoke College
Eric PenningtonTexas A&M

Axel DoerrerUniversity Mainz
Scott PerryUniversity of Chicago

Shelley DoljackStanford UniversityxNatalya PikulikCornell University

Stefan DombekLeipzig University
Vandana ShahCornell University

Jennifer EustisU. Massachusetts Amherst / Five CollegexLinnea ShiehStanford University
       Lisa F.Texas A&M
Clare SpitzerStanford University

Alissa HafeleStanford University
Amelia SuttonU. Massachusetts

Jeanette KalchikStanford University
Simona TabacaruTexas A&M
xKevin KishimotoStanford University
Huey-Ning Tan

Stanford University

Ingolf Kusshbz
Vitus TangStanford University

Joanne LearyCornell UniversityxIrina TrapidoStanford University
xEliana LimaFenway Library Organization
Kevin WalkerThe University of Alabama
xAlexander LaoStanford UniversityxAngela ZossDuke University

Discussion Items




Attendance & NotesAngela

Attendance & Notes

  • Today's attendance-taker: Linda (or substitute)
  • Today's note-takers:  Team Leads for project updates

Announcements /


How to find our latest recordings

(Always) Recruiting New Query Developers

  • The Reporting SIG is always on the look-out for new query developers. Please let us know if you are interested in doing query development or if there are others at your institution who might be a good fit.

Discussion on possible changes to FOLIO Analytics documentation, report queriesAll
  • In the reporting development meetings, we talked a bit about developing more of a template or recipe model for report queries
  • Goal would be to help people get up and running with reporting quickly, provide tips and code snippets, but not spend a lot of time writing queries that aren't getting used
  • Another model might be an FAQ - start with a specific set of questions people want to answer, then share examples of how to do that
  • What other models might we consider? Any good examples you know about? Are there pros and cons to different approaches?
  • Discussion
    • one of the cases - trying to work with circulation tables, don't know what each table does; just want to basically know what each table does, what you would find in it, how you use them together, what are the related derived tables
    • less clicks is key, everything on GitHub is very nested
    • also very confused by the different schema - didn't know what to do with the history schema, different from historical records in other systems 
    • for the documentation, are you planning to host it on GitHub itself? or separately?
      • yes probably?
    • maybe different formats - text and video
    • for Koha, rely on SchemaSpy because it has field-level comments that show up nicely
    • we do try to add comments to both derived tables and public schema tables, but coverage is spotty and the public schema tables are hard to pin down (they change a lot)
    • should we try to do one long page with all of schemata, tables, and fields?
Finishing the Reporting SIG vision: mission statementAll

"The Reporting SIG will examine the reporting needs of institutions, offer advice on the design of FOLIO data models and reporting solutions, contribute time to the development of sample queries, provide documentation and training on topics related to FOLIO reporting, and offer a space for subject matter experts and technical experts to engage in discussion and collaboration."

  • very beefy. too beefy?
  • will run this by the Monday group/Wednesday devs
  • where would it go?
  • probably on About the SIG page, along with statement of scope; roadmap would be a subpage of that page
Ad hoc trainingAngela
  • Any questions we can cover in SIG today?
  • Examples: how to run a report query, how to write a simple query, how to review a proposed query on GitHub
Updates and Query Demonstrations from Various Reporting Related Groups and EffortsCommunity & Coordination, Reporting Subgroup Leads

Project updates

Reporting development is using small subgroups to address priorities and complete work on report queries.  Each week, these groups will share reports/queries with the Reporting SIG.  Reporting development team leads are encouraged to enter a summary of their work group activities below.

RA/UM Working Group

MM Working Group

  • Meetings are 1st Tuesday of the month, 12-1pm ET via zoom using the usual FOLIO password. Our lab sessions are open to everyone. Please bring your questions, examples, and comments about reporting and metadata.
  • We are still looking for reviewers. Please contact MM reports working group.
  • Our next meeting will be focused on working with marc srs and folio data in particular for use cases related to Stanford reports.

ERM Working Group

  • Group transformation / Documentation
    • Group transformation: In addition to the specific developments, meetings will have become more of a lab session, working through specific problems.
    • We started creating a tutorial like RA/UM together with RM wg.
  • ERM Prototype and Query Development Status
    • folio-analytics version 1.5:
      • add comments to columns in all derived tables
      • Fixes
    • New directory structure for report queries in folio-analytics (metadb) with PR#666
  • eUsage
    • Goal: a derived table.
    • The development requires time and expertise from SME.
      • We're going to take it in phases:
        1. Get test data and enter it into our test environment. started
        2. Analyze data structure.
        3. Invite SME and have data explained.
        4. Creation of derived tables
  • Topics for the future
    • Open Access
  • Meetings are bi-weekly on tuesdays 11am ET alternating with RM Working Group
    • Next meeting will be at 13th, Sept
    • Contact Stefan Dombek if you would like to get a calendar invitation

RM Working Group

Reporting SIG Documentation Subgroup

  • Lotus documentation is live on https://docs.folio.org/docs/
  • Morning Glory documentation is complete and submitted
  • Nolana documentation will be in progress soon
  • Additional Context
    • The Reporting SIG has representation on the Documentation Working Group, which is building end-user documentation for https://docs.folio.org/docs/ (mostly linking to existing documentation over on GitHub)

External Statistics Working Group

  • no updates currently
  • new organizational/tracking scheme for JIRA, with pointers to queries in folio-analytics repository
  • New organizational structure for External Statistics reports
    • external statistics reports (e.g., ACRL) typically require running queries from different functional reporting areas
    • these reports will be captured in JIRA under one UXPROD-XXXX report cluster issue, then the descriptions will point to each of the queries required to run them on the folio-analytics repository
    • institutions will need to rank each of these 8 new UXPROD-XXXX report cluster issues
    • each reporting development team will take responsibility for the queries in their area for the external statistics clusters

Product Council

For all recent work on FOLIO Reporting SQL development:

Topics for Future MeetingsAll
  • How to deal with External Stats reports?
    • maybe subteam leads check in about that
    • probably wait until after Metadb conversion is more complete
  • ask for presenters: hosting experiences from implementers
  • Annual Reporting Goals
    • (in progress) Support the transition from LDP to Metadb (e.g., update derived table and report queries, update documentation, outreach, new training)
    • (ready to start) Developing training/onboarding for new SIG members/report users (esp. FOLIO-specific data model and transformation stuff)
    • (in progress) Develop a FOLIO Reporting Vision and Strategy
      • secondary goal: come up with language that can be used to explain to our institutions the importance of the work and of devoting resources
    • Improve communication between SIG and developers of apps so we hear about data model changes in advance
    • continued advocacy on part of the SIG to governance groups
    • (ready to start) Review JIRA issues, clean up, revisit strategy for JIRA
  • Additional data model training
  • Regular review of Milestones
  • Exploring new recruitment/onboarding strategies (e.g., buddy system)

Review and update Topics for Future Reporting SIG Meetings 

  • A test Action Item (Ingolf)