2022-04-25 Reporting SIG Meeting notes









Arthur AguileraUniversity of Colorado, BoulderxLinda MillerCornell University
xSharon BeltaineCornell University
Nassib NassarIndex Data

Erin BlockUniversity of Colorado, BoulderxElena O'MalleyEmerson

Nancy BolducCornell UniversityxTod OlsonUniversity of Chicago

Suzette CanedaStanford UniversityxJean PajerekCornell University

Lloyd ChittendenMarmotxMichael PatrickThe University of Alabama
xAxel DoerrerUniversity MainzxEric PenningtonTexas A&M

Shelley DoljackStanford UniversityxScott PerryUniversity of Chicago


Stefan DombekLeipzig UniversityxNatalya PikulikCornell University
xJennifer EustisU. Massachusetts Amherst / Five CollegexVandana ShahCornell University

Alissa HafeleStanford University
Linnea ShiehStanford University

Jeanette KalchikStanford University
Clare SpitzerStanford University

Kevin KishimotoStanford UniversityxAmelia SuttonU. Massachusetts
xIngolf Kusshbz
Simona TabacaruTexas A&M

Joanne LearyCornell University
Huey-Ning Tan

Stanford University

Eliana LimaFenway Library Organization
Vitus TangStanford University

Alexander LaoStanford University
Irina TrapidoStanford University

Eric LuhrsLehigh UniversityxKevin WalkerThe University of Alabama

xAngela ZossDuke University

Discussion Items




Attendance & NotesAngela

Attendance & Notes

  • Today's attendance-taker: ?
  • Today's note-takers:  Team Leads for project updates

Announcements /


New meeting: LDP Users Group

  • May 12 (11:00 a.m. Eastern) 
  • informal: share news and recent work
  • Both LDP 1.x and Metadb users, but introducing more Metadb content, more cross-FOLIO-ReShare content

How to find our latest recordings

Time for a Training Working Group?

  • One of our Goals for 2022 is to develop training and onboarding resources specific to reporting
  • Is it time to form a small team?
  • Anyone interested?

(Always) Recruiting New Query Developers

  • The Reporting SIG is always on the look-out for new query developers. Please let us know if you are interested in doing query development or if there are others at your institution who might be a good fit.

Data model changes in KiwiStefan

There are a few small deviations in the Kiwi data schema that affect our derived tables and report queries in ERM.
Question for the group: How do we would like to offer different FOLIO releases in the repository in the future?

  • changes were announced by Owen, so not a complete surprise, but it means we have to change some reports and derived tables
  • how would we like to offer different FOLIO release versions of reports in our repository
  • documentation is tied to FOLIO releases, and we're trying to tie the documentation to specific versions of FOLIO Analytics
  • if there's something that comes to our attention that needs to be changed, we should backport to the previous versions of the document
  • two proposals:
    • we have a specific release for each FOLIO release (Juniper, Kiwi, Lotus, etc.); each new query added to, e.g., Lotus would also need to get added to previous versions
    • add subdirectories: sql/juniper; sql/kiwi; sql/lotus
  • probably need to talk to Nassib
  • the issue is not just maintaining FOLIO Analytics; it's also making sure that people who have built on top of our existing derived table can continue to run their queries
  • for people testing against sandbox, we only have Juniper right now; people without their own LDP can't test against Kiwi right now (much less Lotus)
  • in documentation, only going back to one previous version, so maybe we could draw that boundary
  • if the derived tables will be different for different releases, this will impact our queries based on those derived tables
  • more a question to Nassib: is it possible to create branches in repository for different releases; then we can have copies of derived tables and could change them differently in different branches; right now we have branches for old releases and current release, so we should ask Nassib
  • how many releases should we support?
  • what changed in ERM? Stefan can show next time
  • note: the pressure to stay current increases after implementation
2022 Goal: Develop a FOLIO Reporting Vision and StrategyAll

Following up on statement of scope brainstorm (see April 21 and April 11 notes)

Summary of previous discussions:

Communication/Training: [maybe split communication and training into two separate sections]

  • training and onboarding, marketing and outreach
  • helping people build SQL skills
  • helping people understand FOLIO data structures
    • helping SIG members understand what's under the hood of FOLIO
    • helpful in understanding the structure of FOLIO.  It's come in handy in debugging/identifying issues after going live.
  • been a good place for people to come as a centralized point of information; with the subgroups, people get split up, but the SIG meeting brings everyone together to get acquainted with things
  • Identifying reporting-specific needs as it relates to the Folio community - like need for good communication in development; influencing and advocating for FOLIO governance practices that meet the needs of the reporting community
  • Helping other SIGs - outreach to other SIGs, maintaining strong ties for information flow/support of each other's work
    • making sure we're not duplicating effort, making sure we're not working on something that is already available in-app
    • giving feedback to FOLIO app developers on what is needed to support reporting
  • bridging the knowledge gap between the technical and non-technical (functional?) people (or devs/SMEs)
  • advocate for resources for reporting
  • Advertise benefits of SIG membership/participation:
    • Benefits to community - other perspectives and workflows
    • Benefits to institutions - visibility, validation
    • Benefits to individuals getting involved - personal development and skills (e.g., SQL, GitHub, networking/connections)
  • documentation
  • help clarify the differences between different FOLIO reporting solutions (different software, different hosting options)

Supporting the LDP tools

  • the growing LDP ecosystem (LDP 1.x, Metadb, LDLite, LDP MARC, the LDP app, FOLIO Analytics) - committed to being active stakeholders in these systems
  • not writing the code for the LDP project software, but contributing substantial testing, QA, specifying new features needed, troubleshooting problems as they come up; making it useful; feedback on system design proposals; develop reporting tools that serve the needs of different kinds of users
  • aligning reporting software development with the FOLIO application software development
  • providing community support for using the LDP project software and queries;
    • but for scope, how long do we continue to answer questions about past versions of the software
    • maybe build an FAQ???
  • developing/testing/updating derived table and report queries tailored for LDP project systems (scope question: how regularly/for how long)
    • open the repository for non-SIG members and review their queries; may need to discuss, does this make sense, since it adds work?
    • influencing the shape of the product by building derived table queries - responding directly to user needs by designing helpful tables on top of the basic structure
  • maintaining documentation
  • gather requirements from the broader FOLIO community related to reporting needs, document them, advocate for them
    • providing information about what our users need, through needs assessment
    • create/maintain connections to functional area SIGs
    • be responsive to new apps and functional areas
  • connecting non-FOLIO datasets to FOLIO data
    • especially ReShare, because the reporting platform is part of the LDP ecosystem
    • even for non-ReShare data, this work is well supported by the LDP ecosystem
    • legacy system data
    • being able to help people work on non-FOLIO data ingest and reporting (example: ARES data into LDP) - maybe out of scope, but where does the line go?
    • importing data into the LDP on the fly - little tables, etc.
      • bringing an Excel file into the LDP in local schema to work with it, figuring out how to change data attributes to make it work
      • getting a full, large dataset into the LDP as a different schema

Supporting other types of FOLIO reporting

  • additional reporting solutions (e.g., direct reporting against FOLIO APIs, in-app FOLIO reports, additional third-party tools) - preference for open over closed tools; serve as a resource to other reporting solutions looking for feedback or partnership
    • in order to have that, there has to be engagement from someone who wants support for that tool; SIG is not responsible for being proactive about supporting new tools
    • as a SIG we can offer advice on the things that we have knowledge about within the SIG; if a third-party tool has no connection to the SIG, we won't have enough knowledge about that tool to directly offer support
    • would be nice to frame the scope in the positive: we have expertise in certain tools and techniques, and we're happy to share/support that
  • in past, have been asked to comment on/help design features for in-app reporting (still in scope?)
  • for tools we don't have direct access to, can we support users of those tools? help with SQL, understanding FOLIO data might be topics that cut across tools; should part of our vision be to welcome non-LDP users into the group more?
Updates and Query Demonstrations from Various Reporting Related Groups and EffortsCommunity & Coordination, Reporting Subgroup Leads

Project updates

Reporting development is using small subgroups to address priorities and complete work on report queries.  Each week, these groups will share reports/queries with the Reporting SIG.  Reporting development team leads are encouraged to enter a summary of their work group activities below.

RA/UM Working Group

MM Working Group

  • Meetings are 1st Tuesday of the month, 12-1pm ET via zoom using the usual FOLIO password. Our lab sessions are open to everyone. Please bring your questions, examples, and comments about reporting and metadata.
  • We are still looking for help with derived tables and reviewers. See our sign up sheet.
  • Our next session will go through the review process and one more example of porting over a derived table from LDP to metaDB.

ERM Working Group

RM Working Group

Reporting SIG Documentation Subgroup

  • Honeysuckle documentation is live on https://docs.folio.org/docs/
  • Iris documentation is in progress, due December 15
  • Additional Context
    • The Reporting SIG has representation on the Documentation Working Group, which is building end-user documentation for https://docs.folio.org/docs/ (mostly linking to existing documentation over on GitHub)

External Statistics Working Group

  • no updates currently
  • new organizational/tracking scheme for JIRA, with pointers to queries in folio-analytics repository
  • New organizational structure for External Statistics reports
    • external statistics reports (e.g., ACRL) typically require running queries from different functional reporting areas
    • these reports will be captured in JIRA under one UXPROD-XXXX report cluster issue, then the descriptions will point to each of the queries required to run them on the folio-analytics repository
    • institutions will need to rank each of these 8 new UXPROD-XXXX report cluster issues
    • each reporting development team will take responsibility for the queries in their area for the external statistics clusters

Product Council

For all recent work on FOLIO Reporting SQL development:

Topics for Future MeetingsAll
  • Next: Reviewing a pull request 
  • How to deal with External Stats reports?
    • maybe subteam leads check in about that
    • probably wait until after Metadb conversion is more complete
  • Converting queries to dashboards: Sharon and Vandana (May 9, tentative)
  • hosting experiences from implementers

Review and update Topics for Future Reporting SIG Meetings 

  • A test Action Item (Ingolf)