2021-09-13 Reporting SIG Meeting notes









xSharon BeltaineCornell UniversityxElena O'MalleyEmerson

Nancy BolducCornell University
Tod OlsonUniversity of Chicago
xLloyd ChittendenMarmotxJean PajerekCornell University
xAxel DoerrerUniversity MainzxMichael PatrickThe University of Alabama
xShelley DoljackStanford UniversityxEric PenningtonTexas A&M


Stefan DombekLeipzig UniversityxScott PerryUniversity of Chicago
xJennifer EustisU. Massachusetts Amherst / Five CollegexNatalya PikulikCornell University
xIngolf Kusshbz
Vandana ShahCornell University

Kim LaineCornell UniversityxAmelia SuttonU. Massachusetts

Jesse LambertsonUniversity of ChicagoxSimona TabacaruTexas A&M
xEliana LimaFenway Library OrganizationxKevin WalkerThe University of Alabama
xLinda MillerCornell UniversityxAngela ZossDuke University
xNassib NassarIndex Data

Discussion Items




Attendance & NotesAngela

Attendance & Notes

  • Today's attendance-taker: Linda Miller
  • Today's note-takers:  Team Leads for project updates

Announcements /


Recruiting New Query Developers

  • The Reporting SIG is always on the look-out for new query developers. Please let us know if you are interested in doing query development or if there are others at your institution who might be a good fit.

Cluster Ranking

New Report Clusters are added on a regular basis, so it is important to make sure your institution is reviewing these clusters and ranking them to establish report development priorities. If you rank reports for your institution, please follow the instructions below. If someone else ranks, please pass this information along to that person so your institution's vote can be included.

  • Action =>> Please review Reporting SIG-All Report Clusters (57 issues) in JIRA and RANK each report cluster for your institution (R1-R5)
  • For reporting, institutions only need to rank the UXPROD Report Cluster JIRA issues. All reporting requirements, which are captured in REP-XXX issues, roll up to the UXPROD Report Clusters. Report clusters cover one or more report (REP-XXX issue) requirements.

Update LDP implementers grid

New FOLIO Query Writing Knowledge Base

  • See "Writing Queries" section in the sidebar of the wiki
Community ownershipNassib
  • over the last few months, SIG activities (especially query development) have slowed a bit
  • how do we think about sustaining the work and our community moving forward
  • query writing is very time consuming
  • making LDP generalizable to other projects has opened up our community for support
  • LDP is being taken on as a project by OLF - yay!
  • for query development, hard to incentivize contributions
  • right now relying just on volunteers who have stepped up
  • worth asking the question of what we should work on? derived tables that enable ad hoc queries? or should we continue to work on JIRA issues (or both)?
  • how can we commit to doing this work without any funding for query development?
  • could Reporting SIG members volunteer for specific roles?
    • outreach to other groups in FOLIO
    • training
    • others?
  • this work benefits everyone who may rely on open source reporting for FOLIO
  • members can also remind institutions that reporting work is not funded and could really benefit from institutional support
  • by moving under OLF, can bring library leaders together to brainstorm on opportunities and potentially find funding
  • would still like to stay in touch with LDP users and developers, so maybe there's some kind of membership model
  • major kudos to early Reporting SIG members to get us where we are, which is a fantastic tool
  • Goose and the Golden Egg: we have to make sure we're taking good care of the goose, don't kill the goose
  • hard because we don't have a requirement to participate, everyone is volunteering
  • how to incentivize people to join in reporting activities?
  • both scoping our work and encouraging new engagement
  • anyone willing to share what is keeping them from getting more involved in activities?
  • to summarize for new people, note that we have several subteams for query development: RA/UM, RM, ERM, MM
    • MM is having "lab sessions" to offer help; most MM members are not 100% reporting specialists, they just do reports to assist in doing their jobs
      • 1st Tuesday of month, 12-1pm Eastern (see Zoom link below)
    • RM meets Thursdays at 9am Eastern
  • should we try to use a lab model across all functional areas? not make as many distinctions between users and developers of queries?
  • what is the long-term goal of the reporting group? is it to have a sustainable group of developers who can help on particular reports? generate SQL queries? or is it more, are we hoping for some kind of (visualization tool?)
  • have to balance what would be useful for the community and what people can justify spending their time on?
  • how helpful are the derived tables? the report queries? should we continue doing both? focus on one?
  • also, what kind of support do we provide for the queries we do write?
  • do we need a training process? how do we implement that?
  • just focusing on queries isn't really going to be fancy, like a visualization tool
  • some people just want a very simple way to run queries and get results; one solution for that is to continue building out the LDP app (maybe have the app pull queries from GitHub and take parameters from the user to generate the report)
  • one incentive might be that if institutions need a report, they can participate in a subteam and influence the priorities
  • maybe we should have more specific deadlines for query repository releases
  • maybe closer ties to the other SIGs will unearth some more people who could participate in reporting
  • do we need a subgroup/steering committee - a "harder questions" committee
  • training, counteracting misinformation (outreach)
Interest in a demo of querying non-Latin symbols?all
  • Offer from Stefan:
    • In our library we will later import titles in non-Latin script symbols. I tried out the querying with these symbols. SQL allows you to querying directly non-Latin symbols. I have now gone one step further. I converted the stored strings to datatype bytea and then I have read, searched and manipulated these content. After that I converted it back to UTF-8 (varchar) again. Incidentally, DBeaver has a HEX viewer. I am happy to do a small demo within the group at some point if there is interest.
    • will schedule for end of a meeting in the near future
Interest in project to build dynamic parameter listsall
  • at Cornell, people are using the queries, and they have to adjust the filters at the top of the query
  • to do that, you have to know the different values that are valid for those filters
  • there are two kinds of options: hard coded (written into the FOLIO software) and soft coded (configured by institutions)
  • for soft coded options, they may change more frequently, would like to be able to create dynamic lists of all valid options (e.g., all valid locations)
  • at 5C, grab lists of locations from API (those are in settings, maybe under tenant); sometimes there are just in tables (statistical codes are in their own tables); order and agreements statuses are hard coded in mods on github
  • maybe share lists with the subteams and ask for help 
Demonstration - connect to LDP via R and RStudioAngelapush to next week, pending other important topics
Updates and Query Demonstrations from Various Reporting Related Groups and EffortsCommunity & Coordination, Reporting Subgroup Leads

Project updates

Reporting development is using small subgroups to address priorities and complete work on report queries.  Each week, these groups will share reports/queries with the Reporting SIG.  Reporting development team leads are encouraged to enter a summary of their work group activities below.

RA/UM Working Group

MM Working Group

  • The zoom link for the meeting is https://umass-amherst.zoom.us/j/99254861410. Usual FOLIO password.
  • The group has a new schedule - once a month starting Sept. 7th, Tuesday, from 12-1pm ET. Our next meeting will be 10/5.
  • Ongoing, this group has decided to be a lab session or a virtual place to learn from each other, seek help, and report back on any issues with derived tables or reports for metadata.

ERM Working Group

  • Going to tweak existing ERM queries on resource type and read dates
  • New report requests:
    • list of agreements by cancellation dates
    • agreement fund distribution by years
    • price trending per agreement (or agreement line) over years
  • New app "Dashboard" might provide inApp reporting → group to discover more of its possibilities
  • ERM Prototype and Query Development Status

RM Working Group

  • reviewed fund_expenditures_summary; will consider turning this into fund_expenditure_summary_by_expense_class
  • created fund_expenditures_by_po_line
  • exploring ways to make it easier to find options for reporting parameters in your LDP (e.g., a list of locations, item statuses, etc.)
  • several queries for RM completed, but still need documentation, testing, and review
  • working on queries for the 1.2 FOLIO Analytics release
  • for latest updates, see RM Prototype and Query Development Status

Reporting SIG Documentation Subgroup

  • No changes
  • Additional Context

External Statistics Working Group

  • no updates currently
  • ACRL query set included in FOLIO-Analytics 1.1 release
  • new organizational/tracking scheme for JIRA, with pointers to queries in folio-analytics repository
  • New organizational structure for External Statistics reports
    • external statistics reports (e.g., ACRL) typically require running queries from different functional reporting areas
    • these reports will be captured in JIRA under one UXPROD-XXXX report cluster issue, then the descriptions will point to each of the queries required to run them on the folio-analytics repository
    • institutions will need to rank each of these 8 new UXPROD-XXXX report cluster issues
    • each reporting development team will take responsibility for the queries in their area for the external statistics clusters

Product Council

For all recent work on FOLIO Reporting SQL development:

Topics for Future MeetingsAll
  • Follow-up on MARC status, Quickmarc/Data Import conflicts
  • How to strengthen connections to SIGs and their developers to be kept in the loop about changes to the data model
  • Show and tell
    • how are institutions using the LDP
    • examples of using the local schema
    • Cornell's report ticketing system
    • Rollout plans from institutions
    • Ask someone on the sysadmin side to talk about LDP administration (Jason Root?)
  • Training topics
    • adding test data in FOLIO snapshot
    • How to do ad hoc querying with the derived tables
    • How to use the LDP app
    • using KNIME to build reports (LDP edition)
    • use of local schema for custom tables
    • more on MARC
    • using different applications (other than DBeaver)
    • Insomnia for API queries?
    • SQL
    • SQL/Notebooks in RStudio (Angela)
    • non-Roman script querying (Stefan)
  • Upcoming:

Review and update Topics for Future Reporting SIG Meetings 

  • A test Action Item (Ingolf)