2022-09-26 Reporting SIG Meeting notes









Arthur AguileraUniversity of Colorado, Boulder
Eric LuhrsLehigh University
xSharon BeltaineCornell UniversityxLinda MillerCornell University

Erin BlockUniversity of Colorado, Boulder
Nassib NassarIndex Data

Nancy BolducCornell UniversityxElena O'MalleyEmerson

Shannon BurkeTexas A&M
Tod OlsonUniversity of Chicago

Suzette CanedaStanford UniversityxJean PajerekCornell University

Lloyd ChittendenMarmotxMichael PatrickThe University of Alabama
xTim DannayMount Holyoke College
Eric PenningtonTexas A&M
xAxel DoerrerUniversity MainzxScott PerryUniversity of Chicago

Shelley DoljackStanford UniversityxNatalya PikulikCornell University
xStefan DombekLeipzig UniversityxVandana ShahCornell University
xJennifer EustisU. Massachusetts Amherst / Five College
Linnea ShiehStanford University
       Lisa FurubottenTexas A&M
Clare SpitzerStanford University

Alissa HafeleStanford University
Amelia SuttonU. Massachusetts

Jeanette KalchikStanford University
Simona TabacaruTexas A&M

Kevin KishimotoStanford University
Huey-Ning Tan

Stanford University

xIngolf Kusshbz
Vitus TangStanford University

Joanne LearyCornell University
Irina TrapidoStanford University
xEliana LimaFenway Library Organization
Kevin WalkerThe University of Alabama

Alexander LaoStanford UniversityxAngela ZossDuke University

Discussion Items




Attendance & NotesAngela

Attendance & Notes

  • Today's attendance-taker: Linda (or substitute)
  • Today's note-takers:  Team Leads for project updates

Announcements /


New Subgroup Opportunity: Exploring Data Integrity

  • Sharon has proposed an exploratory project to think about ways to review data integrity in LDP 1.x (and Metadb?)
  • Forming a subgroup would allow us to collect use cases and advocate for new queries or features (and resources to implement those features)
  • Anyone else interested?

How to find our latest recordings

(Always) Recruiting New Query Developers

  • The Reporting SIG is always on the look-out for new query developers. Please let us know if you are interested in doing query development or if there are others at your institution who might be a good fit.

LDP Hosting
  • Would like to be working with hosting providers to get a sense of who is willing to make a commitment to offering LDP hosting
  • Who should this go to? PC?
  • Is it possible to host LDP through a vendor different from your FOLIO vendor? (EBSCO for FOLIO, ID for LDP?)
  • EBSCO still saying LDP is not secure, would like to get some set answers on that
  • Also more transparency on cost
  • Options:
    • survey institutions again about hosting provider and use of reporting solutions (add hosting to current page)
    • talk with the vendors directly to ask for some kind of commitment or policy or pricing model
    • could give a technical presentation - how LDP is hosted by several institutions, how to avoid pitfalls around security
    • have vendor presentations on their hosting services (EBSCO, Index Data)
  • reporting seems to be secondary to FOLIO, not considered a core service (only a few in-app reports in FOLIO that are not consistent), which means vendors offer their own solutions (Panorama for EBSCO, LDP for ID) and can choose not to support others
  • only interact with hosts when there is a problem/upgrade, curious what communication and testing are like for others about upgrades; could be another topic for the survey or a presentation
    • talk to Cornell!
    • having this kind of second system for reporting is common for IT more broadly, but seems like our hosting providers are still struggling with this workflow for getting LDP updates that doesn't impact users; testing is important, especially as LDP software becomes more complex
    • At Duke, communication around upgrades isn't perfect either
    • Would like vendors to have a set of answers about how they host something like LDP - how do you handle updates, etc.
  • As a SIG, how could we incentivize hosting providers to either skill up for hosting LDP or make it easier for institutions to connect their FOLIO to a separately hosted LDP; ask for their concerns in either of these scenarios and establish what the requirements are for offering that service
  • note: Reporting SIG came up in the PC, Angela wrote up a big reply
  • if vendors really want time in a SIG meeting, all they have to do is ask for time; they have been told this repeatedly
  • Good to involve the various councils, OLF, etc. in these questions
  • Ingolf, what do you think about TC, Systems SIG weighing in on security concerns? Should we partner with them to come up with guidance?
  • ask EBSCO why LDP is not secure but FOLIO is?
  • Next steps
    • survey institutions to get a complete list
    • talking to vendors: have a solution you're looking for, talk to vendors about what they have. come up with questions about the services provided, the level of expertise, long-term commitment, any barriers/requirements (but cost is going to be tricky)
    • talk to technical/security experts within the FOLIO community about whether there are security concerns
    • should we do more outreach to SIGs - what do you need from reporting, be more visible; listening session about in-app reporting
  • Reporting SIG should/could be leading discussions about the dashboard
  • put on journalist hat - rather than saying, "why is LDP not secure,?" ask "how do you as a hosting provider ensure data security?"
Training topic; Using FOLIO APIs (Postman, LDLite)Angela

Quick demonstration of using Postman and LDLite to pull data from FOLIO via APIs


(did not get to this; will do on Thursday)

Updates and Query Demonstrations from Various Reporting Related Groups and EffortsCommunity & Coordination, Reporting Subgroup Leads

Project updates

Reporting development is using small subgroups to address priorities and complete work on report queries.  Each week, these groups will share reports/queries with the Reporting SIG.  Reporting development team leads are encouraged to enter a summary of their work group activities below.

RA/UM Working Group

MM Working Group

  • Meetings are 1st Tuesday of the month, 12-1pm ET via zoom using the usual FOLIO password. Our lab sessions are open to everyone. Please bring your questions, examples, and comments about reporting and metadata.
  • We are still looking for reviewers and testers for ldpmarc. Please contact MM reports working group.
  • ldpmarc: Cornell noticed an issue with ldpmarc. Nassib is aware of this and coming up with a fix which will be tested.

ERM Working Group

RM Working Group

Reporting SIG Documentation Subgroup

  • Lotus documentation is live on https://docs.folio.org/docs/
  • Morning Glory documentation is complete and submitted
  • Nolana documentation will be in progress soon
  • Additional Context
    • The Reporting SIG has representation on the Documentation Working Group, which is building end-user documentation for https://docs.folio.org/docs/ (mostly linking to existing documentation over on GitHub)

External Statistics Working Group

  • no updates currently
  • new organizational/tracking scheme for JIRA, with pointers to queries in folio-analytics repository
  • New organizational structure for External Statistics reports
    • external statistics reports (e.g., ACRL) typically require running queries from different functional reporting areas
    • these reports will be captured in JIRA under one UXPROD-XXXX report cluster issue, then the descriptions will point to each of the queries required to run them on the folio-analytics repository
    • institutions will need to rank each of these 8 new UXPROD-XXXX report cluster issues
    • each reporting development team will take responsibility for the queries in their area for the external statistics clusters

Product Council

For all recent work on FOLIO Reporting SQL development:

Topics for Future MeetingsAll
  • How to deal with External Stats reports?
    • maybe subteam leads check in about that
    • probably wait until after Metadb conversion is more complete
  • ask for presenters: hosting experiences from implementers
  • Annual Reporting Goals
    • (in progress) Support the transition from LDP to Metadb (e.g., update derived table and report queries, update documentation, outreach, new training)
    • (ready to start) Developing training/onboarding for new SIG members/report users (esp. FOLIO-specific data model and transformation stuff)
    • Improve communication between SIG and developers of apps so we hear about data model changes in advance
    • continued advocacy on part of the SIG to governance groups
    • (ready to start) Review JIRA issues, clean up, revisit strategy for JIRA
  • Regular review of Milestones
  • Exploring new recruitment/onboarding strategies (e.g., buddy system)
  • Guest presentations: Foliage (Mike Hucka and Laurel Narizny from CalTech)
  • Demo latest version of LDP app, any new features?
  • Training: Using APIs
  • More work on asynchronous collaboration, how to engage in discussions and question answering more broadly
    • consider connecting discuss.folio.org with a Slack channel, to make sure any forum topics get highlighted on Slack as well?

Review and update Topics for Future Reporting SIG Meetings 

  • A test Action Item (Ingolf)