2022-06-02 Reporting SIG Meeting notes









Arthur AguileraUniversity of Colorado, BoulderxLinda MillerCornell University
xSharon BeltaineCornell University
Nassib NassarIndex Data

Erin BlockUniversity of Colorado, BoulderxElena O'MalleyEmerson

Nancy BolducCornell University
Tod OlsonUniversity of Chicago
xSuzette CanedaStanford UniversityxJean PajerekCornell University

Lloyd ChittendenMarmot
Michael PatrickThe University of Alabama

Axel DoerrerUniversity Mainz
Eric PenningtonTexas A&M

Shelley DoljackStanford UniversityxScott PerryUniversity of Chicago

Stefan DombekLeipzig UniversityxNatalya PikulikCornell University

Jennifer EustisU. Massachusetts Amherst / Five College
Vandana ShahCornell University

Alissa HafeleStanford UniversityxLinnea ShiehStanford University
xJeanette KalchikStanford University
Clare SpitzerStanford University
xKevin KishimotoStanford University
Amelia SuttonU. Massachusetts

Ingolf Kusshbz
Simona TabacaruTexas A&M

Joanne LearyCornell University
Huey-Ning Tan

Stanford University

xEliana LimaFenway Library Organization
Vitus TangStanford University
xAlexander LaoStanford UniversityxIrina TrapidoStanford University

Eric LuhrsLehigh University
Kevin WalkerThe University of Alabama

xAngela ZossDuke University

Discussion Items




Attendance & NotesAngela

Attendance & Notes

  • Today's attendance-taker: ?
  • Today's note-takers:  Team Leads for project updates

Announcements /


WOLFCon 2022

  • Aug 31-Sep 2; Hamburg, Germany
  • Registration Open
    • Early registration for in person is a $50 savings, extended to June 3 
    • Virtual attendance: $15, already open
  • Current planned sessions
  • Three reporting-related sessions planned:
    • SIG organized: Reporting Lessons from Implementers (Panel, Standard (50 min), Hall (up to 250 people), Live presentation)
    • LDP/Nassib organized:
      • Introduction to LDP
      • Library Data Platform: Analytics in the Library

How to find our latest recordings

  • Separate folders right now: January/February recordings, recordings from March and onward
  • New problem with recordings: now need a password. Use the same password from our Zoom meetings. If you enter the password and it just takes you to the password screen again, refresh the browser page and the recording should show up.

(Always) Recruiting New Query Developers

  • The Reporting SIG is always on the look-out for new query developers. Please let us know if you are interested in doing query development or if there are others at your institution who might be a good fit.

PO Line data updates in Kiwi

Should we fix the derived tables to accommodate new data? If so, RM can take that on. It is impacting reports at Cornell.

  • With Kiwi, there is a direct link between PO line and holdings; before had to infer from item
  • updated how a field is used, but old data in that field
  • creating a new PO, you can create the link at that time
  • but old stuff is not linked
  • updating inventory location doesn't update PO location
  • right now is getting either location ID or holding ID, not both, so can lose the link to holding
  • if order is created but you don't check holding, it doesn't create instance and holding, just creates location
  • already have a link between item to POL, and instance to POL, but holding is new; and there is a way to manually create the holdings link
  • maybe use CASE WHEN or COALESCE to have the query work for multiple cases; if location is empty, use holding to find the holding location
  • why not start with holding location? are you trying to find where it actually is, or where it was supposed to be when you ordered it?
  • derived table right now just get location ID, ignoring holding id; can use holding id instead if there is no location
    • po_instance
    • po_pieces
    • po_lines locations
    • po_lines eresources
  • also a data cleanup issue, try to get the links created for past records
  • maybe be careful about coalesce, might be safer to keep locations separate, at least in the derived tables
  • maybe there is script that could help - maybe write a query to pull together the data for PO Lines that don't have a linkage to holding; for those that have item ID, look for the holdings ID connected to that item; PO Line could have instance link, so could do it from that direction and just look for ones that have only one holding; if there are more than one holdings, would have to be manual
  • maybe we need a slack channel - stuff we're doing to the code/apps that might impact you
  • when we talk about changes to existing tables, maybe be careful about redefining existing fields, make announcements that tables are changing, etc.
Reporting Vision and Strategy work
Reporting SIG Roadmap
Keeping track of new query requests: JIRA vs. GitHub
  • What do we do with the old requests?
    • REP requests vs. UXPROD clusters
    • How much do we do with report queries vs. derived tables?
    • How much are we willing to maintain?
  • Where should new report requests go?
  • Talked about it in Wednesday development meeting
    • Jen suggested discussion board feature on GitHub for new report requirements, then have the team discuss who should do the development work, then create an issue based on that discussion
    • talked about staying in JIRA, but there are different points of view on that
    • if we move away from JIRA, we lose the connection to the broader FOLIO community, who are more familiar with it; GitHub may be a bit technical for some of the broader community
Updates and Query Demonstrations from Various Reporting Related Groups and EffortsCommunity & Coordination, Reporting Subgroup Leads

Project updates

Reporting development is using small subgroups to address priorities and complete work on report queries.  Each week, these groups will share reports/queries with the Reporting SIG.  Reporting development team leads are encouraged to enter a summary of their work group activities below.

RA/UM Working Group

MM Working Group

  • Meetings are 1st Tuesday of the month, 12-1pm ET via zoom using the usual FOLIO password. Our lab sessions are open to everyone. Please bring your questions, examples, and comments about reporting and metadata.
  • We are still looking for help with derived tables and reviewers. See our sign up sheet.
  • Our next session will go through the review process and one more example of porting over a derived table from LDP to metaDB.

ERM Working Group

  • ERM Prototype and Query Development Status
    • Currently working on bugfixes that came with the kiwi release (data schema changes)
    • In consultation with the RM wg, comments on the columns of the derived tables are created for derived tables from folio-analytics release 1.5
  • Content topics for the future
    • Open Access: There will be a module on OA, the content of which overlaps with the Agreements (erm) app.
    • eUsage: As soon as reporting is possible, possibilities are sought to develop derived tables / report queries for this.
  • Meetings are bi-weekly on tuesdays 11am ET alternating with RM Working Group

RM Working Group

Reporting SIG Documentation Subgroup

  • Honeysuckle documentation is live on https://docs.folio.org/docs/
  • Iris documentation is in progress, due December 15
  • Additional Context
    • The Reporting SIG has representation on the Documentation Working Group, which is building end-user documentation for https://docs.folio.org/docs/ (mostly linking to existing documentation over on GitHub)

External Statistics Working Group

  • no updates currently
  • new organizational/tracking scheme for JIRA, with pointers to queries in folio-analytics repository
  • New organizational structure for External Statistics reports
    • external statistics reports (e.g., ACRL) typically require running queries from different functional reporting areas
    • these reports will be captured in JIRA under one UXPROD-XXXX report cluster issue, then the descriptions will point to each of the queries required to run them on the folio-analytics repository
    • institutions will need to rank each of these 8 new UXPROD-XXXX report cluster issues
    • each reporting development team will take responsibility for the queries in their area for the external statistics clusters

Product Council

For all recent work on FOLIO Reporting SQL development:

Topics for Future MeetingsAll
  • Reviewing a pull request 
  • How to deal with External Stats reports?
    • maybe subteam leads check in about that
    • probably wait until after Metadb conversion is more complete
  • hosting experiences from implementers
  • Reporting Vision and Strategy work
    • How to do report project tracking moving forward
  • End of June: how is the alternating week schedule working for SIG meetings?

Review and update Topics for Future Reporting SIG Meetings 

  • A test Action Item (Ingolf)