/
2022-04-07 Reporting SIG Meeting notes

2022-04-07 Reporting SIG Meeting notes

Apr 07, 2022

Date

Apr 7, 2022

Attendees

Present?

Name

Organization

Present?

Name

Organization

Present?

Name

Organization

Present?

Name

Organization

x

Arthur Aguilera

University of Colorado, Boulder

x

Linda Miller

Cornell University

x

Sharon Beltaine

Cornell University

x

Nassib Nassar

Index Data

 

Erin Block

University of Colorado, Boulder

x

Elena O'Malley

Emerson

 

Nancy Bolduc

Cornell University

 

Tod Olson

University of Chicago

x

Suzette Caneda

Stanford University

x

Jean Pajerek

Cornell University

 

Lloyd Chittenden

Marmot

 

Michael Patrick

The University of Alabama

 

Axel Doerrer

University Mainz

x

Eric Pennington

Texas A&M

 

Shelley Doljack

Stanford University

x

Scott Perry

University of Chicago

 

Stefan Dombek

Leipzig University

x

Natalya Pikulik

Cornell University

 

Jennifer Eustis

U. Massachusetts Amherst / Five College

 

Vandana Shah

Cornell University

x

Alissa Hafele

Stanford University

x

Linnea Shieh

Stanford University

x

Jeanette Kalchik

Stanford University

x

Clare Spitzer

Stanford University

x

Kevin Kishimoto

Stanford University

 

Amelia Sutton

U. Massachusetts

 

Ingolf Kuss

hbz

x

Simona Tabacaru

Texas A&M

 

Kim Laine

Cornell University

 

Huey-Ning Tan

Stanford University

x

Joanne Leary

Cornell University

x

Vitus Tang

Stanford University

x

Eliana Lima

Fenway Library Organization

x

Irina Trapido

Stanford University

 

Eric Luhrs

Lehigh University

 

Kevin Walker

The University of Alabama

 

 

 

x

Angela Zoss

Duke University

Discussion Items

Item

Who

Notes

Item

Who

Notes

Attendance & Notes

Angela

Attendance & Notes

  • Today's attendance-taker: ?

  • Today's note-takers:  Team Leads for project updates

LDP query performance

Nassib

 Basic tips for faster queries in LDP databases

  • Indexes

  • Vacuum analyze

  • Temporary tables

  • Breaking apart large queries

  • Use NOT EXISTS instead of NOT IN

  • Simple column names

  • System configuration

Announcements /
Reminders

Angela

Trying to find our latest recordings

 

Recruiting New Query Developers

  • The Reporting SIG is always on the look-out for new query developers. Please let us know if you are interested in doing query development or if there are others at your institution who might be a good fit.

 

Open Discussion

 

  • Kiwi demo site has an LDP app; should we use that to learn about creating reports?

    • LDP app is easy to use and is intuitive to use, but it's simple and can only do a few things

    • LDP app draws on same information as the LDP, one table at a time

    • right now, if you don't use the LDP app, you have to connect to the database through another software application, and you probably need to write SQL to query the data

    • you probably don't want to have all of the library staff connecting to the database directly

    • the LDP app is a new app in Kiwi expands access to the LDP database to a larger group of people

    • it avoids the requirement of having to write SQL

    • this current version is pretty limited, but there are mock-ups to support table joins and really be a visual query builder

    • the next feature coming is to be able to run reporting queries created by FOLIO reporting community; the app will allow you to fill in parameters and just generally give more staff access to running reports

    • don't have dedicated staff for development; different developers are available at different times

    • also able to access derived tables, which is a custom table that can join multiple FOLIO tables, or a local table (something created by someone who has direct access to the LDP database)

    • you can thus program the LDP app by creating useful tables in the LDP database that the app can then see

    • you say it will be able to run queries from GitHub, can we point to our local query repository?

      • yes

    • can we edit SQL in FOLIO?

      • yes, but you probably want to be careful

      • if you have 100 people running custom SQL queries, if they don't have SQL training, there could be performance issues

      • for example, someone could do a CROSS JOIN without any filters; if those tables are large, the query could take days to run and fill up a terabyte of data

      • but in principle, having a text box where you could type SQL is not a problem, but might want to be careful about permissions

    • requests for upcoming meeting topics?

 

 

Updates and Query Demonstrations from Various Reporting Related Groups and Efforts

Community & Coordination, Reporting Subgroup Leads

Project updates

Reporting development is using small subgroups to address priorities and complete work on report queries.  Each week, these groups will share reports/queries with the Reporting SIG.  Reporting development team leads are encouraged to enter a summary of their work group activities below.

RA/UM Working Group

 

MM Working Group

  • Meetings are 1st Tuesday of the month, 12-1pm ET via zoom using the usual FOLIO password. Our lab sessions are open to everyone. Please bring your questions, examples, and comments about reporting and metadata.

  • We are still looking for help with derived tables and reviewers. See our sign up sheet.

  • In next our lab session, we will use the LDP instance identifiers and go through the process of porting this over to metadb, how to contribute a pull request with the changes and update the runlist.txt, and go through a review.

 

ERM Working Group

 

RM Working Group

 

Reporting SIG Documentation Subgroup

  • Honeysuckle documentation is live on https://docs.folio.org/docs/

  • Iris documentation is in progress, due December 15

  • Additional Context

    • The Reporting SIG has representation on the Documentation Working Group, which is building end-user documentation for https://docs.folio.org/docs/ (mostly linking to existing documentation over on GitHub)

 

External Statistics Working Group

  • no updates currently

  • new organizational/tracking scheme for JIRA, with pointers to queries in folio-analytics repository

  • New organizational structure for External Statistics reports

    • external statistics reports (e.g., ACRL) typically require running queries from different functional reporting areas

    • these reports will be captured in JIRA under one UXPROD-XXXX report cluster issue, then the descriptions will point to each of the queries required to run them on the folio-analytics repository

    • institutions will need to rank each of these 8 new UXPROD-XXXX report cluster issues

    • each reporting development team will take responsibility for the queries in their area for the external statistics clusters

 

Product Council

 

 

For all recent work on FOLIO Reporting SQL development:

 

Topics for Future Meetings

All

  • Next: Reviewing a pull request 

  • How to deal with External Stats reports?

    • maybe subteam leads check in about that

    • probably wait until after Metadb conversion is more complete

  • Definitely do GitHub Desktop demonstration at a SIG meeting in future, probably about 15 minutes

  • Converting queries to dashboards: Sharon and Vandana (April 25)

 

Review and update Topics for Future Reporting SIG Meetings 

 

 

 

 

A test Action Item (Ingolf)