2022-12-15 Reporting SIG Meeting notes

Arthur Aguilera

University of Colorado, Boulder

Eric Luhrs

Lehigh University

Sharon Beltaine

Cornell University

Linda Miller

Cornell University

Erin Block

University of Colorado, Boulder

Nassib Nassar

Index Data

Nancy Bolduc

Cornell University

Elena O'Malley

Emerson

Shannon Burke

Texas A&M

Tod Olson

University of Chicago

Suzette Caneda

Stanford University

Jean Pajerek

Cornell University

Lloyd Chittenden

Marmot

Michael Patrick

The University of Alabama

Tim Dannay

Mount Holyoke College

Eric Pennington

Texas A&M

Axel Doerrer

University Mainz

Scott Perry

University of Chicago

Shelley Doljack

Stanford University

Natalya Pikulik

Cornell University

Stefan Dombek

Leipzig University

Vandana Shah

Cornell University

Jennifer Eustis

U. Massachusetts Amherst / Five College

Linnea Shieh

Stanford University

Lynne Fors

Wellesley College

Bob Scheier

Holy Cross

Lisa Furubotten

Texas A&M

Clare Spitzer

Stanford University

Alissa Hafele

Stanford University

Amelia Sutton

U. Massachusetts

Kara Hart

Wellesley College

Simona Tabacaru

Texas A&M

Jamie Jesanis

Wellesley College

Huey-Ning Tan

Stanford University

Jeanette Kalchik

Stanford University

Vitus Tang

Stanford University

Kevin Kishimoto

Stanford University

Irina Trapido

Stanford University

Ingolf Kuss

hbz

Catherine Tuohy

Emmanuel College

Alexander Lao

Stanford University

Kevin Walker

The University of Alabama

Joanne Leary

Cornell University

Angela Zoss

Duke University

Eliana Lima

Fenway Library Organization

Attendance & Notes

Angela

Attendance & Notes

• Today's attendance-taker: Linda (or substitute)

• Today's note-takers:  Team Leads for project updates

Announcements /
Reminders

Angela

Schedule for December

• cancelling December 21 development team meeting and December 26 SIG meeting

One convener is great, but two is better!

• Happy to announce that Sharon Beltaine will begin serving as co-convener (again!) in January. Thanks so much, Sharon!

Any new members?

• Welcome/introductions

• Would anyone like a buddy? Like to be a buddy?

How to find our latest recordings

• Recording details available on our Reporting SIG Meetings and Notes page and Reporting SIG Subteam Meetings page

(Always) Recruiting New Query Developers

• The Reporting SIG is always on the look-out for new query developers. Please let us know if you are interested in doing query development or if there are others at your institution who might be a good fit.

Update on documentation of derived tables

Stefan, Axel

• Axel and Stefan have met to talk about automating documentation of derived tables

• Stefan created a script to extract comments and tables from the derived table scripts

• Would like to automate diagrams for these tables, but it could be tricky. Have to parse the SQL to extract the tables and the joining fields.

• In Monday SIG, we decided diagrams would really be helpful if possible

• Discussion

  • today in RM/ERM, there were some questions about how we designed the derived tables, why are some separate

  • maybe this documentation will help us think more systematically about the derived tables across all areas

  • diagrams are hard to automate, but if it was something we had to do manually, that would be pretty tricky

Review of In-Progress Projects

• Build a directory of extracted and derived tables

  • (see previous discussion)

• Build Training Program for FOLIO Data Model

  • RA/UM: willing to try to build out some more diagrams for how Inventory tables connect, especially Instance/Holdings/Items (have actually already covered materials hierarchy and locations). Didn't really start this work yet, though.

  • Question: where will this live?

    • Reporting SIG wiki

    • FOLIO tips & tricks - probably good to at least link to our work from here

    • docs.folio.org

    • CrossApp group?

    • GitHub

• Investigate LDP Hosting Support

• Port LDP derived tables to Metadb

• Recruit from new institutions not represented

  • Included this request in our recent update to PC, and MM Convener actually made an announcement for us. Expect to do a series of visits to SIG in the new year to try to make a stronger case.

  • Also did get some confirmation that reaching out to Community Council would be a good idea; they are talking about FOLIO-wide onboarding, and we might be able to suggest that reporting be added as a good place to have a representative

Updates and Query Demonstrations from Various Reporting Related Groups and Efforts

Community & Coordination, Reporting Subgroup Leads

Project updates

Reporting development is using small subgroups to address priorities and complete work on report queries.  Each week, these groups will share reports/queries with the Reporting SIG.  Reporting development team leads are encouraged to enter a summary of their work group activities below.

RA/UM Working Group

• Meetings have become more of a lab session, working through specific problems

• Contact Angela if you would like to join these meetings; second Tuesdays at 1pm Eastern

• Context

  • Meeting notes: https://docs.google.com/document/d/1UnzG64tl917LOH2FtWhCEPlSOsnJWKL-0eu88Ouo1DU/edit

  • Current status of RA/UM issues: https://github.com/folio-org/folio-analytics/issues?q=is%3Aissue+is%3Aopen+RA%2FUM

  • Status of LDP → Metadb querying porting: https://docs.google.com/spreadsheets/d/1efLc6QMJyGuXyQM26pZbti-B6wFhJuZt7s7FJbhdXYo/edit#gid=0

• Recording of porting a table from LDP to Metadb: https://drive.google.com/drive/u/0/folders/1FrhrIM8-MeF0BPbI3kuD5ofoAqwGu1P1

MM Working Group

• Meetings are 1st Tuesday of the month, 12-1pm ET via zoom using the usual FOLIO password. Our lab sessions are open to everyone. Please bring your questions, examples, and comments about reporting and metadata. MM notes

• We will be fixing the LDP derived table instance_ext.sql which has a duplicate field (record_source and instance_source). The column record_source will be removed and comments will be added to the derived table.

• If you are able to test ldpmarc 1.6 new release, please do so.

ERM Working Group

• FOLIO Data Model Training (in progress)

• ERM Query Development Status

• Open Access

  • Report requests are being collected in the OA SIG Link (in progress)

  • The test environment from the SIG Reporting will get the app OA (in progress)

  • Test data will imported

  • Recent FOLIO Forum on Open Access App

• Meetings are bi-weekly on tuesdays 11am ET alternating with RM Working Group

  • Next meeting will be at 17th, Jan

  • Contact @Stefan Dombek if you would like to get a calendar invitation

RM Working Group

• group is working with ERM group on RM-ERM FOLIO Data Model Training 

• group worked on RM-related derived tables for Metadb and adding comments to columns for folio-analytics Nolana 1.5 release

• group will start working on RM-related derived tables for Orchid 1.6 release 

  • see RM Derived Tables to Convert from LDP to Metadb

• for latest query development updates, see RM Prototype and Query Development Status

• Meetings are biweekly on Tuesdays 11am-noon ET; contact @Sharon Markus if you would like a calendar invitation

• Meeting Recordings are here: FOLIO ERM-RM Report Dev Teams

Reporting SIG Documentation Subgroup

• Morning Glory documentation is live on https://docs.folio.org/docs/

• Nolana documentation is in review

• Orchid documentation will be in progress soon, and plans are underway to include beta-level documentation for Metadb

• Additional Context

  • The Reporting SIG has representation on the Documentation Working Group, which is building end-user documentation for https://docs.folio.org/docs/ (mostly linking to existing documentation over on GitHub)

External Statistics Working Group

• no updates currently

• new organizational/tracking scheme for JIRA, with pointers to queries in folio-analytics repository

• New organizational structure for External Statistics reports

  • external statistics reports (e.g., ACRL) typically require running queries from different functional reporting areas

  • these reports will be captured in JIRA under one UXPROD-XXXX report cluster issue, then the descriptions will point to each of the queries required to run them on the folio-analytics repository

  • institutions will need to rank each of these 8 new UXPROD-XXXX report cluster issues

  • each reporting development team will take responsibility for the queries in their area for the external statistics clusters

Product Council

• (December 8 meeting) PC shared a report of the SIG check-ins

• would love for PC to check-in yearly

For all recent work on FOLIO Reporting SQL development:

• https://github.com/folio-org/folio-analytics/commits/main

Bound-withs

• MM meeting from Dec 15 included a demo from Charlotte, looked really cool

Topics for Future Meetings

All

• How to deal with External Stats reports?

  • maybe subteam leads check in about that

  • probably wait until after Metadb conversion is more complete

• Exploring new recruitment/onboarding strategies (e.g., buddy system)

• More work on asynchronous collaboration, how to engage in discussions and question answering more broadly

  • consider connecting discuss.folio.org with a Slack channel, to make sure any forum topics get highlighted on Slack as well?

• Open question: should we update FOLIO LDP1-based Reporting First Implementers Grid

• Cleaning up our wiki pages

• Monday, Dec 12: update on documentation of derived tables

• January: tips on querying date fields from Sharon and Joanne

• talk about preparation for Metadb: do institutions have questions? is there anything we should be working on to try to support them?

Review and update Topics for Future Reporting SIG Meetings