2022-04-25 Reporting SIG Meeting notes

2022-04-25 Reporting SIG Meeting notes

Date

Apr 25, 2022

Attendees

Present?

Name

Organization

Present?

Name

Organization

Present?

Name

Organization

Present?

Name

Organization

 

Arthur Aguilera

University of Colorado, Boulder

x

Linda Miller

Cornell University

x

Sharon Beltaine

Cornell University

 

Nassib Nassar

Index Data

 

Erin Block

University of Colorado, Boulder

x

Elena O'Malley

Emerson

 

Nancy Bolduc

Cornell University

x

Tod Olson

University of Chicago

 

Suzette Caneda

Stanford University

x

Jean Pajerek

Cornell University

 

Lloyd Chittenden

Marmot

x

Michael Patrick

The University of Alabama

x

Axel Doerrer

University Mainz

x

Eric Pennington

Texas A&M

 

Shelley Doljack

Stanford University

x

Scott Perry

University of Chicago

x

Stefan Dombek

Leipzig University

x

Natalya Pikulik

Cornell University

x

Jennifer Eustis

U. Massachusetts Amherst / Five College

x

Vandana Shah

Cornell University

 

Alissa Hafele

Stanford University

 

Linnea Shieh

Stanford University

 

Jeanette Kalchik

Stanford University

 

Clare Spitzer

Stanford University

 

Kevin Kishimoto

Stanford University

x

Amelia Sutton

U. Massachusetts

x

Ingolf Kuss

hbz

 

Simona Tabacaru

Texas A&M

 

Joanne Leary

Cornell University

 

Huey-Ning Tan

Stanford University

 

Eliana Lima

Fenway Library Organization

 

Vitus Tang

Stanford University

 

Alexander Lao

Stanford University

 

Irina Trapido

Stanford University

 

Eric Luhrs

Lehigh University

x

Kevin Walker

The University of Alabama

 

 

 

x

Angela Zoss

Duke University

Discussion Items

Item

Who

Notes

Item

Who

Notes

Attendance & Notes

Angela

Attendance & Notes

  • Today's attendance-taker: ?

  • Today's note-takers:  Team Leads for project updates

Announcements /
Reminders

Angela

New meeting: LDP Users Group

  • May 12 (11:00 a.m. Eastern) 

  • informal: share news and recent work

  • Both LDP 1.x and Metadb users, but introducing more Metadb content, more cross-FOLIO-ReShare content

 

How to find our latest recordings

 

Time for a Training Working Group?

  • One of our Goals for 2022 is to develop training and onboarding resources specific to reporting

  • Is it time to form a small team?

  • Anyone interested?

 

(Always) Recruiting New Query Developers

  • The Reporting SIG is always on the look-out for new query developers. Please let us know if you are interested in doing query development or if there are others at your institution who might be a good fit.

 

Data model changes in Kiwi

Stefan

There are a few small deviations in the Kiwi data schema that affect our derived tables and report queries in ERM.
Question for the group: How do we would like to offer different FOLIO releases in the repository in the future?

  • changes were announced by Owen, so not a complete surprise, but it means we have to change some reports and derived tables

  • how would we like to offer different FOLIO release versions of reports in our repository

  • documentation is tied to FOLIO releases, and we're trying to tie the documentation to specific versions of FOLIO Analytics

  • if there's something that comes to our attention that needs to be changed, we should backport to the previous versions of the document

  • two proposals:

    • we have a specific release for each FOLIO release (Juniper, Kiwi, Lotus, etc.); each new query added to, e.g., Lotus would also need to get added to previous versions

    • add subdirectories: sql/juniper; sql/kiwi; sql/lotus

  • probably need to talk to Nassib

  • the issue is not just maintaining FOLIO Analytics; it's also making sure that people who have built on top of our existing derived table can continue to run their queries

  • for people testing against sandbox, we only have Juniper right now; people without their own LDP can't test against Kiwi right now (much less Lotus)

  • in documentation, only going back to one previous version, so maybe we could draw that boundary

  • if the derived tables will be different for different releases, this will impact our queries based on those derived tables

  • more a question to Nassib: is it possible to create branches in repository for different releases; then we can have copies of derived tables and could change them differently in different branches; right now we have branches for old releases and current release, so we should ask Nassib

  • how many releases should we support?

  • what changed in ERM? Stefan can show next time

  • note: the pressure to stay current increases after implementation

2022 Goal: Develop a FOLIO Reporting Vision and Strategy

All

Following up on statement of scope brainstorm (see April 21 and April 11 notes)

Summary of previous discussions:

Communication/Training: [maybe split communication and training into two separate sections]

  • training and onboarding, marketing and outreach

  • helping people build SQL skills

  • helping people understand FOLIO data structures

    • helping SIG members understand what's under the hood of FOLIO

    • helpful in understanding the structure of FOLIO.  It's come in handy in debugging/identifying issues after going live.

  • been a good place for people to come as a centralized point of information; with the subgroups, people get split up, but the SIG meeting brings everyone together to get acquainted with things

  • Identifying reporting-specific needs as it relates to the Folio community - like need for good communication in development; influencing and advocating for FOLIO governance practices that meet the needs of the reporting community

  • Helping other SIGs - outreach to other SIGs, maintaining strong ties for information flow/support of each other's work

    • making sure we're not duplicating effort, making sure we're not working on something that is already available in-app

    • giving feedback to FOLIO app developers on what is needed to support reporting

  • bridging the knowledge gap between the technical and non-technical (functional?) people (or devs/SMEs)

  • advocate for resources for reporting

  • Advertise benefits of SIG membership/participation:

    • Benefits to community - other perspectives and workflows

    • Benefits to institutions - visibility, validation

    • Benefits to individuals getting involved - personal development and skills (e.g., SQL, GitHub, networking/connections)

  • documentation

  • help clarify the differences between different FOLIO reporting solutions (different software, different hosting options)

Supporting the LDP tools

  • the growing LDP ecosystem (LDP 1.x, Metadb, LDLite, LDP MARC, the LDP app, FOLIO Analytics) - committed to being active stakeholders in these systems

  • not writing the code for the LDP project software, but contributing substantial testing, QA, specifying new features needed, troubleshooting problems as they come up; making it useful; feedback on system design proposals; develop reporting tools that serve the needs of different kinds of users

  • aligning reporting software development with the FOLIO application software development

  • providing community support for using the LDP project software and queries;

    • but for scope, how long do we continue to answer questions about past versions of the software

    • maybe build an FAQ???

  • developing/testing/updating derived table and report queries tailored for LDP project systems (scope question: how regularly/for how long)

    • open the repository for non-SIG members and review their queries; may need to discuss, does this make sense, since it adds work?

    • influencing the shape of the product by building derived table queries - responding directly to user needs by designing helpful tables on top of the basic structure

  • maintaining documentation

  • gather requirements from the broader FOLIO community related to reporting needs, document them, advocate for them

    • providing information about what our users need, through needs assessment

    • create/maintain connections to functional area SIGs

    • be responsive to new apps and functional areas

  • connecting non-FOLIO datasets to FOLIO data

    • especially ReShare, because the reporting platform is part of the LDP ecosystem

    • even for non-ReShare data, this work is well supported by the LDP ecosystem

    • legacy system data

    • being able to help people work on non-FOLIO data ingest and reporting (example: ARES data into LDP) - maybe out of scope, but where does the line go?

    • importing data into the LDP on the fly - little tables, etc.

      • bringing an Excel file into the LDP in local schema to work with it, figuring out how to change data attributes to make it work

      • getting a full, large dataset into the LDP as a different schema

Supporting other types of FOLIO reporting

  • additional reporting solutions (e.g., direct reporting against FOLIO APIs, in-app FOLIO reports, additional third-party tools) - preference for open over closed tools; serve as a resource to other reporting solutions looking for feedback or partnership

    • in order to have that, there has to be engagement from someone who wants support for that tool; SIG is not responsible for being proactive about supporting new tools

    • as a SIG we can offer advice on the things that we have knowledge about within the SIG; if a third-party tool has no connection to the SIG, we won't have enough knowledge about that tool to directly offer support

    • would be nice to frame the scope in the positive: we have expertise in certain tools and techniques, and we're happy to share/support that

  • in past, have been asked to comment on/help design features for in-app reporting (still in scope?)

  • for tools we don't have direct access to, can we support users of those tools? help with SQL, understanding FOLIO data might be topics that cut across tools; should part of our vision be to welcome non-LDP users into the group more?

Updates and Query Demonstrations from Various Reporting Related Groups and Efforts

Community & Coordination, Reporting Subgroup Leads

Project updates

Reporting development is using small subgroups to address priorities and complete work on report queries.  Each week, these groups will share reports/queries with the Reporting SIG.  Reporting development team leads are encouraged to enter a summary of their work group activities below.

RA/UM Working Group

 

MM Working Group

  • Meetings are 1st Tuesday of the month, 12-1pm ET via zoom using the usual FOLIO password. Our lab sessions are open to everyone. Please bring your questions, examples, and comments about reporting and metadata.

  • We are still looking for help with derived tables and reviewers. See our sign up sheet.

  • Our next session will go through the review process and one more example of porting over a derived table from LDP to metaDB.

 

ERM Working Group

  • no updates due to vacation absences

  • ERM Prototype and Query Development Status

  • Meetings are bi-weekly on tuesdays 11am ET alternating with RM Working Group

    • Next meeting will be at 26th, April

    • Contact @Axel Dörrer if you would like to get a calendar invitation.

 

RM Working Group

 

Reporting SIG Documentation Subgroup

  • Honeysuckle documentation is live on https://docs.folio.org/docs/

  • Iris documentation is in progress, due December 15

  • Additional Context

    • The Reporting SIG has representation on the Documentation Working Group, which is building end-user documentation for https://docs.folio.org/docs/ (mostly linking to existing documentation over on GitHub)

 

External Statistics Working Group

  • no updates currently

  • new organizational/tracking scheme for JIRA, with pointers to queries in folio-analytics repository

  • New organizational structure for External Statistics reports

    • external statistics reports (e.g., ACRL) typically require running queries from different functional reporting areas

    • these reports will be captured in JIRA under one UXPROD-XXXX report cluster issue, then the descriptions will point to each of the queries required to run them on the folio-analytics repository

    • institutions will need to rank each of these 8 new UXPROD-XXXX report cluster issues

    • each reporting development team will take responsibility for the queries in their area for the external statistics clusters

 

Product Council

 

 

For all recent work on FOLIO Reporting SQL development:

 

Topics for Future Meetings

All

  • Next: Reviewing a pull request 

  • How to deal with External Stats reports?

    • maybe subteam leads check in about that

    • probably wait until after Metadb conversion is more complete

  • Converting queries to dashboards: Sharon and Vandana (May 9, tentative)

  • hosting experiences from implementers

 

Review and update Topics for Future Reporting SIG Meetings 

 

 

 

 

A test Action Item (Ingolf)