2022-02-07 Reporting SIG Meeting notes

Date

07 Feb 2022

Attendees

Present?	Name	Organization	Present?	Name	Organization
x	Arthur Aguilera	University of Colorado, Boulder	x	Linda Miller	Cornell University
x	Sharon Beltaine	Cornell University	x	Nassib Nassar	Index Data
x	Erin Block	University of Colorado, Boulder	x	Elena O'Malley	Emerson
	Nancy Bolduc	Cornell University		Tod Olson	University of Chicago
	Lloyd Chittenden	Marmot	x	Jean Pajerek	Cornell University
x	Axel Doerrer	University Mainz	x	Michael Patrick	The University of Alabama
	Shelley Doljack	Stanford University	x	Eric Pennington	Texas A&M
x	Stefan Dombek	Leipzig University		Scott Perry	University of Chicago
x	Jennifer Eustis	U. Massachusetts Amherst / Five College	x	Natalya Pikulik	Cornell University
x	Alissa Hafele	Stanford University	x	Vandana Shah	Cornell University
	Ingolf Kuss	hbz	x	Amelia Sutton	U. Massachusetts
	Kim Laine	Cornell University	x	Simona Tabacaru	Texas A&M
	Joanne Leary	Cornell University	x	Kevin Walker	The University of Alabama
x	Eliana Lima	Fenway Library Organization	x	Angela Zoss	Duke University
	Eric Luhrs	Lehigh University

Absent

Ingolf Kuss (due to a conflicting appointment)

Discussion Items

Item	Who	Notes
Attendance & Notes	Angela	Attendance & Notes Today's attendance-taker: Linda Miller Today's note-takers: Team Leads for project updates
Announcements / Reminders	Angela	Recruiting New Query Developers The Reporting SIG is always on the look-out for new query developers. Please let us know if you are interested in doing query development or if there are others at your institution who might be a good fit.
The latest from Folio Analytics	All	Recent updates to queries on our Folio Analytics repository https://github.com/folio-org/folio-analytics/commits/main
Migration tables for Metadb	Nassib	New "migration tables" allow Metadb to run LDP 1.x queries: https://github.com/folio-org/folio-analytics/tree/main/sql_metadb/migration_tables This enables a Metadb database to be effectively compatible with LDP 1.x queries and derived tables.. The tables are designed to be loaded into a Metadb database once every evening, until all legacy queries have been migrated to use the new "near real-time" Metadb tables. 97 out of 115 LDP 1.x tables are currently emulated, including 674 columns. The remaining tables may be added, pending more information on the location of source data. The status of each table is tracked at: https://docs.google.com/spreadsheets/d/19iP5hnUHqCuGMM5OQKdUWMNCT75ZK3EDa0Vx8LgUZ24/edit?usp=sharing Additional columns may be submitted via pull request. More information at: https://github.com/folio-org/folio-analytics/blob/main/sql_metadb/migration_tables/README.md
Intro to Git	Angela	(today) Intro to Version Control with Git and GitHub (eventually) Git interfaces: GitHub, GitHub Desktop, Git plugin for DBeaver Notes: "Git" __ is tracking\|version changes between files. init __ initialize that you are wanting to track changes. "Staging" __ files currently working on. add __ file(s) wanting to track, adding to staging, changes made to. commit message __ describe the change\|addition\|deletions. commit __ saving point for an individual\|group changes. Goes into your git log.
Updates and Query Demonstrations from Various Reporting Related Groups and Efforts	Community & Coordination, Reporting Subgroup Leads	Project updates Reporting development is using small subgroups to address priorities and complete work on report queries. Each week, these groups will share reports/queries with the Reporting SIG. Reporting development team leads are encouraged to enter a summary of their work group activities below. RA/UM Working Group Meetings have become more of a lab session, working through specific problems Angela has been attending RA SIG meetings to open the lines of communication, now once a month Contact Angela if you would like to join these meetings; second Wednesdays at 1pm Eastern (but for Feb, actually Tuesday the 8th at 2pm Eastern) Context Meeting notes: https://docs.google.com/document/d/1UnzG64tl917LOH2FtWhCEPlSOsnJWKL-0eu88Ouo1DU/edit Current status of RA/UM issues: https://github.com/folio-org/folio-analytics/issues?q=is%3Aissue+is%3Aopen+RA%2FUM MM Working Group Meetings are 1st Tuesday of the month, 12-1pm ET via zoom using the usual FOLIO password. Our lab sessions are open to everyone. Please bring your questions, examples, and comments about reporting and metadata. We have began porting over LDP derived tables to Metadb. We have a sign up sheet. If you'd like to participate, please add your name. If you aren't in the our mm reports Slack channel, let Jennifer know so that you can be added. Current pull requests for: item extended derived metadb table instance alternative titles derived metadb table ERM Working Group Reviewed ERM Goals 2022 Complete documentation on existing derived tables Migrate LDP queries to Metadb (ERM?) Move on with derived tables portfolio for ERM Getting ERM SIG and implementers more involved for reporting requirements Still housekeeing: Github issues left overs, how long to keep meeting recordings etc. ERM Prototype and Query Development Status Meetings are bi-weekly on tuesdays 11am ET alternating with RM Working Group Next meeting will be at 15th, february Contact Axel Dörrer if you would like to get a calendar invitation. RM Working Group group is working on RM-related derived tables and queries for Metadb working on LDP to Metadb table mapping spreadsheet to prepare for transition to Metadb looking at using https://mermaid-js.github.io/mermaid/#/ to document data models in RM areas of FOLIO several queries for RM completed, but still need documentation, testing, and review for latest updates, see RM Prototype and Query Development Status Meetings are biweekly on Tuesdays 11am-noon ET; contact Sharon Markus if you would like to join us Reporting SIG Documentation Subgroup Honeysuckle documentation is live on https://docs.folio.org/docs/ Iris documentation is in progress, due December 15 Additional Context The Reporting SIG has representation on the Documentation Working Group, which is building end-user documentation for https://docs.folio.org/docs/ (mostly linking to existing documentation over on GitHub) External Statistics Working Group no updates currently new organizational/tracking scheme for JIRA, with pointers to queries in folio-analytics repository New organizational structure for External Statistics reports external statistics reports (e.g., ACRL) typically require running queries from different functional reporting areas these reports will be captured in JIRA under one UXPROD-XXXX report cluster issue, then the descriptions will point to each of the queries required to run them on the folio-analytics repository institutions will need to rank each of these 8 new UXPROD-XXXX report cluster issues each reporting development team will take responsibility for the queries in their area for the external statistics clusters Product Council For all recent work on FOLIO Reporting SQL development: https://github.com/folio-org/folio-analytics/commits/main
Topics for Future Meetings	All	Follow-up on MARC status, Quickmarc/Data Import conflicts How to strengthen connections to SIGs and their developers to be kept in the loop about changes to the data model Show and tell how are institutions using the LDP examples of using the local schema Cornell's report ticketing system Rollout plans from institutions Ask someone on the sysadmin side to talk about LDP administration (Jason Root?) What is done in JIRA? (JIRA clean up) Training topics adding test data in FOLIO snapshot How to do ad hoc querying with the derived tables How to use the LDP app using KNIME to build reports (LDP edition) use of local schema for custom tables more on MARC (ask Jennifer) using different applications (other than DBeaver) Insomnia for API queries? SQL LDP to metadb Metadb implications if schema changes, will that be relatively seamless? old fields still in history, but new fields in current? if fields are deprecated and removed, what happens if they come back again? how will deleted records show up? Git/GitHub Panorama LDLite (again?) Discussion: how to rank clusters with institutional rankings going away? consortia SIG is talking about a central office app of some kind; that app might need to deal with consortia-wide reporting how to train: wiki pages with examples? form a small project team? (e.g., how to pull from marc, how to use local schema) each dev team create training about the data structures in that area? Query style (ask Nassib) Gathering institutional query repositories; someone can propose a new page or a new addition to an existing page on FOLIO Analytics? Follow up on how devs can work with SMEs better to decide on and advertise data model changes Upcoming: SQL advice/query optimization (Axel) query demo - MARC (Tod) Intro to SQL training (as time allows) revisit discussion on openness/transparency in communication and open source software; decide as a group what we would like to advocate for reports from implementers on experiences with hosting solutions Review and update Topics for Future Reporting SIG Meetings

A test Action Item (Ingolf)