/
2024-03-11 Reporting SIG Meeting notes

2024-03-11 Reporting SIG Meeting notes

Mar 11, 2024

Date

Mar 11, 2024

Attendees

Present?

Name

Organization

Present?

Name

Organization

Present?

Name

Organization

Present?

Name

Organization

 

Arthur Aguilera

University of Colorado, Boulder

 

Eliana Lima

Fenway Library Organization

x

Sharon Beltaine

Cornell University

 

Eric Luhrs

Lehigh University

 

Erin Block

University of Colorado, Boulder

 

Lisa McColl

Lehigh University

 

Kathleen Moore

 

x

Linda Miller

Cornell University

 

Shannon Burke

Texas A&M

x

Nassib Nassar

Index Data

 

Suzette Caneda

Stanford University

x

Elena O'Malley

Emerson

x

Dung-Lan Chen

Skidmore College

 

Tod Olson

University of Chicago

 

Lloyd Chittenden

Marmot

 

Jean Pajerek

Cornell University

 

Tim Dannay

Mount Holyoke College

 

Kimberly Pamplin

Texas A&M University 

 

Axel Doerrer

University Mainz

x

Scott Perry

University of Chicago

 

Shelley Doljack

Stanford University

x

Natalya Pikulik

Cornell University

x

Stefan Dombek

Leipzig University

x

Emily Sanford

Michigan State University

x

Jennifer Eustis

U. Massachusetts Amherst / Five College

 

Bob Scheier

Holy Cross

 

Lynne Fors

Wellesley College

 

Vandana Shah

Cornell University

 

Lisa Furubotten

Texas A&M

 

Linnea Shieh

Stanford University

 

Alissa Hafele

Stanford University

 

Susie Skowronek

Oakland University

 

Kara Hart

Wellesley College

x

Kimberly Smith

Middle Tennessee State University

x

Corrie Hutchinson

Index Data

 

Clare Spitzer

Stanford University

 

Jamie Jesanis

MCPHS

 

Amelia Sutton

U. Massachusetts

 

Jeanette Kalchik

Stanford University

 

Simona Tabacaru

Texas A&M

 

Harry Kaplanian

Ebsco

 

Huey-Ning Tan

Stanford University

 

Tim Kiser

Michigan State University

 

Vitus Tang

Stanford University

x

Kevin Kishimoto

Stanford University

 

Christie Thomas

University of Chicago

 

Ingolf Kuss

hbz

x

Irina Trapido

Stanford University

 

Alexander Lao

Stanford University

 

Catherine Tuohy

Emmanuel College

 

Joanne Leary

Cornell University

x

Patrick Waite

U. Mass Amherst

 

 

 

 

Joseph Molloy

Spokane Public Library

Discussion Items

Item

Who

Notes

Item

Who

Notes

Attendance & Notes

Sharon

Attendance & Notes

  • Today's attendance-taker: Linda (or substitute)

  • Today's note-takers:  Team Leads for project updates

 

Announcements/Reminders

Sharon

Changes in Reporting SIG Meetings

  • Reporting SIG meetings will take place at 11am ET on the 2nd and 4th Mondays of the month only starting Monday February 12, 2024

  • "Workshopping your queries" - part of each Reporting SIG meeting will be devoted to time to work through any query questions you may have

Cookbook Documentation Updates (Stefan)

Impacts of New Fields and Features (Sharon)

 

Upcoming meeting topics (tentative)

  • Open Refine

  • LDP to Metadb Transition Overview

 

Any new members?

  • Welcome/introductions

 

How to find our latest recordings

 

SIG Recruitment

We will need to be recruiting for a variety of roles in the coming months. Please consider whether you would be interested. Reach out to @Scott Perry  or @Sharon Markus with any questions.

  • Representative for the Documentation Working Group 

  • Query developers

 

Metadb Update

Nassib Nassar

Metadb Update:

 

Notes

Metadb 1.2

  • Metadb version 1.2 is the latest stable release

  • See documentation at https://metadb.dev/doc/

  • LDP uses nightly data transfer, while Metadb does continuous, streaming, real-time updates to the data 

  • metadb.base_table stores a list of all the tables metadb (from FOLIO) is managing

  • metadb is designed to support more than one source of data

  • transformed boolean true means table is transformed from another table, e.g., parent_schema_name and parent_table_name

  • metadb.log is like a system log for metadb, reporting errors and warnings; you can call the mdblog function (see functions section) to see log messages from the last 24 hours

  • metadb.table_update shows information about the updating of MARC and derived tables (for now) and plans are for it to cover all tables in the future; good for troubleshooting

  • ALTER TABLE can be used by a system administrator to change the data type of a column to uuid if that is the preference (ERM, for instance, does not use the uuid type so they are inferred by metadb)

  • As of Metadb version 1.2, the default data type is text. You can still use VARCHAR in your SQL.

  • Metadb attaches to the FOLIO db replication system to provide the functionality of streaming data from the database to get changes; if something happens to break the stream of data, your devops team needs to resynchronize the data as a way of doing recovery on the data; the sync command reads the entire source database as a snapshot; then the devops team runs the endsync command to clean up the data; you can still use Metadb when the sync process is running, but you can't be sure the data is up-to-date until this process is complete; historical data is lost when the sync is broken; generally speaking, it is not necessary to stop the streaming when the FOLIO source database is getting updated to a new software release; Index data stops the sync and then does a resync

Metadb 1.3

  • new feature to important historical LDP data into the main tables in Metadb

  • performance improvements to the resynchronization process

 

WOLFCon 2024

Sharon, Scott

Here is a summary of information about WOLFCon 2024 from Jesse Koennecke:

  • Location and Dates: Senate House, University of London, September 24-26, 2024

  • Call for Proposals Now Open: Got ideas about open-source to share? Talk about it at WOLFcon. Submit a presentation, panel, short talk, or pre-conference workshop. The deadline for submissions is March 31, 2024. Submit a session here.

  • Early Bird Registration Now Open: Join us at Senate House, University of London. September 24-26, 2024. Register now through July 31, 2024 for an early bird discounted rate.

  • Learn more about WOLFcon 2024: Want to learn more about the Open Library Foundation and WOLFcon? Be sure to visit our website where you can learn more about the foundation, members projects, communities, and the annual conference.

Reporting SIG Planning for WOLFCon 2024:

  • Please help us with topics and ideas for reporting at the WOLFCon 2024 conference! 

  • See ideas we have so far at  WolfCon 2024 Planning

 

Workshopping Queries

Stefan, Christie

Bring your questions on queries written for the LDP or Metadb so we can "workshop" them! This week we will focus on some new updates for derived tables related to finance from Stefan.

 

Project "Build Metadb query cookbook-style documentation"

Stefan

The cookbooks are currently being created step by step and are located in the folio-analytics wiki.

https://github.com/folio-org/folio-analytics/wiki

FOLIO Product Council Update

Jennifer Eustis

2024-02-29 Product Council Agenda and Meeting Notes. At the last PC meeting, the Prioritization and Roadmap working group continued with their discussion on the test in Acquisitions SIG to implement a clearer method to tracking development. This is a prototype that PC will consider its viability for SIGs in general.

There's some movement on how to track changes in property names. More to come.

WOLFCon 2024! Think of registering and/or submitting.

3/14/2024 PC and SIG conveners meeting.

3/21/2024 PC Asia friendly time meeting 8pm EST

 

Recurring Items (Updated weekly, but not always discussed in meeting)

Item

Who

Notes

Item

Who

Notes

Review of In-Progress Projects (Recurring)

 

Review the release notes for FOLIO Analytics, LDP, LDLite, LDP Reporting App, ldpmarc, Metadb Projects (Recurring)

 

Updates and Query Demonstrations from Various Reporting Related Groups and Efforts Projects (Recurring)

Community & Coordination, Reporting Subgroup Leads

Project updates

Reporting development is using small subgroups to address priorities and complete work on report queries.  Each week, these groups will share reports/queries with the Reporting SIG.  Reporting development team leads are encouraged to enter a summary of their work group activities below.

D-A-CH Working Group (D-Reporting)

  • Ongoing topics

    • DBS statistics

      • JIRA-Issues, Rpt-Clusters

      • Gap analysis to statistic codes, user counts etc.

      • Identifying functions that we need for statistics in Germany but are not yet implemented in FOLIO

  • Meetings

    • Contact @Stefan Dombek if you would like to get a calendar invitation

 

Metadb FOLIO User Group

  • Meetings

    • Contact @Nassib Nassar if you would like to get a calendar invitation

 

Product Council

 

Reporting SIG Documentation Subgroup

  • Orchid documentation is live on https://docs.folio.org/docs/

  • Poppy documentation is in development

  • Additional Context

    • The Reporting SIG has representation on the Documentation Working Group, which is building end-user documentation for https://docs.folio.org/docs/ (mostly linking to existing documentation over on GitHub)

 

For all recent work on FOLIO Reporting SQL development: