2024-03-11 Reporting SIG Meeting notes

Announcements/Reminders

Changes in Reporting SIG Meetings

• Reporting SIG meetings will take place at 11am ET on the 2nd and 4th Mondays of the month only starting Monday February 12, 2024
  
• "Workshopping your queries" - part of each Reporting SIG meeting will be devoted to time to work through any query questions you may have

Cookbook Documentation Updates (Stefan)

• Stefan updated the cookbook Finances and added the chapter "Vouchers" https://github.com/folio-org/folio-analytics/wiki/Cookbook:-Finances#4-vouchers

Impacts of New Fields and Features (Sharon)

• Are there any new fields or features we should know about from any of the functional areas that may impact reporting?
• Review and update the Impacts of New Fields and Features on Reporting grid 

Upcoming meeting topics (tentative)

• Open Refine
• LDP to Metadb Transition Overview

Any new members?

• Welcome/introductions

• Representative for the Documentation Working Group 
  • urgently needed
  • work with reporting documentation team to  update Reporting section of FOLIO documentation for each FOLIO software release 
  • see https://docs.folio.org/docs/reporting/
• Query developers
  
  • ongoing need
  • work on derived tables and report queries for the FOLIO Analytics repository and to support Metadb
  • see https://github.com/folio-org/folio-analytics

Metadb Update:

• Nassib will provide us with an update on the Metadb project and software
• Metadb repository: https://github.com/metadb-project/metadb

Notes

Metadb 1.2

• Metadb version 1.2 is the latest stable release
• See documentation at https://metadb.dev/doc/
• LDP uses nightly data transfer, while Metadb does continuous, streaming, real-time updates to the data 
• metadb.base_table stores a list of all the tables metadb (from FOLIO) is managing
• metadb is designed to support more than one source of data
• transformed boolean true means table is transformed from another table, e.g., parent_schema_name and parent_table_name
• metadb.log is like a system log for metadb, reporting errors and warnings; you can call the mdblog function (see functions section) to see log messages from the last 24 hours
• metadb.table_update shows information about the updating of MARC and derived tables (for now) and plans are for it to cover all tables in the future; good for troubleshooting
• ALTER TABLE can be used by a system administrator to change the data type of a column to uuid if that is the preference (ERM, for instance, does not use the uuid type so they are inferred by metadb)
• As of Metadb version 1.2, the default data type is text. You can still use VARCHAR in your SQL.
• Metadb attaches to the FOLIO db replication system to provide the functionality of streaming data from the database to get changes; if something happens to break the stream of data, your devops team needs to resynchronize the data as a way of doing recovery on the data; the sync command reads the entire source database as a snapshot; then the devops team runs the endsync command to clean up the data; you can still use Metadb when the sync process is running, but you can't be sure the data is up-to-date until this process is complete; historical data is lost when the sync is broken; generally speaking, it is not necessary to stop the streaming when the FOLIO source database is getting updated to a new software release; Index data stops the sync and then does a resync

Metadb 1.3

• new feature to important historical LDP data into the main tables in Metadb
• performance improvements to the resynchronization process

Here is a summary of information about WOLFCon 2024 from Jesse Koennecke:

• Location and Dates: Senate House, University of London, September 24-26, 2024
• Call for Proposals Now Open: Got ideas about open-source to share? Talk about it at WOLFcon. Submit a presentation, panel, short talk, or pre-conference workshop. The deadline for submissions is March 31, 2024. Submit a session here.
• Early Bird Registration Now Open: Join us at Senate House, University of London. September 24-26, 2024. Register now through July 31, 2024 for an early bird discounted rate.
• Learn more about WOLFcon 2024: Want to learn more about the Open Library Foundation and WOLFcon? Be sure to visit our website where you can learn more about the foundation, members projects, communities, and the annual conference.

Reporting SIG Planning for WOLFCon 2024:

• Please help us with topics and ideas for reporting at the WOLFCon 2024 conference! 
• See ideas we have so far at  WolfCon 2024 Planning

Workshopping Queries

Bring your questions on queries written for the LDP or Metadb so we can "workshop" them! This week we will focus on some new updates for derived tables related to finance from Stefan.

• (Stefan) Required updates for derived tables with transactions have been created (LDP and Metadb) Need reviewers:
  
  • finance_invoice_transactions https://github.com/folio-org/folio-analytics/pull/831
  • finance_transaction_invoices https://github.com/folio-org/folio-analytics/pull/832
  • We need an additional column to create correct calculations of the transaction amounts. This particularly applies to the “Credit” transaction types. Leipzig and Cornell confirm this using live data.
  • folio-analytics wiki: https://github.com/folio-org/folio-analytics/wiki/Cookbook:-Finances#221-transaction-amounts-for-credits

The cookbooks are currently being created step by step and are located in the folio-analytics wiki.

https://github.com/folio-org/folio-analytics/wiki

2024-02-29 Product Council Agenda and Meeting Notes. At the last PC meeting, the Prioritization and Roadmap working group continued with their discussion on the test in Acquisitions SIG to implement a clearer method to tracking development. This is a prototype that PC will consider its viability for SIGs in general.

There's some movement on how to track changes in property names. More to come.

WOLFCon 2024! Think of registering and/or submitting.

3/14/2024 PC and SIG conveners meeting.

3/21/2024 PC Asia friendly time meeting 8pm EST