Page Comparison

Attendance

Attendance & Notes

• Today's attendance-taker: Linda (or substitute)

• See attendance list at bottom of page
  

Announcements and Reminders

Sharon

Announcements:

• About the Reporting SIG meeting schedule

  • Meetings are held on the first 4 Mondays of each month at 11:00 am Eastern U.S. time (see this time in your time zone). The first and third Mondays focus on Reporting Development topics, and the second and fourth Mondays focus on Reporting Business topics. 

  • "business" means topics like presentations on reporting functionality and new features, new reporting applications, surveys and studies on reporting, etc.

  • "development" means working on derived tables and report queries for the folio-analytics GitHub repository

  • "workshopping" queries could be scheduled during any of these meetings, and it would be great to have topics and/or questions in advance so we can prepare to walk through the answers/approaches, such as "how to I fix this inventory query to get rid of the duplicates?" or "what is the best way to calculate totals in this finance query?" 

• Important Upcoming Changes to LDP software support:

  • end of support is May 2025

  • see Mike Gorrell at Index Data for questions

  • institutions running LDP on a reporting database are encouraged to move to running Metadb software
    

Ongoing Topics:

• Workshopping your queries

  • part of each Reporting SIG business meeting will be devoted to time to work through any query questions you may have

  • please reach out to Christie Thomas if you have a question you would like to "workshop" during an upcoming Reporting SIG meeting

• Impacts of New Fields and Features (Sharon)

  • Are there any new fields or features we should know about from any of the functional areas that may impact reporting?

  • Review and update the Impacts of New Fields and Features on Reporting grid 

• Upcoming Reporting SIG meeting topics (tentative)
  

  • Moving from LDP to Metadb

  • Updates from the Data Array Extraction for Metadb 1.4 Workgroup

• Any new members?

  • Welcome/introductions

• How to find our latest recordings

  • Recording details available on our Reporting SIG Meetings and Notes page and Reporting SIG Subteam Meetings page

SIG Recruitment:

We will need to be recruiting for a variety of roles in the coming months. Please consider whether you would be interested. Please reach out to Scott Perry or Sharon Markus with any questions.

• Representative for the Documentation Working Group 

  • urgently needed

  • work with reporting documentation team to  update Reporting section of FOLIO documentation for each FOLIO software release 

  • see https://docs.folio.org/docs/reporting/

• Query developers

  • ongoing need

  • work on derived tables and report queries for the FOLIO Analytics repository and to support Metadb

  • see https://github.com/folio-org/folio-analytics

User-Defined Functions (UDF) with your Metadb reporting databaseMetadb Mapping Next Steps

Mike Gorrell

Stefan Dombek will give a presentation on implementing “User-Defined Functions (UDF): An introduction and comparison to derived tables” with your Metadb reporting database.

• UDF (User-Defined Function)

  • Explanation

  • Define your own function

  • Use of UDF in a query

• Comparison to derived tables

  • Performance

  • Derived tables vs. UDF

Slides: https://docs.google.com/presentation/d/1Emn3tERFwaQw-_1DlLYJhuKEA4XONwXFPWiav6n5U3E/edit?usp=sharing

See FUNCTION examples for the Reporting App here:
https://github.com/stdombek/test_folio_app_reporting As posted on the FOLIO slack project channel, #metadb-mapping -

I have clarity around which tables the transformed data will appear in. There is an image of the data we are working with in the Metadb Documentation under section 2.5.4 Create Data Mapping:

Image Added

And the CREATE DATA MAPPING commands:

CREATE DATA MAPPING FOR json
    FROM TABLE library.inventory__ COLUMN jsondata PATH '$'
    TO 't';
CREATE DATA MAPPING FOR json
    FROM TABLE library.inventory__ COLUMN jsondata PATH '$.metadata'
    TO 'metadata';
CREATE DATA MAPPING FOR json
    FROM TABLE library.inventory__ COLUMN jsondata PATH '$.tags'
    TO 'tags';
CREATE DATA MAPPING FOR json
    FROM TABLE library.inventory__ COLUMN jsondata PATH '$.tags.tagList'
    TO 'taglist';

Mapping a JSON object creates columns, and mapping a JSON array creates a table.  The only array in this example is tagList.  This results in an inventory__taglist table.  Everything else results in columns in the corresponding parent table, which in this case is inventory__t.The outermost object is a special case because there is no parent table.  So it results in the top-level inventory__t table.But if you imagine there is already a table inventory__t, the top-level columns and nested objects are added to it.

Review of In-Progress Projects (Recurring)

• Build a directory of extracted and derived tables

  • Beta version available.

• Build Metadb query cookbook-style documentation

  • ongoing project

  • The first cookbooks are available and are being expanded. See https://github.com/folio-org/folio-analytics/wiki#cookbooks

• Build Training Program for FOLIO Data Model

  • ACQ/ERM drafts have been presented and will be updated

• Recruit from new institutions not represented

  • recommend that the development teams reach out to SIGs for an update, invite new members

Review the release notes for FOLIO Analytics, LDP, LDLite, LDP Reporting App, ldpmarcmarct, Metadb Projects (Recurring)

• LDP release notes

• Metadb release notes (via tags) - (includes marct for MARC data ETL)

• LDLite release notes (via tags)

• ldpmarc release notes

• LDP Reporting App release notes (mod-ldp, ui-ldp) and JIRA issues (mod-ldp, ui-ldp)

• FOLIO Analytics release notes 

Updates and Query Demonstrations from Various Reporting Related Groups and Efforts Projects (Recurring)

Community & Coordination, Reporting Subgroup Leads

Project updates

Reporting development is using small subgroups to address priorities and complete work on report queries.  Each week, these groups will share reports/queries with the Reporting SIG.  Reporting development team leads are encouraged to enter a summary of their work group activities below.

D-A-CH Working Group (D-Reporting)

• A few libraries in Germany would like to start using Metadb soon. In most cases, hosting is in own data centers. The working group needs experience reports on what resources are required.

• Ongoing topics: Workshopping, DBS statistics

  • Unification of reference data when mapping data from German union catalogs to FOLIO. Collaboration with the MM working group (D-A-CH).

  • Statistics after data anonymization according to GDPR. Collaboration with the RA/UM working group (D-A-CH).

• Meetings: Contact Stefan Dombek if you would like to get a calendar invitation

Reporting SIG Documentation Subgroup

• Quesnelia documentation is live on https://docs.folio.org/docs/

• Ramsons documentation is in development

• Additional Context

  • The Reporting SIG has representation on the Documentation Working Group, which is building end-user documentation for https://docs.folio.org/docs/ (mostly linking to existing documentation over on GitHub)

For all recent work on FOLIO Reporting SQL development:

• https://github.com/folio-org/folio-analytics/commits/main

FOLIO Product Council

Jennifer Eustis

The PC is asking SIGs to adopt a new prioritization process. This involves using Jira more to keep track of issues important to the SIG.

2024-11-21 Product Council Agenda & Meeting Notes