Page Comparison

Attendance

Attendance & Notes

• Today's attendance-taker: Linda (or substitute)
  

Announcements and Reminders

Scott

Announcements:

• About the Reporting SIG meeting schedule

  • Meetings are held on the first 4 Mondays of each month at 11:00 am Eastern U.S. time (see this time in your time zone). The first and third Mondays focus on Reporting Development topics, and the second and fourth Mondays focus on Reporting Business topics. 

  • "business" means topics like presentations on reporting functionality and new features, new reporting applications, surveys and studies on reporting, etc.

  • "development" means working on derived tables and report queries for the folio-analytics GitHub repository

  • "workshopping" queries could be scheduled during any of these meetings, and it would be great to have topics and/or questions in advance so we can prepare to walk through the answers/approaches, such as "how to I fix this inventory query to get rid of the duplicates?" or "what is the best way to calculate totals in this finance query?" 

• November 25 and December 23 meetings are canceled

Ongoing Topics:

• Workshopping your queries

  • part of each Reporting SIG business meeting will be devoted to time to work through any query questions you may have

  • please reach out to Christie Thomas if you have a question you would like to "workshop" during an upcoming Reporting SIG meeting

• Impacts of New Fields and Features (Sharon)

  • Are there any new fields or features we should know about from any of the functional areas that may impact reporting?

  • Review and update the Impacts of New Fields and Features on Reporting grid 

• Upcoming Reporting SIG meeting topics (tentative)
  

  • Derived Tables

  • Reporting App use at various institutions

  • More Metadb training

• Any new members?

  • Welcome/introductions

• How to find our latest recordings

  • Recording details available on our Reporting SIG Meetings and Notes page and Reporting SIG Subteam Meetings page

SIG Recruitment:

We will need to be recruiting for a variety of roles in the coming months. Please consider whether you would be interested. Please reach out to Scott Perry or Sharon Markus with any questions.

• Representative for the Documentation Working Group 

  • urgently needed

  • work with reporting documentation team to  update Reporting section of FOLIO documentation for each FOLIO software release 

  • see https://docs.folio.org/docs/reporting/

• Query developers

  • ongoing need

  • work on derived tables and report queries for the FOLIO Analytics repository and to support Metadb

  • see https://github.com/folio-org/folio-analytics

Mapping Working Group Instructions:

Metadb version 1.4 mappings for data array extraction

Mike

This will be a working group meeting where Mike Gorrell, LDP/Metadb Project Director at Index Data, will walk us through the process of preparing mappings for data array extraction for the upcoming release of Metadb 1.4 software

• Deployed in 2 environments (CU Boulder and Lehigh)

• Testing to be conducted over the next few weeks:

  • Add Mappings to correspond to some of the derived tables (See example below). Run queries using those new mappings.

    • Can people help with these mappings? Note the documentation in section 2.5.4

  • Add Mappings of more complex extractions (thank you for some test cases to use I will publish the list we have later this week). Run queries using those new mappings

  • ReSync to measure impact of CREATE MAPPING - First or second week of December

• Working group being formed to create the mapping over the next several weeks.

• Close to collecting daily statistics on Derived Table usage

Update on new FOLIO Reporting Environment

Mike

The task:

1. We have over 80 Derived Tables scripts that currently extract JSON objects/arrays

2. Metadb 1.4 can do this for us automatically by creating appropriate MAPPING commands

3. We need to create the MAPPINGs

For today:

Review Metadb documentation which outlines how to create these mappings

Example of a Mapping created from a derived table creation script. From the po_lines_cost.sql script:

• FROM:

jsonb_extract_path_text(pol.jsonb, 'cost', 'listUnitPrice')::numeric(19,4) AS po_line_list_unit_price_phys,
jsonb_extract_path_text(pol.jsonb, 'cost', 'quantityPhysical') AS po_line_quant_phys,
jsonb_extract_path_text(pol.jsonb, 'cost', 'listUnitPriceElectronic')::numeric(19,4) AS po_line_list_unit_price_elec,
jsonb_extract_path_text(pol.jsonb, 'cost', 'quantityElectronic') AS po_line_quant_elec,
jsonb_extract_path_text(pol.jsonb, 'cost', 'additionalCost')::numeric(19,4) AS po_line_additional_cost,
jsonb_extract_path_text(pol.jsonb, 'cost', 'currency') AS po_line_currency,
jsonb_extract_path_text(pol.jsonb, 'cost', 'discount')::numeric(19,4) AS po_line_discount,
jsonb_extract_path_text(pol.jsonb, 'cost', 'discountType') AS po_line_discount_type,
jsonb_extract_path_text(pol.jsonb, 'cost', 'poLineEstimatedPrice')::numeric(19,4) AS po_line_estimated_price

TO this in the Mapping file

CREATE DATA MAPPING FOR json FROM TABLE folio_orders.po_line__ COLUMN jsonb PATH '$.cost' TO 'cost';

note that there is no way to limit the fields extracted, or to rename the columns as part of this feature.

NOTE LDP Software retirement is May 1, 2025

Meta analysis. I have written a script that tries to collect all Fields+Tables that are being extracted. This might speed up the analysis

Divvying up the work. This spreadsheet might help.

Notes:

-multiple derived tables may be extracting the same data fields in the data arrays, so need to make sure we do not have duplicates

-see create json mappings section of Metadb documentation

-What would be the new name of the table after mapping? Is the suffix taken from the value of TO and "pasted" behind the underscores? (They are going to the “__t” tables)

-See the Derived Table JSON Mappings Workgroup folder on the Reporting SIG google drive folder to contribute to this project