2020-07-06 Reporting SIG Meeting notes

Sharon Beltaine

Cornell University

Sara Colglazier

Mount Holyoke College/Five Colleges

Elizabeth Berney

Duke University

Erin Nettifee

Duke University

Joyce Chapman

Duke University

Karen Newbery

Duke University

Jean Pajerek

Cornell University

Tod Olson

University of Chicago

Claudius Herkt-Januschek

SUB Hamburg

Scott Perry

University of Chicago

Doreen Herold

Lehigh University

Stefan Stadtherr

MPIL Heidelberg

Sarah Park

Duke University

Simona Tabacaru

Texas A&M

Clint Bellanger

Auburn University

Kevin Walker

The University of Alabama

Ingolf Kuss

hbz

Christie Thomas

University of Chicago

Joshua Lambert

Missouri State

Andi Bihler

Munich Technical University Library

Joanne Leary

Cornell University

Cheryl Malmborg

University of Chicago

Michael Patrick

The University of Alabama

Vandana Shah

Cornell University

Nassib Nassar

Index Data

Angela Zoss

Duke University

Veit Köppen

University Magdeburg

Lisa DeCarolis

Smith College/Five Colleges

Linda Miller

Cornell University

Elena O'Malley

Emerson

Matt Harrington

Duke University

Holly Mistlebauer

Cornell University

Cathy Tuohy

Emmanuel College (FLO)

Nancy Bolduc

Cornell University

Shirley Moentnish

Missouri State

Eric Pennington

Texas A&M

Stefan Dombek

Leipzig University

Axel Doerrer

University Mainz

Natalya Pikulik

Cornell University

Owen Stephens

K-Int

Eliana Lima

Fenway Library Organization

Marcia Borensztajn

EBSCO

Harry Kaplanian

EBSCO

Attendance & Notes

Sharon

Attendance & Notes

• Today's attendance-taker: Linda Miller

• Today's note-takers:  Team Leads for project updates

Updates from Various Reporting Related Groups and Efforts

Community & Coordination, Reporting Subgroup Leads

Project updates

The Reporting SIG is using small working groups to address priorities and complete our work.  Each week, we will provide updates to the Reporting SIG from these various reporting-related groups and efforts. Please include updates on specific JIRA issues for prototype or query development workflow. Do we need to create any additional JIRA tickets?

Community & Coordination: 

• working on reporting survey

FOLIO Reporting development:

1. Reporting reference environment: folio_release is now deprecated because the FOLIO source does not appear to be maintained at present.  Please switch to using folio_snapshot instead when convenient (not urgent).

2. Reviewing approaches to query factoring

   1. Pre-computing joins
      SELECT ... FROM (r1 JOIN r2 JOIN r3 JOIN r4 JOIN r5);
-> CREATE TABLE r123 AS SELECT ... FROM (r1 JOIN r2 JOIN r3);
   SELECT ... FROM (r123 JOIN r4 JOIN r5);

   2. Pre-computing "with" clauses (aka CTEs or common table expressions)
      WITH r1 AS SELECT ...;  ->  CREATE TABLE r1 AS SELECT ...;

   3. Combining "with" clauses that use the same table

   4. Preparing intermediate/derived tables

      1. Adding the local schema to the search path (optional) (no longer recommended)
         SET search_path TO local, public;

      2. Creating indexes
         CREATE INDEX ON table_name (column_name);

      3. Vacuuming and analyzing
         VACUUM table_name;
ANALYZE table_name;

   5. Beginning to think about defining commonly useful derived tables (after Forum)

      1. Commonly used JSON fields not currently available in relational attributes

      2. Commonly used denormalizing joins

      3. Other repetition that can be factored

   6. Automatic regeneration of tables

Reporting Data Privacy Working Group:

• active borrowers count increment

• @Ingolf Kuss met with Sven, the Data Privacy Officer for ZBW. ZBW is a library in the GBV consortium. ZBW is an early implementer. According to Ingolf, if Sven approves of the measures implemented to protect data privacy in the LDP, then they are likely to be acceptable to all ZBW and GBV libraries (assuming that is all German libraries that are affiliated with Folio at this point).  Detailed meeting notes are at 2020-06-30 Reporting Data Privacy Working Group Meeting Notes. A few key issues are:

  1. Create a list that contains personal data fields, regardless of storage and module. Maintenance of the list by developers of the respective modules in data base conception, i.e. think about data protection right from the start (privacy by design).

  2. For anonymized data fields in the LDP, the values should be recognizable as anonymized (use of '0' or 'null' may be confusing).

  3. Long-term solution for European libraries: As U.S. libraries use the LDP data for quality control and so need user data included (fees/fines, etc.), whereas anonymizing this information for European libraries makes annual reporting difficult, Sven proposes a database (QM) system for personal data where all the data must be deleted after a fixed time period; this system is completely separate from the LDP.

RA/UM Working Group

• no updates

• looking for query developers

MM Working Group

• preparing HathiTrust query for FOLIO Forum

ERM Working Group

• for latest updates, see ERM Prototype and Query Development Status

• Bjorn Muschall generating test data and Axel preparing data for reference LDP 

RM Working Group

• for latest updates, see RM Prototype and Query Development Status

• prepared slides for FOLIO Forum

External Statistics Working Group

• Recategorize External Statistics REP Issues

Reminder: July 15 Reporting FOLIO Forum

Sharon

Upcoming Reporting FOLIO Forum Presentation

• Save the date, mark your calendars!

• The upcoming Reporting FOLIO Forum is scheduled for Wednesday, July 15 from 11:00 am to 12:30 pm ET

• Title = FOLIO Reporting Update on Community-Developed Reports

• presentation will be approximately one hour long, leaving a half hour for questions

Topics for Future Meetings

All

Review and update Topics for Future Reporting SIG Meetings