2019-08-12 RPWG Meeting notes

• Considerations:
  • Many reports have been ranked by institutions for "go live (MVP)".
  • LDP critical dependencies on FOLIO core development which have been requested and flagged as critical beginning in late 2018 will likely not be addressed until mid-2020 or beyond, based on discussions with the FOLIO capacity planning and project management groups.
  • FOLIO is requesting that "go live (MVP)" features be ready by January 2020, to be released in Summer 2020.
  • Future developer resources appear to continue to be roughly around 1 FTE on average, consisting of several part-time developers.
• LDP road map is based on these considerations, and aims to release features according to FOLIO's projected time lines for addressing critical dependencies as well as available developer resources.
• For "go live (MVP)":
  • LDP will support ad hoc, cross-domain queries for all, or a very large proportion, of FOLIO data.  We would ask this working group to help us determine, in the near future, the definition of "all FOLIO data" required for inclusion in the LDP.
  • The LDP developers and this working group will aim to implement all reports requested for "go live (MVP)" if possible.
  • Historical data will be retained in the LDP but not transformed into a single schema.
  • Report queries at "go live (MVP)":
    • Queries can be written using relational attributes when using the most common attributes (e.g. loans.loanDate), but will require JSON paths to access nested attributes (e.g. json_extract_path_text(loans.data, 'status', 'name') which refers to the "name" subfield located within the "status" field).
    • In other words, a set of "common" attributes will be available as relational attributes, while other attributes will have to be retrieved from JSON data (in the same table) using the json_extract_path_text() function.
    • The usage takes the form json_extract_path_text(A, F1, F2, ...) where A is the attribute containing the JSON data, and F1, F2, ... is a list of up to five nested JSON fields which specifies a path into the JSON data.
    • The JSON data include the complete set of data provided by FOLIO, while the relational attributes are created as a convenience to make writing queries easier.
• Historical queries within a single schema, ETL, and star schema are features planned for later releases of the LDP but are dependent on the identified critical dependencies and availability of developer resources.

Shall we build these reporting data models next?

1. Circ Item Detail at https://wiki.folio.org/display/RPT/Circulation+Item+Detail+Report+Prototype
2. Services Usage at https://wiki.folio.org/display/RPT/Services+Usage+Report+Prototype
3. Shelf List Location at https://wiki.folio.org/display/RPT/Shelf+List+Location+Report+Prototype

Questions:

• What do RPWG members think of these (above) as our current highest priorities?
• Would the prototype developers like to develop the SQL queries for each of these?

Outline of how to contribute a new SQL report query--or a modification of an existing query--using the shared community space in GitHub that has been set up for this purpose.  We will use GitHub's web interface which provides a relatively easy process.

The basic steps are:

1. Create or update your own copy of the ldp-analytics repository by following either step 1(a) or 1(b):
   
   1. If this is your first contribution, make a copy ("fork") of the repository at https://github.com/folio-org/ldp-analytics by pressing the "Fork" button.  The original repository is called folio-org/ldp-analytics, and your copy will be something like your-github-name/ldp-analytics.  As you continue through these steps, pay attention to whether you are using the original or your copy.
   2. If you had already previously made a copy, then you only need to synchronize your copy with the original repository by "merging" any new commits from the original into your copy.  To do this, use the "Compare" button and select the branches so that GitHub will merge updates into your copy from the original.  The merge direction is from right to left:
      (your-github-name/ldp-analytics) ← (folio-org/ldp-analytics)
      If you find yourself seeing this:
      (master) ← (master)
      then click on "compare across forks".  Once you have set these up correctly, then create a "pull request" by pressing the "New pull request" button and follow the on-screen instructions to merge the pull request.
2. If you need to create a new file or directory/folder, ask one of the maintainers of ldp-analytics.  It appears that GitHub's web interface only lets you edit existing files.
3. Make a set of edits ("commits") in your copy of the repository.  To do this, find a file you want to edit and then click the pen icon to use the editor.  After editing, don't forget to scroll down on the page, fill in the "Commit changes" form, and click the "Commit changes" button.
4. Create a "pull request" by pressing the "New pull request" button.  The pull request will be reviewed by the maintainers of ldp-analytics, and (if there are no problems) they will merge your commits into the original repository.  If there are problems, they may ask you to make specific further commits and then create a new pull request.
5. If you need help, ask one of the maintainers of ldp-analytics, or the nearest programmer.

Future Topics:

• Definition of "all FOLIO data" required for inclusion in the LDP.
• Evaluate the data model of the LDP's MARC implementation.
• Identify representative report queries on MARC records.

Next meeting date: