| Attendance & Notes | Angela | Attendance & Notes - Today's attendance-taker: Linda Miller
- Today's note-takers: Team Leads for project updates
|
| Update on LDP app |
| - still trying to get LDP app into Kiwi release
- TC initially reviewed and failed LDP app for Kiwi, but opened up discussion with developers on requirements
- Came up with a blend of short deadline for some changes and longer deadline for others
- Reporting SIG members attended PC last week to advocate
- TC vote was contentious, some ongoing questions about why there is a rush
- Still not completely sure how everything will work out
- ID developers have been working around the clock to meet the requirements; the biggest problem was multi-tenancy
- note: many of the existing apps have mistakes and have been approved
|
Announcements /
Reminders | Angela | Recruiting New Query Developers - The Reporting SIG is always on the look-out for new query developers. Please let us know if you are interested in doing query development or if there are others at your institution who might be a good fit.
Cluster Ranking New Report Clusters are added on a regular basis, so it is important to make sure your institution is reviewing these clusters and ranking them to establish report development priorities. If you rank reports for your institution, please follow the instructions below. If someone else ranks, please pass this information along to that person so your institution's vote can be included. - Action =>> Please review Reporting SIG-All Report Clusters (57 issues) in JIRA and RANK each report cluster for your institution (R1-R5)
- For reporting, institutions only need to rank the UXPROD Report Cluster JIRA issues. All reporting requirements, which are captured in REP-XXX issues, roll up to the UXPROD Report Clusters. Report clusters cover one or more report (REP-XXX issue) requirements.
Update LDP implementers grid
New FOLIO Query Writing Knowledge Base - See "Writing Queries" section in the sidebar of the wiki
|
| Introduction to LDLite (~35 minutes) | Nassib | - LDLite is open source software
- part of LDP project, but not part of LDP or metadb
- lightweight tool that queries FOLIO via API, but it handles connection and paging details
- automatically stored the data in tables which you can query using SQL/DBeaver
- transforms data from JSON in similar ways to metadb
- only thing you need is python
- needs the login credentials for OKAPI (same as for postman)
- everything else is bundled with software
- can use it on Windows, MacOS, and Linux
- useful for institutions without access to LDP or metadb
- if you do have metadb, LDLite is even more real-time data than metadb (which has a streaming queue)
- LDLite can put data into metadb
- if you have LDP but not metadb yet, this tool also gives you access to data that you will have in metadb that don't appear in LDP (e.g., permissions, COUNTER stats)
- limitations: currently too slow for very large data (e.g., full inventory), but that's a problem with the current FOLIO APIs; possible in the future that that might change
- might have to formulate a CQL query (the query language used by FOLIO modules) to reduce the amount of data you're retrieving, rather than pulling all of the data
- can use this as a data extraction tool and also just for querying the FOLIO APIs
- doesn't support incremental updates and historical data, which metadb does support
- LDLite creates tables and stores them in a database, after which you can use SQL queries
- by default, LDLite uses an embedded database, storing all the data in a file on your machine, so you wouldn't need permission from an administrator
- you can also set up a separate database to store the data LDLite extracts from FOLIO, like PostgreSQL or Redshift. There are advantages to this - you can share data, back it up, etc. - but you might need an admin to create the database for you
- could potentially also hook the LDP app to this shared database, basically setting up your own LDP/metadb instance
- can also point LDLite at your own LDP or metadb and have LDLite store data (e.g., in local schema)
- you would typically call LDLite from python, but if you don't know python, you don't have to learn very much, just a few calls to get the data into the database (and it's a very common and easy language to learn)
- Demo
- note: good idea to sort queries because of paging, if possible
|
| Updates and Query Demonstrations from Various Reporting Related Groups and Efforts | Community & Coordination, Reporting Subgroup Leads | Project updates Reporting development is using small subgroups to address priorities and complete work on report queries. Each week, these groups will share reports/queries with the Reporting SIG. Reporting development team leads are encouraged to enter a summary of their work group activities below. RA/UM Working Group - Getting back into query writing, will talk soon about whether to focus on metadb or keep going with new LDP queries
- Angela will begin attending RA SIG meetings every two weeks or so to open the lines of communication
- Context
MM Working Group - The zoom link for the meeting is https://umass-amherst.zoom.us/j/99254861410. Usual FOLIO password.
- The group has a new schedule - once a month starting Sept. 7th, Tuesday, from 12-1pm ET. Our next meeting will be 10/5 .Ongoing, this group has decided to and will be a lab session or a virtual place to learn from each other, seek help, and report back on any issues with derived tables or reports for metadata.
ERM Working Group - Improvement to some ERM queries / derived tables are currently in review
- Date fields are available again for in agreement cost & count query
- Adding erm resource data to package content item derived table
- Added erm resource types(+ publication & subtypes) for erm agreement count query
- Group is working on ERM-specific datamodel documentation
- Overview of Cross-module connections
- Add more documentation and their purpose for derived tables
- Had a Dashboard demo by Owen Stephens at last meeting → recording (demo from 6:45 on)
- Discussed the distinction between workflow reporting and adhoc reporting
- ERM Prototype and Query Development Status
- Bi-weekly schedule (next meeting 28th, september 9:00 AM ET)
- Zoomlink (password is the common zoom password in the FOLIO community)
RM Working Group - dividing fund_expenditures_by_po_line into 2 queries: one that uses invoice payments as date range and another that uses fiscal year
- reviewed HathiTrust serial holdings report, refined for use at Cornell
- exploring ways to make it easier to find options for reporting parameters in your LDP (e.g., a list of locations, item statuses, etc.)
- several queries for RM completed, but still need documentation, testing, and review
- working on queries for the next FOLIO Analytics release
- for latest updates, see RM Prototype and Query Development Status
Reporting SIG Documentation Subgroup - No changes
- Additional Context
External Statistics Working Group - no updates currently
- new organizational/tracking scheme for JIRA, with pointers to queries in folio-analytics repository
- New organizational structure for External Statistics reports
- external statistics reports (e.g., ACRL) typically require running queries from different functional reporting areas
- these reports will be captured in JIRA under one UXPROD-XXXX report cluster issue, then the descriptions will point to each of the queries required to run them on the folio-analytics repository
- institutions will need to rank each of these 8 new UXPROD-XXXX report cluster issues
- each reporting development team will take responsibility for the queries in their area for the external statistics clusters
Product Council
For all recent work on FOLIO Reporting SQL development:
|
| Special advance topic: non-Latin script querying (~15 minutes) | Stefan |
|
| Topics for Future Meetings | All | - Follow-up on MARC status, Quickmarc/Data Import conflicts
- How to strengthen connections to SIGs and their developers to be kept in the loop about changes to the data model
- Show and tell
- how are institutions using the LDP
- examples of using the local schema
- Cornell's report ticketing system
- Rollout plans from institutions
- Ask someone on the sysadmin side to talk about LDP administration (Jason Root?)
- Training topics
- adding test data in FOLIO snapshot
- How to do ad hoc querying with the derived tables
- How to use the LDP app
- using KNIME to build reports (LDP edition)
- use of local schema for custom tables
- more on MARC
- using different applications (other than DBeaver)
- Insomnia for API queries?
- SQL
- non-Roman script querying (Stefan)
- Discussion:
- table of FOLIO reporting options somewhere on the wiki
- SIG wiki? Tips and Tricks?
- What to include?
- Upcoming:
Review and update Topics for Future Reporting SIG Meetings
|
|
|
|