2019-06-24 RPWG Meeting notes

eUsage Use Case for using LDP

Status of eUsage App

-cost per use is the most important report

-in the eUsage app, usage data being collected about a resource from various providers or services 

-in agreements, the way the resource is being ordered, licensed, invoiced, etc. is set up

-no link between eUsage resource and agreements yet

Questions

-is the work better done in LDP or within a FOLIO app?

-mod agreements could do a query against LDP and provide data back to the agreements UI, evaluates the permissions, and performs the query itself

Frequency

-stats not needed in real time; most users gather data monthly

Is this a unique need from a FOLIO app? If so, might be strange to do anything special.

• Probably not unique, no.

Nassib

How to decide between doing it using LDP or doing it through the APIs for the modules?

• Probably makes most sense to use LDP for ad hoc queries; if it's just a few very specific elements you need, might be okay to just use API.

-need to think about queries

-might be easier if distribution of queries arranges over

-if small number of attributes, might be more efficient to develop in FOLIO application

Is the "overnight data" problem relevant? If you need real-time data, is that a problem here?

• Actually, could have the eUsage app trigger a data push before sending the query, so might not be a problem

• Would need to be able to have the data loader load incremental data

-could push data into LDP as needed (faster than overnight)

-FOLIO db needs to respond to request in real time; LDP is queried by analysts; use of LDP could create a latency in providing the data

Data Storage Concerns

Is data size a problem?

• Probably the same problem if it's LDP or API

• Could cache it?

• We have per month usage statistics, so could cache the previous months which should stay stable

• We could outline a representative query, the typical amounts of data (in a schematic form), then we could pretty easily analyze it. Might have to optimize it (could do that in the database or on the app side).

Proof of Concept

-outline representative query

-set up amount of data needed for good test

Permissions for queries?

• Permissions may be a bit of a problem; we don't have an OKAPI interface to the LDP right now. If you want FOLIO's permissions system, we would have to develop something like that. If we could simplify the permissions... the queries could be direct ODBC queries, but it sounds like we need some way to ask the LDP data loader to load some incremental data.

• We can proxy - apply OKAPI permissions on the app side

• Reporting dev resources are limited, don't have any OKAPI or STRIPES developers; it's not the complexity but the time to do it that would be difficult

• So yes, app can evaluate permissions and then send query

-no OKAPI interface to LDP; a permissions system would require developing this

-queries could be direct ODBC queries, but we also need to add the LDP loader to load incremental data

-eUsage can proxy to the LDP

Moving forward - prototype?

• simplify every aspect, just make something to look at the behavior of it

• more concerned about the quality of service issue - right now, FOLIO operational database has a concern/responsibility to respond to requests in a real-time way. LDP doesn't do that, doesn't have any way of enforcing that. Analysts are querying LDP directly, so use of LDP could create some latency. The user interface for eUsage needs to be designed to account for that - you would have to sit and wait for it. I think that's the main concern.

• Maybe the user can keep using the app while the query is being delivered by the LDP - send a request to the LDP, but don't freeze the app waiting for it to be fulfilled

• Yes, some way to absorb latency that results from, for example, a very heavy query

Timeline Constraints?

-eUsage wants to finish application by December 2019

-ask Annika for specifics on timing

Next Steps

-small workgroup discusses issues further and returns to Reporting SIG with recommendations

-small workgroup is Richard, Nassib, Owen, Annika, Kevin, and Matt from this meeting

-note that Richard and Annika have documented a good use case the small workgroup can start with

Preparation for next meeting

-document what is needed in a schematic way (e.g., typical query, typical dataset with appropriate size)

-describe data needed, query patterns