Addition to Spike: MODKBEKBJ-459 Ability to filter titles by package(s) + titles/selection status + tags/access status types
https://folio-org.atlassian.net/browse/MODKBEKBJ-459
Problem
There is no way for the user to filter to view the desired titles inside the package that s/he wants.
Goals
What do we currently have
There is no RM API endpoint exists so satisfy expected output as one request. In this case, we can analyze what we can be used in the existing system and what changes are required.
I picked out two actions
get a list of unique packages by title name
return data (selection status, title name, package name, publisher name, and type) to UI
To search titles regardless of the package, mod-kb-ebsco-java uses existing GET /{custid}/titles endpoint, which can return package information. This means in return, we have a set of unique titles (by title id) but not packages. So, this endpoint can hardly be used to serve the required behavior without changes.
To get a list of unique packages from endpoint mentioned above we have to perform additional actions:
get the full list of the titles - RM API return batches 100 entries max, so, in case of 1000 titles we have to perform 10 requests to get all of them
find unique package names
and return them on UI.
What can be done
The current implementation does not support filtering titles by packages at one time. Several options can be considered as solutions to provide the desired output.
On RM API side
1. Retrieve information from RM API using separate endpoint
This option assumes creation of a new endpoint on the RM API side with the option to filter titles by package name.
Requirement | Can be done? | Explanation |
|---|---|---|
Filter title results by package(s) | ✅ | should be supported by the RM API endpoint |
This filter must work in tandem with searching titles | ✅ | should be supported by the RM API endpoint |
This filter must work in tandem with selection status | ✅ | should be supported by the RM API endpoint |
NICE TO HAVE - filter must work in tandem with access status types filter | ❌ | not supported by an existing mod-kb-ebsco-java implementation |
NICE TO HAVE - filter must work in tandem with tags filter | ❌ | |
Sort the results | ✅ | should be supported by the RM API endpoint |
Paging results | ✅ | should be supported by the RM API endpoint |
2. Additional query parameter to the existing RM API endpoint.
RM API already has an endpoint(GET /{custid}/titles), which can retrieve titles regardless of the package with the list of following filters
Endpoint | /{custid}/titles |
|
|---|---|---|
Available filters
| search | <search_word> |
searchfield | titlename, publisher, isxn, subject and zdbid. | |
selection | all, selected, notselected, orderedthroughebsco | |
resourcetype | all, journal, newsletter, report, proceedings, website, newspaper, unspecified, book, bookseries, database, thesisdissertation, streamingaudio, streamingvideo, and audiobook | |
searchtype | any, contains, exactmatch, beginswith, proximity, exactphrase and advanced |
that could be updated by an additional filter to search by package name. Example:
Proposed filter | packagename |
|---|
The filter name 'packagename' should not be treated as the desired name. It is an example name and should be changed to a reasonable one.
Requirement | Can be done? | Explanation |
|---|---|---|
Filter title results by package(s) | ✅ | should be supported by the RM API endpoint |
This filter must work in tandem with searching titles | ✅ | should be supported by the RM API endpoint |
This filter must work in tandem with selection status | ✅ | supported by the RM API endpoint |
NICE TO HAVE - filter must work in tandem with access status types filter | ❌ | not supported by an existing mod-kb-ebsco-java implementation |
NICE TO HAVE - filter must work in tandem with tags filter | ❌ | |
Sort the results | ✅ | supported by the RM API endpoint |
Paging results | ✅ | supported by the RM API endpoint |
Cons
There is no possibility to have this functionality done in the nearest future because of the workload of the RM API team.
If we stick to this option we have two options to consider:
wait until RM API will have the capacity to include this work into the scope
propose to implement needed functionality by the Spitfire team.
On kb-ebsco-java side
The other possibility is to perform filtering on the kb-ebsco-java side.
1. Use stored holdings information
As another option, we can think about using holdings that are load every 5 days for each KB credentials. The information available to get from RM API much more extensive compared to stored info in the holdings table. Let's compare the information stored in the module and provided by RM API.
According to the screenshot at the top of the page, the desired information includes the following fields:
Required info | Stored in holdings | Can be added to holdings info |
|---|---|---|
Selection status | ❌ | ❌ |
Title name | ✅ |
|
Package name | ❌ | ✅ |
Publisher name | ✅ |
|
Publication/Resource type | ✅ |
|
Detailed information about holding fields can be found here - GET /{custid}/reports/holdings/deltas/{deltaReportId}
Requirement | Can be done? | Explanation |
|---|---|---|
Filter title results by package(s) | ✅ |
possible because needed data stored in holdings table |
This filter must work in tandem with searching titles | ✅ | |
This filter must work in tandem with selection status | ❌ | holdings do not have information about the selection status |
NICE TO HAVE - filter must work in tandem with access status types filter | ❌ | not supported by an existing implementation |
NICE TO HAVE - filter must work in tandem with tags filter | ❌ | not supported by an existing implementation |
Sort the results | ✅ | as we operate with holdings table, maintained in the module the sorting and paging will be done on the DB side by SQL query. |
Paging results | ✅ |
Cons
view outdated data set - holdings table updates every 5 days for each KB credentials. So, there is a possible situation that the user may see the information about titles that is no longer exists.
RM API returns only selected holdings.
selection status filter can not be applied because the holdings table does not have this information
in case of data absence, we should have "second plan"
2. Load titles data and process it in mod-kb-ebsco-java module
This is the most undesirable option. As it was described in "What do we currently have" section module should perform the work for getting unique package names by loading full titles list. That leads us to the option of storing loaded title list wheater in the cache or in the database.
It means that the package filter will be considered as a secondary filter after all titles loaded we are able to segregate the package list.
Requirement | Can be done? | Explanation |
|---|---|---|
Filter title results by package(s) | ✅ | search by the package as a secondary filter |
This filter must work in tandem with searching titles | ✅ | search as a first filter |
This filter must work in tandem with selection status | ✅ | supported by the RM API endpoint |
NICE TO HAVE - filter must work in tandem with access status types filter | ❌ | not supported by an existing implementation |
NICE TO HAVE - filter must work in tandem with tags filter | ❌ | not supported by an existing implementation |
Sort the results | ❌ | hard to maintain because of the complicated solution |
Paging results | ❌ |
Cons
two separate calls to the same RM API endpoint to get
page with titles - the result list from RM API is pageable
list of packages - separate call to load all packages that contain searched title
time cost for loading titles and selecting unique package names
storing packages and titles in mod-kb-ebsco-java module(in cache or db table) - leads to performance issues
Example: below we have a title search example with 10_000 results. To have a unique package list for this request, we have to load all records and store them somewhere, for instance in cache. Then the user decides to change search request and in return also have ~10_000 results and we also have to store it.