The design of mod-rtac assumes an instance with a relatively small number of items (10s). This causes a performance lag for instances with large collections of items. When a mod-rtac request is made, it makes an expensive aggregation of data from mod-inventory, mod-orders and mod-circulation each time an RTAC response is needed.

RTAC would benefit from a cache. Rather than doing an expensive fetch, it can do a cheap fetch of pre-computed data from the cache.

When we combine this cache with paging and search enhancements, mod-rtac-cache will support new requirements like instances with thousands of items and multi-item batch requesting – two features needed for libraries with extensive archives and special collections.

This document proposes a new module, mod-rtac-cache, which will be a drop-in replacement for mod-rtac. Mod-rtac-cache will implement existing mod-rtac interfaces for backwards compatibility.

Components

The cache, a searchable persistent data store, sits in front of FOLIO modules inside of mod-rtac-cache. The cache is populated on first instance request and by an updater when items in the cache change. Optionally, depending on the outcome of performance testing, a warmer may add data to the cache by pre-fetching data from the FOLIO modules in batches. An RTAC discovery system user interacts with the cache through a batch API and a search API. A human admin user starts warmer jobs.

Batch API 

The RTAC batch request of mod-rtac-cache implements a new POST API which takes a list of FOLIO Instance object ids in the body. This API will allow offset and limit (paging) of batch request results to facilitate user browsing of RTAC results. Mod-rtac-cache will implement the existing GET API of mod-rtac for backwards compatibility. Paging, in addition to user browsing, has the added benefit of reducing system load.

Search API 

The search API is a new RTAC API that allows the user to search for items within instances having a large number of items. Searching within an instance’s items is needed to support multi-item requesting since the user must be able to easily narrow their choices among thousands of items.

Lazy Loader

Whenever the batch or search APIs are called, mod-rtac-cache will first check the cache. If the item is not found in the cache, or the item has been marked as invalid by the updater, mod-rtac-cache will fetch the the data it needs from FOLIO, return the results, and populate the cache. In this way, mod-rtac-cache will support lazy-loading.

Depending on performance testing, the lazy loader may also need to be supplemented in some cases by the warmer.

Warmer 

The warmer is an optional component that may be needed for certain instances with a large number of items, depending on performance testing. The warmer will use the same FOLIO client methods as the lazy loader for fetching data.

The warmer is a job, started by the human administrative user, that takes 10s of minutes or several hours depending on the size of the tenant’s collection. It can be run once or many times. The warmer will have an optional item threshold, which, when exceeded will cause an instance and its items to be pre-loaded into the cache.

Updater 

The updater’s role is to keep the cache fresh with real-time updates to what the cache stores. The updater is a set of message queue listeners. These listeners consume already existing message queue topics, which FOLIO modules notify when inventory, circulation and orders data changes.

To reduce load, the updater will only mark cache items as invalid, relying on the lazy loader to take care of loading on first request. Optionally the updater may pre-warm some instances that have more than a certain threshold of items.

The updater will operate on the level of item updates, using the item id as the primary identifier.

FOLIO Client

The lazy loader, updater, and optionally the warmer, will use a client for fetching data from FOLIO that is performant enough to support lazy loading in most cases. Mod-fqm is a strong candidate for providing this FOLIO client due to its ability to aggregate queries across FOLIO modules in a performant manner. The client will require a paging API to not overload the system when fetching large collections.

Zero Downtime for Cache Rebuild 

The cache schema may change as part of feature enhancements. When this occurs, there should be zero downtime for the cache data, since un-cached instances with a large number of items will be too expensive to load on the fly. The cache updater will pause while the warmer works to accomplish this requirement.

Depending on how performance testing goes for the lazy-loader, there may be no need for the cache to be rebuilt, which is the primary benefit of lazy loading. There does need to be a method for invalidating all cache items regardless of performance testing however, to handle schema changes for cache items.

Data Model, Classes and Computed Fields

The cache will persist objects of the existing RtacHoldings schema. Mod-rtac-cache will continue to use other classes like RtacHoldingsBatch and RtacRequest, wherever possible leveraging existing code.

Mod-rtac-cache may re-use existing logic for computing fields such as RtacHoldings.volume which is a concatenation of various inventory Item volume-related fields. It may also take advantage of mod-fqm’s ability to return computed fields.

Stack 

FOLIO already makes extensive use of Kafka, OpenSearch. and Postgres. Mod-rtac-cache will make use of these technologies, as needed, requiring no additional infrastructure. Since FOLIO already has Kafka topics, which modules like inventory and circulation notify about changes, mod-rtac-cache will require no code changes in those modules. 

Mod-rtac-cache may get type-ahead searching and paging from OpenSearch or from Postgres depending on PoC testing. We will implement mod-opac-cache in Spring Boot. Mod-rtac-cache will implement the same interfaces as mod-rtac, in addition to new ones.