Spike: Changing /titles/{titleId}?include=resources to a paged endpoint

MODKBEKBJ-591 - Getting issue details... STATUS

In scope of this spike we need to answer two questions:

  • how does current implementation handle titles with a large number of resources (a few thousand)?
  • can we implement a paged endpoint to get resources of a title that will work as /packages/{packageId}/resources endpoint?

Current implementation for titles


Api '/titles/{titleId}?include=resources' of mod-kb-ebsco java uses following holdingsIQ api:

And there is no parameters for resources so they are returned in response every time:


According to the current implementation, resources always come from holdingsIQ and option "include=resources" is used only to include this resources in mod-kb-ebsco response and retrieve tags for related resources from database.

Current implementation for packages

'/packages/{packageId}/resources' endpoint of mod-kb ebsco java uses '/vendors/{providerId}/packages/{packageId}/titles'.

So titles here are treated as resources for packages and this holdingsIQ endpoint has 'count' and 'offset' parameters for pagination, so mod-kb-ebsco endpoint also provides pagination.

HoldingsIQ api:


HoldingsIQ doesn't provide endpoints to paginate titles' resources currently:

Possible solutions:

Solutionmod-kb-ebsco workAdventagesDisadventages
pagination on HoldingsIQ side (f.e. /{custId}/titles/{titleId}/resources api with pagination)Update api in mod-kb-ebsco accordinglyAbility to paginate, no additional requests or excessive logicNeeds work to be done from ebsco side
pagination only on mod-kb-ebsco side

Implement caching of HoldingsIQ' title response(with some ttl and invalidation on PUT/POST/DELETE requests)

Add pagination parameters to current '/titles/{titleId}?include=resources' api

Paginate over cached data(if cache exists, otherwise get data from HoldingsIQ, then cache response and return desired page)

No need for work from ebsco side

Additional database calls on every api call (check cache exists, save cache, query cache when paginate)

Additional database calls on other api calls (when updating title then cache must be deleted)

Also there should be a scheduler which will check cache ttl in background and expire cache for all tenants (additional application resources for scheduler and additional load on database for ttl check and expiration)

Note: this approach will not save us from situation when there are too much resources in a title and they cannot fit into http response (because pagination is absent on holdingsIQ side and we will not be able to get all resources from holdingsIQ which returns all resources for a title on each api call)

It was decided to implement pagination only on front-end side as this solution will not have side effects for performance and will add only ux. Story was created: UIEH-1258 - Getting issue details... STATUS .