Spike: Changing /titles/{titleId}?include=resources to a paged endpoint
MODKBEKBJ-591 - Getting issue details... STATUS
In scope of this spike we need to answer two questions:
- how does current implementation handle titles with a large number of resources (a few thousand)?
- can we implement a paged endpoint to get resources of a title that will work as /packages/{packageId}/resources endpoint?
Current implementation for titles
HoldingsIQ
Api '/titles/{titleId}?include=resources' of mod-kb-ebsco java uses following holdingsIQ api:
And there is no parameters for resources so they are returned in response every time:
Mod-kb-ebsco-java
According to the current implementation, resources always come from holdingsIQ and option "include=resources" is used only to include this resources in mod-kb-ebsco response and retrieve tags for related resources from database.
Current implementation for packages
'/packages/{packageId}/resources' endpoint of mod-kb ebsco java uses '/vendors/{providerId}/packages/{packageId}/titles'.
So titles here are treated as resources for packages and this holdingsIQ endpoint has 'count' and 'offset' parameters for pagination, so mod-kb-ebsco endpoint also provides pagination.
HoldingsIQ api:
Summary
HoldingsIQ doesn't provide endpoints to paginate titles' resources currently:
Possible solutions:
| Solution | mod-kb-ebsco work | Adventages | Disadventages |
|---|---|---|---|
| pagination on HoldingsIQ side (f.e. /{custId}/titles/{titleId}/resources api with pagination) | Update api in mod-kb-ebsco accordingly | Ability to paginate, no additional requests or excessive logic | Needs work to be done from ebsco side |
| pagination only on mod-kb-ebsco side | Implement caching of HoldingsIQ' title response(with some ttl and invalidation on PUT/POST/DELETE requests) Add pagination parameters to current '/titles/{titleId}?include=resources' api Paginate over cached data(if cache exists, otherwise get data from HoldingsIQ, then cache response and return desired page) | No need for work from ebsco side | Additional database calls on every api call (check cache exists, save cache, query cache when paginate) Additional database calls on other api calls (when updating title then cache must be deleted) Also there should be a scheduler which will check cache ttl in background and expire cache for all tenants (additional application resources for scheduler and additional load on database for ttl check and expiration) Note: this approach will not save us from situation when there are too much resources in a title and they cannot fit into http response (because pagination is absent on holdingsIQ side and we will not be able to get all resources from holdingsIQ which returns all resources for a title on each api call) |
It was decided to implement pagination only on front-end side as this solution will not have side effects for performance and will add only ux. Story was created: UIEH-1258 - Getting issue details... STATUS .