[FOLIO-869] Aggregator module Created: 29/Sep/17 Updated: 18/Jan/19 |
|
| Status: | Open |
| Project: | FOLIO |
| Components: | None |
| Affects versions: | None |
| Fix versions: | None |
| Type: | New Feature | Priority: | P3 |
| Reporter: | Heikki Levanto | Assignee: | Unassigned |
| Resolution: | Unresolved | Votes: | 0 |
| Labels: | None | ||
| Remaining Estimate: | Not Specified | ||
| Time Spent: | 4 hours, 30 minutes | ||
| Original estimate: | Not Specified | ||
| Sprint: | |
| Development Team: | Core: Platform |
| Description |
|
We discussed a simple aggregator module in Cph. Summarize design here. |
| Comments |
| Comment by Heikki Levanto [ 29/Sep/17 ] |
|
The idea is to build something not unlike graphQL, but simplified to the bare minimum that we can get away with. Main points:
In some cases it makes sense to extend the main record with the expanded data, for example when getting a loan record that has a userid, put a whole user record next to it. In other cases less so, for example when getting a list of notes about an item, each of them has a userid who created the note, but these are often the same. Then it is better to return a list of notes with just the userids, and a separate map from userids to user records. We need to be able to expand in both directions: Starting from a loan record, find the userid, and get the matching user. Or, starting from a user record, take the userid, and find all loans that the user has (and probably expand the items in those too) |
| Comment by Heikki Levanto [ 29/Sep/17 ] |
|
Over the lunch we agreed that the way is to post the aggregation rules to this module, which will then create endpoints that provide the aggregated data, for example notes-with-user-names. The aggregation rules can be part of a moduleDescriptor, either in the back-end module (mod-notes can provide the notes-with-user-names), or in the UI module, depending what the UI needs. |
| Comment by Heikki Levanto [ 29/Sep/17 ] |
|
Simple example: endpoint notes-with-users gets notes, and expands the userid into user records. The /notes endpoint returns an object that contains a notes array of note records. endpoint: /notes-with-users
get: /notes
aggregate {
each: notes as {note}
get: /users/{note.metadata.creatorUid}
into {note}.userdata
}
|
| Comment by Heikki Levanto [ 29/Sep/17 ] |
|
A more complex example: endpoint: /loans-with-detail?{query},{limit},{maxlocs}
get: /loans?query=$query&limit=$limit
aggregate {
each: loans[*] as {loan}
get: /items/{loan.item}
into: {loan}.itemdetails
aggregate {
each: {loan}.itemdetails.locations[*] as {loc}
get: /locations/{loc.locationcode}?limit={maxlocs}
into {loc}.locationdetails
}
}
aggregate {
join: loans[*].userid with " OR " as {userlist}
get: /users?query=userid any {userlist}
into: users[]
aggregate {
each: users as {usr}
get: /perms/?query=user={usr.userid}
into: {usr}.perms
}
}
This iterates over all loans, and expands the items in them by looking up locations. Then it takes all the userids, joins them into one list, does a lookup on that, and then iterates over each user looking up permissions too. The join functionality may not be needed in version 1 of the aggregator, but it is here as an example of the directions we may expand the definition language. Of course the language will be proper Json, we don't want to parse our own string representation of it. |
| Comment by Heikki Levanto [ 29/Sep/17 ] |
|
The first implementation will be a stand-alone module with only in-memory data, to get a proof of concept up and running quickly. Then one that persists its data, and Okapi inegration, so we can define aggregation endpoints in ModuleDescriptors, using the redirect facility, with a new header that tells what the original path was. |