[FOLIO-406] Design module life cycle Created: 07/Dec/16 Updated: 11/Oct/19 Resolved: 19/Apr/17 |
|
| Status: | Closed |
| Project: | FOLIO |
| Components: | None |
| Affects versions: | None |
| Fix versions: | None |
| Type: | Umbrella | Priority: | P3 |
| Reporter: | Heikki Levanto | Assignee: | Heikki Levanto |
| Resolution: | Done | Votes: | 0 |
| Labels: | sprint6, sprint7 | ||
| Remaining Estimate: | Not Specified | ||
| Time Spent: | 3 days | ||
| Original estimate: | Not Specified | ||
| Issue links: |
|
||||||||||||||||||||
| Sprint: | |||||||||||||||||||||
| Description |
|
Modules need to be installed on the system, enabled for tenants, upgraded, disabled, and deprecated and removed. At many of these points default configurations, permission sets, and other things may need to be created, modified, or deleted. How will all this be organized? |
| Comments |
| Comment by Heikki Levanto [ 07/Dec/16 ] |
|
My most pressing worry is related to the permissions and permission sets that must come from ModuleDescriptors and find their way into mod-permissions. But there is also the question about creating the necessary database tables, and probably a lot of other things that needs to be set up. We need to have a systematic way to handle all this. And this all is related to the app store concept too... |
| Comment by Heikki Levanto [ 14/Dec/16 ] |
|
I think we can at least identify some stages a module must go through. Later we can try to define all the things that need to happen at each of the stages. Here is a preliminary list:
|
| Comment by Heikki Levanto [ 16/Dec/16 ] |
|
Some loose points about installing a module on a cluster:
|
| Comment by Heikki Levanto [ 16/Dec/16 ] |
|
Some points about enabling a module for a tenant, for the first time:
|
| Comment by Heikki Levanto [ 16/Dec/16 ] |
|
Upgrading a module may need to do many of the same things as a first-time enabling. But with the caveat that we may already have important data in the system that must be preserved. I hope we can get away without supporting downgrading, that could quickly get messy. |
| Comment by Heikki Levanto [ 16/Dec/16 ] |
|
Since the module itself is the only one who can know what configuration, tables, settings, and data it needs to install, I think it must be responsible of doing it. So I propose that we add a well-defined service endpoint that does all that. The same endpoint may also take on the responsibility of upgrading the data, settings, etc to a newer version. It should get two parameters: The currently enabled version (if any), and the version about to be enabled. I propose we use something analoguous to our .../health, maybe .../upgrade that every module should implement if they wish to do any initialization or upgrading at all. I have proposed above that we tell the module what version was previously enabled. This is not strictly necessary, each module could store that data in its own configuration, and take it from there. But that means that every module should do the same kind of thing, I think it might be better to do it once, in Okapi. Okapi will have to remember what modules are enabled for what tenants, so it can understand the versions too, and pass them along to the .../upgrade endpoint. |
| Comment by Heikki Levanto [ 04/Jan/17 ] |
|
Some loose thoughts after today's discussions:
|
| Comment by Heikki Levanto [ 04/Jan/17 ] |
|
Some loose thoughts about dependecies and permissions:
All in all, this looks a bit messy, but perfectly doable. It would be cleaner if we had implemented permission system for Okapi's own operations too, like I proposed in
|
| Comment by Heikki Levanto [ 04/Jan/17 ] |
|
How to get there from here? This design requires some changes to the auth module, and the admin console, but surprisingly little for Okapi itself:
The disabling call behaves about the same way, and has the same permission complications, as does the upgrade thing. That needs more thinking. Even without the changes in auth and admin console, the init calls can be used for developing, testing, and demoing. If we have an Okapi setup with no auth module, Okapi can still make the init call, and it can intialize some internal state, like setting up a database. The init call can even make calls to other modules, but without an auth module, no permissions will be checked. If we have an Okapi setup with the auth module in place, we can still fake things a bit. If the module has a module-level permission to run its own init, things ought to work. It can even have module-level permissions for any operations the init call needs to make. That way, we might be able to use the current admin console, without any log in or token manipulation, since the auth module will make a "not-yet-logged-in" token when it sees a request without auth tokens. This token grants no user-level permissions, but allows use of moduile-level permissions. |
| Comment by Heikki Levanto [ 05/Jan/17 ] |
|
Some more thoughts:
|
| Comment by Heikki Levanto [ 05/Jan/17 ] |
|
I created
Some points that came up in slack discussions:
|
| Comment by Heikki Levanto [ 05/Jan/17 ] |
|
The permission stuff may not work as easy as I had hoped. The reason is that the admin console makes this enabling request directly to Okapi, and Okapi makes the init request directly to the module, so the auth module is never involved in the process. That means that the service in the module can not depend on permissions, and can not have module-level permissions it may need for further calls. The proper way would be to implement
Time being a workaround can be to give the admin user all necessary permissions, somehow. |
| Comment by Heikki Levanto [ 06/Jan/17 ] |
|
I came to think that there is another workaround, at least for the first few packages that may need module-specific permissions. The module can make a regular Okapi call to itself, and when it serves that request, it will have the permissions it needs. This is a messy thing to ask the module developers to do, but we can possibly get away with it in the first module or two. |
| Comment by Heikki Levanto [ 06/Jan/17 ] |
|
I am sure we can work around the permission issues. The next big question is upgrading modules. The first step is to install the upgraded version on the cluster we are running on, and to deploy it. No problem with that, Okapi can happily support two versions of the same module running at the same time. The problems come when we wish to switch a tenant from the old version to the new one. That poses two problems: 1) The new version may have different configuration settings, different database layouts, etc. Only the module can know what data needs to be upgraded, but someone must tell it when to do it. 2) I think we need a special upgrade service in Okapi, and in the admin console. We can not simply disable the old version and enable the new one. If we try to enable first, we run into conflicts. If we disable first, we need to disable everything that depends on this module, and re-enable. That is bound to go wrong. |
| Comment by Jakub Skoczen [ 06/Jan/17 ] |
|
1) the versioning will tell us if there are breaking changes, and if so we may need to perform migration, question is whether our system should provide ways to migrate 2) potentially, I think whatever we do we must always protect the data and be able to revert to the previous version, so for major version upgrades where compatibility is broken it would seem that cloning the data is the way forward |
| Comment by Heikki Levanto [ 12/Jan/17 ] |
|
Some more thoughts about upgrading:
|
| Comment by Heikki Levanto [ 23/Jan/17 ] |
|
Ok, we have some for of tenant interface that allows the module to initialize its tables. I have been speculating about upgrading above. Now I start to get worried about what else a module needs to do when it gets enabled for a tenant and/or installed for the first time. Some examples:
Most of these can be done by regular requets through Okapi. The problem is that when the tenant interface is invoked, we come in through the admin interface, without any auth token or user login. This makes it very hard for the modules to make further requests. I have a solution for that, but it requires using the auth module for admin stuff too, internal modules (Okapi-194), some extra facility for the admin to make requests on behalf of a real tenant, etc. All possible, but will take some work. |
| Comment by Heikki Levanto [ 19/Apr/17 ] |
|
I think we have moved on from this. We have a tenant interface, and a mechanism to pass permissions, etc. let's open new ones when we need to. |