[FOLIO-2185] SPIKE: how to maintain resource deployment descriptors in the module repository? Created: 22/Jul/19 Updated: 03/Jun/20 Resolved: 06/Sep/19 |
|
| Status: | Closed |
| Project: | FOLIO |
| Components: | None |
| Affects versions: | None |
| Fix versions: | None |
| Type: | Task | Priority: | P3 |
| Reporter: | Jakub Skoczen | Assignee: | Wayne Schneider |
| Resolution: | Done | Votes: | 0 |
| Labels: | platform-backlog | ||
| Remaining Estimate: | Not Specified | ||
| Time Spent: | Not Specified | ||
| Original estimate: | Not Specified | ||
| Issue links: |
|
||||||||||||||||||||||||||||||||||||||||||||||||||||||||
| Sprint: | CP: sprint 71 | ||||||||||||||||||||||||||||||||||||||||||||||||||||||||
| Story Points: | 3 | ||||||||||||||||||||||||||||||||||||||||||||||||||||||||
| Development Team: | Core: Platform | ||||||||||||||||||||||||||||||||||||||||||||||||||||||||
| Description |
|
this is a follow up to
We have decided that the way to address this problem is by keeping the settings in ModuleDescriptor and provide a stand-alone script that will generate K8s YAML descriptor from the MD. (
|
| Comments |
| Comment by Ian Hardy [ 30/Aug/19 ] |
|
Tasks: 1. Select test module (should require both environment vars and args ideally mod-login is a good candidate) |
| Comment by Jakub Skoczen [ 03/Sep/19 ] |
|
Wayne Schneider Ian Hardy Guys can we get an example of how all nessesary information would be contained in the launch descriptor before we start rolling this out? It can be a pseudo (imaginary) example? |
| Comment by Wayne Schneider [ 03/Sep/19 ] |
|
Here is an example for mod-login: "launchDescriptor": { "dockerImage": "${artifactId}:${version}", "dockerArgs": { "HostConfig": { "Memory": 268435456, "PortBindings": { "8081/tcp": [{ "HostPort": "%p" }] } } }, "dockerCMD": [ "verify.user=true" ], "dockerPull": false, "env": [ { "name": "JAVA_OPTIONS", "value": "-XX:+UnlockExperimentalVMOptions -XX:+UseCGroupMemoryLimitForHeap" }, { "name": "DB_HOST", "value": "10.0.2.15" }, { "name": "DB_PORT", "value": "5432" }, { "name": "DB_USERNAME", "value": "folio_admin" }, { "name": "DB_PASSWORD", "value": "folio_admin" }, { "name": "DB_DATABASE", "value": "okapi_modules" }, { "name": "DB_QUERYTIMEOUT", "value": "30000" }, { "name": "DB_CHARSET", "value": "UTF8" }, { "name": "DB_MAXPOOLSIZE", "value": "5" } ] } |
| Comment by Wayne Schneider [ 03/Sep/19 ] |
|
The example above works well for Okapi deployment on Vagrant. Some issues:
Ian Hardy, David Crossley – thoughts? Does this seem workable? |
| Comment by Ian Hardy [ 04/Sep/19 ] |
|
One proposal for db host is to set a service in k8s called "postgres" that points to the pg db. In the Vagrant boxes, could create a "postgres" alias that resolves to localhost. That way we could set the default host to the same thing in both cases. Might be a little more convenient, but this doesn't really solve the problem of having to pick some value that doesn't work "out of the box" since it depends on some specific infrastructure configuration. |
| Comment by Wayne Schneider [ 04/Sep/19 ] |
|
Here's what I found in my testing:
This is actually kind of a problem, as we still have potentially quite long-running queries, so setting a timeout might break things. What are opinions? Options are:
|
| Comment by Wayne Schneider [ 04/Sep/19 ] |
|
After further conversion, we've settled on this as a model: "launchDescriptor": { "dockerImage": "${artifactId}:${version}", "dockerPull": false, "dockerCMD": [ "verify.user=true" ], "dockerArgs": { "HostConfig": { "Memory": 268435456, "PortBindings": { "8081/tcp": [ { "HostPort": "%p" } ] } } }, "env": [ { "name": "JAVA_OPTIONS", "value": "-XX:+UnlockExperimentalVMOptions -XX:+UseCGroupMemoryLimitForHeap" }, { "name": "DB_HOST", "value": "postgres" }, { "name": "DB_PORT", "value": "5432" }, { "name": "DB_USERNAME", "value": "folio_admin" }, { "name": "DB_PASSWORD", "value": "folio_admin" }, { "name": "DB_DATABASE", "value": "okapi_modules" }, { "name": "DB_QUERYTIMEOUT", "value": "60000" }, { "name": "DB_CHARSET", "value": "UTF-8" }, { "name": "DB_MAXPOOLSIZE", "value": "5" } ] } I will do some further testing to see what changes would need to be made to prevent Vagrant box builds and the "single server install" document from breaking. |
| Comment by Wayne Schneider [ 04/Sep/19 ] |
|
Since the environment variables set on Okapi's /_/env endpoint override environment variables in the LaunchDescriptor, the only change required in folio-ansible is to remove the JAVA_OPTIONS memory settings in the group_vars files as module descriptors are updated. No changes are necessary in the single-server install runbook in folio-infrastructure. |
| Comment by Wayne Schneider [ 06/Sep/19 ] |
|
The Ansible roles affected by this change are okapi-register-modules and okapi-deployment, both of which modify the stock LaunchDescriptor for a module based on keys in the folio_modules variable. They operate by overriding the env key and/or the dockerCMD key. Changing this behavior would be tricky, and I'm not sure it's desirable – the behavior of overriding is more straightforward and less prone to error than somehow trying to combine the env and dockerCMD lists. Therefore, I think the solution is to update the group_vars files in folio-ansible now to include the -XX:+UnlockExperimentalVMOptions -XX:+UseCGroupMemoryLimitForHeap options as well as the memory settings, then giving ourselves the housekeeping task of cleaning up the group_vars files after
I've created
|
| Comment by steve.osguthorpe [ 13/Sep/19 ] |
|
Can I also suggest the changing of the base image for java 8 to inherit from https://hub.docker.com/r/fabric8/java-alpine-openjdk8-jdk. That image as a base allows default vm memory allocations to be calculated as a ratio of the container limit. Emulating the container memory capabilities of java10+ |
| Comment by Wayne Schneider [ 13/Sep/19 ] |
|
Thanks for the comment, steve.osguthorpe. We do have an open issue for that (
We also have on our backlog a move to OpenJDK 11 (
|