2019-07-10 Kubernetes Subgroup Meeting notes

John Malconian

Meeting notes

John Malconianpresented a diagram for ingress into an EKS cluster.

Cluster uses only one EC2 ALB, controlled by ALB Ingress Controller Pod. ALB hands off traffic to NGINX ingress controller, acting as reverse proxy (running as node port). ALB ingress controller runs in cluster, communicates with Kubernetes control plane to create ALB and update it.

Nginx listens as a node port on every node in cluster. Nginx maps the host header to a backend service. It will run against any namespace, so no need for multiple load balancers.

Service (e.g. Okapi) runs as standard ClusterIP service.

Edge modules would also be exposed to ALB

external-dns service configures Route 53 DNS to point host name to ALB address

Wildcard cert is managed by AWS cert manager

Traffic within the cluster is not encrypted at this point. For this to work, Okapi and RMB (and non-RMB modules as well) would need to support TLS as servers, or the containers they run in would need to run a proxy server like Nginx as a sidecar.

Ian Hardy

Meeting notes

Ian Hardycreated an Ansible role to create an Okapi service on a cluster. The meat of the role are a set of YAML templates that can be sent to the Kubernetes API.

Question: does it make sense to deploy Okapi as a stateful set? If database is external (rather than just shared memory), no real need. Hazelcast discovery seems to work correctly as a scaleable set.

Open source Hazelcast doesn't allow for a rolling upgrades, only way to manage upgrade would be blue/green (flash cut) or scale cluster down to 0 and then back up.

Ian Hardy demo'ed a playbook that uses the role, then also demo'ed using the playbook to scale the cluster by updating an Ansible variable.