Changes list


Jul 26, 2021	Raman Auramau	Moving FOLIO secrets management requirements to a separate document
Jun 9, 2021	Raman Auramau	Proposed solution is finally documented for review
May 18, 2021	Raman Auramau	Added some details about current FOLIO deployment options
May 7, 2021	Raman Auramau	Re-structure the document, add option 2
May 4, 2021	Raman Auramau	Added some requirements, current investigation details, drafted an option with a dedicated module
Apr 29, 2021	Raman Auramau	Initial document

Problem statement

Introduction

A secret is anything to which you want to tightly control access, such as API keys, passwords, certificates, and other sensitive information. Encrypted storage is needed to store secrets in a safe way, as well as a service to provide secrets management capabilities.

At the moment the FOLIO platform has neither dedicated secret storage as a common module or service nor even a standard approach for secrets management.

Note: edge-common has an interface for that, there are 3 implementations as for now - Vault, AWS, and an ephemeral one for dev needs (properties file), with read access implemented only. So, this piece of edge-common can be leveraged as some basic experience though it's definitely not enough.

With that, this document is aimed to provide analysis details and answer the following questions:

What technologies would be appropriate for this?
What might a solution look like? Is there some abstraction layer/interface allowing for different implementations?
What's required for a module to store/retrieve a secret?
What's required on the infrastructure side?

References

https://folio-org.atlassian.net/browse/FOLIO-2278

https://folio-org.atlassian.net/browse/MODORGSTOR-33

https://folio-org.atlassian.net/browse/MODINREACH-19

https://folio-org.atlassian.net/browse/MODPUBSUB-78

List of active topics (see Secret Storage item)

Requirements

Please refer to FOLIO secrets management - Requirements for a list of identified requirements.

Analysis

Current state

Three environment types should be considered:

Standard EC2 machine with all dockers on it (native docker, so Env Var are only available) - IndexData (snapshots, testing)
Kubernetes (rancher envs)
- K8S Secrets with Env Var is used there to ensure compatibility with the previous one
- Rancher has UI for displaying & configuration
AWS Cloud - SSM is used; application knows secret alias, IAM role is set on EC2 level

Sample

https://github.com/folio-org/edge-oai-pmh/tree/master/src/main/resources

https://github.com/folio-org/mod-search/blob/master/descriptors/ModuleDescriptor-template.json

Spring Cloud Config Server

https://cloud.spring.io/spring-cloud-config/reference/html/ Spring Cloud Config Server supports Git/SVN, Vault, JDBC database, Redis, AWS S3 as a backend for configuration properties but not AWS SSM. With that, AWS SSM is an Amazon native secret storage, and its usage for cloud-based deployment looks pretty justified. There is still particular way for thttps://rieckpil.de/resolving-spring-boot-properties-using-the-aws-parameter-store-ssm/.

Kubernetes Secrets

https://kubernetes.io/docs/concepts/configuration/secret/ - Kubernetes Secrets is a resource which is mounted into containers with a new volume type. It enables storing and managing secrets in safer and more flexible way than just putting them verbatim in a Pod definition or in a container image. Secrets can be

mounted as data volumes or
- mounted Secrets are updated automatically
exposed as environment variables or
- if a container already consumes a Secret in an environment variable, a Secret update will not be seen by the container unless it is restarted. There are third party solutions for triggering restarts when secrets change
using imagePullSecrets (i.e. by the kubelet when pulling images for the Pod)

This mechanism looks to be really convenient and efficient to deliver secrets into containers under Kubernetes. At the same time, this is Kubernetes-specific feature, and it does not fit any containers rather than under Kubernetes managements. As far as I could find Kubernetes should not be considered as the only container orchestration tool.

Alternatives for evaluation

Option 1 - Introduce mod-secret-storage and dedicated platform API

The idea of this proposal is to introduce a new dedicated back-end module which will be able to work with a number of specific secret storages providers (from one side) and provide generic platform API for consumers (from the another one).

FOLIO secrets management