FOLIO secrets management - Requirements

This page is aimed to consolidate and document all known requirements (both functional and non-functional) applicable to FOLIO secrets management.

Use cases

  1. A user (administrator) wants to create/manage secrets through the secret storage directly and allow FOLIO modules to access those secrets (read-only access)

    1. No specific Jira examples though as per my understanding all those secrets being specified in modules' ModuleDescriptor-template.json files (in launchDescriptor.env section, e.g. DB_USERNAME, DB_PASSWORD etc.) fit this category
  2. A user (administrator) wants to create/manage secrets through FOLIO. E.g., a module might need to store credentials to some external system (like FTP or SMTP server credentials, API keys etc.) UI forms are involved as well for interaction with a user
    1. Example: INN Reach Central Server Configuration keeps API keys (key and secret) for Central and Local servers. As per requested behavior, UI Screen contains fields for setting / displaying API keys, and a user (administrator) uses them to save, view or update keys. One can use UIINREACH-12 - Getting issue details... STATUS for reference and find mock ups there. Currently, backend module storage is used to store such keys
  3. A module wants to create/manage secrets so that a user is not even in the loop here, and the interaction is only between the module and secret storage.
    1. Examples are modules which are creating system/tenant users (like mod-pubsub), or a case with generating and storing signing keys in such a way that all instances of the mod-authtoken container can easily stay aligned
    2. One more specific example is MODORGSTOR-33 - Getting issue details... STATUS

Secrets access type

(question) Is read-only access enough to cover use case #1 from above, or full access with CRUD support to cover use cases #2 and #3 is mandatory?

Secret types

The following secret types are to be supported:

Secret typeFormatWho creates?Who accesses?How does it work now?Additional comments
login & passwordString
  • FOLIO resource administrator
  • external system administrator
  • modules working with any resource with logo-pass
different modules use their own storages for that
API keyString
  • administrator
  • system
  • modules working with APIs
different modules use their own storages for that
tokenString
  • system
  • mod-authtoken
  • modules working with tokens


SSL certificatefile
  • administrator
  • modules using certificate for work with Kafka

Cannot be stored in SSM, Vault etc.

Encrypted S3?

Secret stores

The following secret stores are to be supported:

Secret store mechanismLinksEnvironment & Comments
HarshiCorp Vaulthttps://www.vaultproject.io/On-premises installations
AWS Systems Manager Parameter Store ( or just AWS SSM)https://docs.aws.amazon.com/systems-manager/latest/userguide/systems-manager-parameter-store.htmlAs an option for AWS cloud hosting
AWS Secrets Managerhttps://aws.amazon.com/secrets-manager/As an option for AWS cloud hosting
properties-file
For dev / test needs
environment variables
Standard EC2 machine with all dockers on it (native docker, so Env Var are only available) - IndexData (snapshots, testing)

Kubernetes Secrets


  • K8S Secrets with Env Var is used there to ensure compatibility with native docker deployment
  • Rancher has UI for displaying & configuration

User interface for Secrets management

(question) Home-grown unified web UI vs. store-specific UIs

Both AWS SSM and Vault have their own management user interfaces, Rancher provides an option to manage secrets in form of environment variables in UI forms.

Tech stack support

The following programming languages used on Folio platform are to be supported:

  • Java Spring way
  • Java Vert.x
  • (question) Node.js (RA: I haven't met such modules/components by my own but have heard they exist - should they be considered?)
  • are there any other languages used?

Audit capabilities

Access to secrets must be logged for further audit

Proposal - start now with option 1, may consider option 2 later if option 1 won't be enough.

Secret store access

Any particular FOLIO module should only access its own secrets

Notes:

Multi-tenancy support

A tenant should only access its own secrets

Notes:

  • tenant are FOLIO internal term, so that it seems now that access verification on tenant level cannot be supported on Secret store level
  • tenant ID can be used as a part of secret name

Secrets update in runtime

Adding / updating tenant's secrets in a multi-tenant environment shouldn't result in downtime for other tenants, if possible.

(question) Secrets updates can be challenging because of more complex mechanism of change notification, e.g. in case of password change or rotation - how to ensure that all instances are notified and use the recent value?

Caching secrets

(question) Can / should secrets be cached in apps?

  • Concern is about storing secrets in memory - there's potential risk of data extracting from RAM, though this can be considered as a low risk / low impact
  • Caching reduces load on Secret Store, and improves performance & availability