Changes list

...

References

/wiki/spaces/TLG/pages/753669 (see Secret Storage item)

Requirements

Below is the list of identified requirements ( To anyone reviewing this document: feel free to specify more requirements if you have any):

...

• certificate
• API key
• login, password
• ... what else

...

• HarshiCorp Vault (used for on-premises installations),
• AWS Systems Manager Parameter Store or AWS SSM (used for AWS cloud hosting),
• properties-file (for dev/test needs)

...

• Are there known benefits in comparison with manual secrets update by platform administrators? Should this be done programmatically?
• Are there specific requirements to runtime updating for secrets? I.e. should consumers be able to update secrets in runtime without restart/downtime?

...

• Is it required? Technically, both AWS SSM and Vault have their own management user interfaces, Rancher provides an option to manage secrets in form of environment variables in UI forms
  

...

•  Using the same credentials for every tenant is a kind of concern

...

• Access to secrets should/must be logged for further audit - any specific requirements?
• Can AWS SSM and Vault logs be efficient for this, or do we need to have any dedicated log storage for that?

...

Please refer to FOLIO secrets management - Requirements for a list of identified requirements.

Analysis

Current state

Three environment types should be considered:

...

Option 2 - Inject secrets during container creation

Draft visualization

Solution components

• FOLIO modules can read secrets from sidecar agent from well-known and standard place (e.g., Environment Variables, hereafter EV) and use them
• secret-storage agent is a jar-library supporting different container type details and move secrets from container-specific paths to EV
  • it is aware about specific details of particular container types and is able to read them
    • take secrets from environment variables for clean docker
    • use environment variable or mounted data volumes for Kubernetes Secrets
    • retrieve secrets from AWS SSM for cloud-based installation
  • pure Java enables both Vert.x and Spring support
  • potentially, it can be able to support additional functionality, e.g. runtime secret updates via event/callback mechanism if this feature is supported by container type, etc.
• pre-deployment script is a part of container building process which is responsible for retrieving secrets from particular Secret Storage and injecting them into containers
  • e.g., this https://github.com/folio-org/mod-search/blob/master/descriptors/ModuleDescriptor-template.json contains now a set of env parameters; the proposal is to have there placeholders only for those parameters that are considered as secrets, and replace placeholders with real secrets in a moment before real deployment
  • secret storage is a persistent storage for long-time keeping of secrets
• Devops / Administrator uses specific secret storage provider UI for secrets management

...

• 10 May 2021 Share this document for very early review
• 18 May 2021 Find FOLIO devops expert(-s) to review some technical capabilities and ideas
  • Current approach implemented in edge-common still looks good
• 26 May 2021 On Tech Leads meeting:
  • how important and priority this task is?
  • present the work done and current state
  • collect known requirements to FOLIO secrets management
  • receive an early feedback on which of proposed options looks more preferable
• 09 Jun 2021 Collect feedback, remarks, notes and thoughts offline
  • Raman A: aggregate them, react on potential action items
• 09 Jun 2021 Tech Leads meeting
  
  • review updates
  • Proposed next steps
    • check with DevOps regarding Kafka certs configuration
    • rename once again (sorry for that) in secret-store-agent to be widely used
    • use

      Jira Legacy
      server FOLIO Issue TrackerSystem JIRA serverId 6ccf3fe401505d01-3301b853-368a3c2e-983e90f1-20c466b11a49ee9b165564fc key MODORGSTOR-33

      to add C.UD support

...