Changes list

...

References

/wiki/spaces/TLG/pages/753669 (see Secret Storage item)

Requirements

Below is the list of identified requirements ( To anyone reviewing this document: feel free to specify more requirements if you have any):

• Secret types to be supported
  • certificate
  • API key
  • login, password
  • ... what else
• Particular implementations for following secret storage providers (with capabilities to extend in future)
  
  • HarshiCorp Vault (used for on-premises installations),
  • AWS Systems Manager Parameter Store or AWS SSM (used for AWS cloud hosting),
  • properties-file (for dev/test needs)
• Unified interface covering needs in secrets management
• Secrets CRUD
  • Are there known benefits in comparison with manual secrets update by platform administrators? Should this be done programmatically?
  • Are there specific requirements to runtime updating for secrets? I.e. should consumers be able to update secrets in runtime without restart/downtime?
• Secrets management UI
  • Is it required? Technically, both AWS SSM and Vault have their own management user interfaces, Rancher provides an option to manage secrets in form of environment variables in UI forms
    
• Both Spring-Way and Vert.x-based modules
  
• Multi-tenancy support
  

  •  Using the same credentials for every tenant is a kind of concern

• Audit capabilities
  
  • Access to secrets should/must be logged for further audit - any specific requirements?
  • Can AWS SSM and Vault logs be efficient for this, or do we need to have any dedicated log storage for that?
• ...
• ... anything else to be added?
• ...
• ... Need to review the vision and requirements with Tech Leads
  

Please refer to FOLIO secrets management - Requirements for a list of identified requirements.

Analysis

Current state

Three environment types should be considered:

...

Solution components

...

• secret-store-agent (or client) as a pure Java library available for both Vert.x and Spring
• Module edge-api-utils (https://github.com/folio-org/edge-api-utils) currently presenting common / shared Edge API utilities for edge-common and edge-common-spring can be used as a basis for secret-store-agent and be enriched with new functionality
• secret-store-agent provides an abstract unified interface (for connecting to Secret Store and work with secrets) and an extendable number of particular implementations
• CRUD support makes sense (see cases A and B from Craig's comment below)
• >> How does authentication and authorization fit into these models? For example, can every module access every secret? does each module access the secret store with different credentials?
  • secret-store-agent in every module should be configured separately
  • each module (more precisely - each secret-store-agent) can access the secret store with different credentials; this can be managed via
    
    • Vault Token https://www.vaultproject.io/docs/concepts/tokens which can be mapped to a set of one or more attached policies. In turn, these policies control what the token holder is allowed to do within Vault. Note that usage of root token in production is not recommended
    • AWS IAM role
  • another option is to manage access on application level, e.g. every module has own static key (or token, or salt) that is used as a part of secret key

• Multi-tenancy support - all the secrets are to be accessible per tenant level

  • it's possible to add secrets for a new tenant without module(-s) restart
• Secrets can be cached with defined TTL by secret-store-agent (under discussion - caching is not the best practice for secrets)
• Logging and audit
  • via native Secret Store provider capabilities
  • still FOLIO-centralized option is possible if need
• Certificates (e.g., for Kafka clients) are to be provided and mounted externally as JKS while secret-store-agent can be used to manage keystore & truststore location and password
  • Enabling SSL and ACL for Kafka

...

• 10 May 2021 Share this document for very early review
• 18 May 2021 Find FOLIO devops expert(-s) to review some technical capabilities and ideas
  • Current approach implemented in edge-common still looks good
• 26 May 2021 On Tech Leads meeting:
  • how important and priority this task is?
  • present the work done and current state
  • collect known requirements to FOLIO secrets management
  • receive an early feedback on which of proposed options looks more preferable
• 09 Jun 2021 Collect feedback, remarks, notes and thoughts offline
  • Raman A: aggregate them, react on potential action items
• 09 Jun 2021 Tech Leads meeting
  
  • review updates
  • Proposed next steps
    • check with DevOps regarding Kafka certs configuration
    • rename once again (sorry for that) in secret-store-agent to be widely used
    • use

      Jira Legacy
      server FOLIO Issue TrackerSystem Jira serverId 6ccf3fe401505d01-3301b853-368a3c2e-983e90f1-20c466b11a49ee9b165564fc key MODORGSTOR-33

      to add C.UD support

...