[FOLIO-2278] Spike: Secret Storage Created: 23/Sep/19  Updated: 02/Aug/22

Status: Open
Project: FOLIO
Components: None
Affects versions: None
Fix versions: None

Type: Task Priority: TBD
Reporter: Craig McNally Assignee: Unassigned
Resolution: Unresolved Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original estimate: Not Specified

Issue links:
Blocks
blocks MODORGSTOR-33 Implement encryption of interface cre... Draft
Relates
relates to MODEMAIL-4 Setup SMTP server: Part 1 Closed
relates to FOLIO-2875 SPIKE: how to distribute configuratio... Closed
relates to MODKBEKBJ-507 Spike: investigate possible ways to s... Draft
Sprint:

 Description   

Overview

Encrypted storage is needed by several apps e.g. for things like (RMAPI keys, SMTP credentials, FTP credentials, etc.). Currently these things are not encrypted at rest. Some have their own APIs (just to access/manage the sensitive data) with their own permissions, others are simply stored in mod-configuration

Specific Questions to be answered:

  1. What technologies would be appropriate for this?
  2. What might a solution look iike? Is there some abstraction layer/interface allowing for different implementations?
  3. What's required for a module to store/retrieve a secret?
  4. What's required on the infrastructure side?

Acceptance Criteria

  • The questions above are answered
  • One or more approach is documented on the wiki
    • If more than one approach is provided, pros & cons are provided along with a recommendation
  • POC code (if applicable) is committed to git and linked to from the wiki page


 Comments   
Comment by Craig McNally [ 23/Sep/19 ]

My initial thought was to create something like mod-vault that uses https://www.vaultproject.io/

We ran into a similar requirement with the Edge APIs; where to store institutional user credentials. The approach we took there is to build a simple SecureStore (java) interface and add a few implementations:

  • Vault
  • AWS Param Store
  • Properties file (intended for dev use only)

However, we sort of punted on the infrastructure side of this and left it entirely up to those setting up/managing the system to setup the secret store and provision it with secrets.

I do think it would be nice to have that sort of flexibility. Maybe we define a secure-store (folio) interface and allow multiple implementations of it... e.g. mod-vault, mod-aws-param-store, etc.

Comment by Jakub Skoczen [ 16/Oct/19 ]

Ian Hardy can you provide some input on secret management with Kubernetes?

Comment by Ian Hardy [ 16/Oct/19 ]

Kubernetes secrets allow for passing secrets as environment variables or configuration files to containers on the cluster. We've used them for infrastructure level secrets like postgres credentials required to connect to the module db. I think k8s secrets are useful for anything that needs to get set once when a container is started, I think it would be less useful for something that a user of a folio system is supposed to be able to modify after startup.

So for an edge module api key the workflow might look like:
1. sysadmin/hosting provider sets secret w/api key
2. edge module is deployed w/instructions for pod to pick up additional environment vars from secret
3. for an update, sysadmin edits secret and redeploys pod.

Note that a separate pod would probably need to be used for separate tenants in this scenario (assuming there's a one to one relation between api keys and tenants).

Comment by Craig McNally [ 13/Feb/20 ]

Ian Hardy I think we need to be able to support multi-tenancy here.

a separate pod would probably need to be used for separate tenants

Is this a reasonable thing to do? Would tenants typically live in separate pods otherwise?

Comment by Craig McNally [ 13/Feb/20 ]

I'd be surprised if the current mechanism for storing the types of secrets mentioned in the description isn't raised as part of the security audit feedback, so this will likely be resurfacing again soon.

Comment by Ian Hardy [ 13/Feb/20 ]

Is this a reasonable thing to do? Would tenants typically live in separate pods otherwise?

It's possible for many tenants to share one deployment (I said pod, last time) of a module as long as they can all coexist with whatever arguments/variables are configured on that deployment. So if we want tenant_a and tenant_b to share a deployment of an RMB based storage module, they can do that as long as they're using the same db. This limitation is because we can only pass 1 set of db credentials to a deployment of say mod-inventory-storage. If we didn't want tenant_a and tenant_b to share a db instance, we'd have to have two deployments of the module each w/different credentials configured.

I think both are reasonable things to do. Having multiple tenants share one instance is possibly more convenient and less expensive from a deployment perspective.

I suppose this isn't necessarily a limit for edge modules as long as there is a way to configure secrets for multiple tenants at once, but the limitation described above must have been on my mind when I wrote that hypothetical scenario where the key is configured through an environment variable.

Comment by Dennis Bridges [ 05/May/21 ]

Craig McNally is there any chance this will be completed in R2. We have a story about encrypting interface credentials that is blocked by this. If it will not be completed I will push that story out to R3. thx!

Comment by Raman Auramau [ 07/May/21 ]

Hi, I'm currently working through this item, please take a look at https://wiki.folio.org/display/~Raman+Auramau/FOLIO+secrets+management for more details.

Generated at Thu Feb 08 23:19:30 UTC 2024 using Jira 1001.0.0-SNAPSHOT#100246-sha1:7a5c50119eb0633d306e14180817ddef5e80c75d.