DR-000026 - S3/MinIO

Owned by Radhakrishnan Gopalakrishnan
Last updated: May 06, 2024 by Jenn Colt
2 min read
Submitted Date

  

Approved Date

 

Status

ACCEPTED

ImpactMEDIUM

 

Overrides/Supersedes 

This decision was migrated from the Tech Leads Decision Log as part of a consolidation process.  The original decision record can be found here.

RFC 

N/A

Stakeholders

  • Technical Leads, Developers

Contributors

Julian Ladisch 

Approvers

Background/Context

Request of the Security Group on 2021-08-20:

  • Binary files must be stored with strict tenant and module separation.
  • A FOLIO MinIO security guide for developers and sysops must be published and reviewed by the security team before more modules start using it.
    • e.g. Including guidance for how to do the tenant/module separation.
    • The tech leads group will discuss this as noted during the TC meeting (see TC 2021-08-18 Meeting notes).

Tech Leads Meeting on 2021-08-25:

  • Jakub Skoczen writes:

    Note: we have discussed this topic during the Tech Leads meeting and decided to form a working group. The task of the group will be to define a recommended standard for object storage, including handling of tenant seperation. Since developers involved in the S3/MinIO inlcusion in Data Import are not longer on the project (Kruthi Vuppala. Taras Spashchenko) we will need volunteers to participate in this effort. Ian Ibbotson, Owen Stephens: I asked Marc Johnson to reach out to you guys regarding this effort as you may be active working in this area at the moment. If you are OK with that I'd propose that we create a slack channel for the group.

  • Kateryna Senchenko writes:

    Hi Jakub Skoczen, please include Vladimir Shalaev, Aliaksandr Fedasiuk and myself to this group. Thank you!

Assumptions

N/A

Constraints

N/A

Rationale

Decision

Decision of the TC on 2021-08-18:

  • The S3 API is the protocol for external (non-Postgres) object storage in FOLIO and standard environment variables/configuration of an S3 server on a per module basis.
  • The module implementation of object storage must conform to the FOLIO standards for tenant separation
    • The recommended standards will be referenced/pointed out by the tech leads.
      • e.g. how tenant separation should be handled, etc.
      • Jakub Skoczen will raise the topic with the tech leads
  • Should the min.io client library be the recommended client as object storage choice agnostic library, but modules could also choose the native S3 implementation?
    • Yes.
  • Should teams be allowed to store binary data in Postgres?  In some cases (amount of data is small, etc.)?
    • Yes.  If external object storage (not Postgres) is required, it must be S3.
    • Regardless of which is used, tenant separation still must be done. (x3)

Implications

  • Pros
    • N/A
  • Cons
    • N/A

Other Related Resources