Versions Compared

Key

  • This line was added.
  • This line was removed.
  • Formatting was changed.

Date

...

TimeItemWhoNotes
Review the Kanban boardTeam

min.io / s3 compatible file storage

How should FOLIO store files (like PDFs attached to orders, agreements, etc.)?

TC discusses whether FOLIO should accept min.io as an official part of FOLIO platform:

Background:

mod-invoice-storage stores files (PDFs) into a JSONB property using base64 encoding:
https://github.com/folio-org/acq-models/blob/master/mod-invoice-storage/schemas/document.json

mod-agreements and mod-licenses store files (PDFs) into pg_largeobject without any tenant or module separation. The ERM development team rejected the request to convert it into a PostgreSQL solution like bytea that provides tenant and module separation (ERM-1779). The ERM development team wants to move to an external solution (UXPROD-3172) like min.io (or some other s3 compatible file storage).

PostgreSQL supports storing binary files: https://wiki.postgresql.org/wiki/BinaryFilesInDB

  • When should files be stored in the database? The common suggestion here is when the files have to be ACID.

  • When is it bad idea to store binary files in the database? Very large files (100MB+), where performance is critical to the application.

  • Do smaller binary files result in bad performce? No, because bytea and text data types both use TOAST (details here).
  • For a detailed discussion see above BinaryFilesInDB link.

min.io server for multi-tenancy is licensed under GNU Affero Public License Version 3 (AGPLv3), this was changed in April 2021, it had been Apache 2 before. Min.io server for bare-metal or single-tenant continues to be released under Apache v2.0.

Proposal of a Security Team decision:

  • Binary files must be stored with strict tenant and module separation.
    • This is to support multi-tenant installations.
    • This is to support modules the sysOp doesn't fully trust.

Action items

  •