This topic was discussed last week. The group decided, that as no decision had been made about the topic, it would be discussed again at the next meeting.
Ian Ibbotson (Use this one)describing status quo and it's issues:
- ERM modules need to attach documents to artifacts, actual using postgres large objects.
- due to design how postgresql handles this objects/binary arrays, it is not possible to backup single tenants documents within a database neither restore them distinct.
- GBV as customer would like to use S3-API but not the S3 service. So suggestion is to implement storage using min.io client
- min.io is already used by other modules
- Tod Olson asking what the implication for self hoster is?
- K-int: "Solution could be a best practice advise for modules with the need of storing files"
- Tod Olson emphasizes the need to be variable in whether using cloud solution or local solution and min.io library seem to support S3 and min.io connection
- Tod Olson does license allow to distribute the min.io library? => Marc Johnson we talked about that in the last meeting and both libraries are distributed under Apache License
- Tod Olson clarifies the question regarding the "storage abstraction" last week: Is there a need to provide a wrapper API for storage and not "is an extra module necessary" =>Ian Ibbotson (Use this one) centralized abstraction / wrapper would be painful.
- Zak Burke agrees with Ian and sees min.io to be more compatible as it would their interest to be compatible to AWS instead of AWS will not have this goal to another S3 implementations.
- Marc Johnson different decisions: Do we want to use S3 API to talk to external services? Will we suggest to run your own min.io server if you won't or can't use the S3 services.
- S3 protocol is de facto industry standard, Amazons libraries (client/server) and min.io's should be cross compatible by implementing the protocol
- Tod Olson no need for additional storage abstraction, S3 is already one. Create a document that gives guidance when there is a need to store large objects.
- Zak Burke Should we talk about the protocol version to choose instead?
- Craig McNally relegates to a decision log entry for Data Export which also discusses minio
- Is there a need to standardize the client library
- Jakub Skoczen mentioned recommend one library could improve reuse of things
- Marc Johnson S3 client library is still in use within FOLIO and known to be working; Server implementation is choosen by hosting provider; client recommendation should be postponed
Configuration of these services:
- standardized environment variables needed
- setting could be done on module level
- modules handle tenants (separate buckets) on their own?