Concern about tenant separation being out of scope.
Goal of RFC was to reduce the partition count and therefore reduce costs
Tenant separation seems to be another large issue and too much scope for this RFC
Original per-tenant Kafka messages at least allowed us to keep
Temporary Kafka security solutionwould be rendered obsolete/deprecated by this RFC (though it seems unlikely this will be done any time soon)
Does theALL tenant topic would close the door to the possibility of separating tenant messages?
RFC describes an option to combine tenant message, not a requirement
Major trade-off is that we are committing in the long term to support two fundamentally different approaches for the foreseeable future, but does keep the door open
RFC is intended to be practical was of addressing pain points, which includes limited development and funding; hurdle is very high to implement enhanced security
Other notes?
Inference: providers do not migrate all tenants to new flower releases all at once, and therefore need multiple versions of a module are active at once. This gives a scenario where multiple versions of Inventory will be consuming a message.
Response was cut off due to end of meeting
To continue in dedicated discussion on Monday
Discussion Notes:
Olamide Kolawole explains how Upgrades of modules using Kafka nowadays work:
If consumer modules are upgraded, both, the old and the new Module may consume messages. If for example the mod-search module in a updated version changes the index structure, the Upgrade cannot work without inconsistencies.
Kafka Tenant Collections don't make it worse neither offer a solution
Julian Ladisch solution can be to use the "environment" in the topic path and put the flower release name in there.
Olamide Kolawole Security could be done using message encryption on tenant basis. This is not part of this RFC, but a standard solution that would have to be implemented
Marc Johnson Astonishing, that this existing upgrade problem is not a issue for hosting providers today
Olamide Kolawole mod-pubsub solves the problem, but mod-pubsub has other problems with performance and lost messages
Marc Johnson pubsub is deprecated, we should not be relying on pubsub. Biggest concern with this proposal is having to support both possibilities. More configuration possibilities - more tests - Bugfest is run by Ebsco, so the Tenant Collections will be testet for sure.
Olamide Kolawole thinks that topics per tenant will be used in Bugfest, because it is default.
Craig McNally If there are no other concerns, we can move forward to the public review. 7 TC members are in the meeting
Approved by lazy consensus.
Julian Ladisch will resolve the discussion and merge the PR.