2023-05-22 - Kafka Topics RFC

Owned by Craig McNally
Last updated: May 22, 2023
2 min read

Date

Attendees 

Discussion items

TimeItemWhoNotes
1 minScribeAll

Craig McNally will take notes


*

Kafka Topics RFC

All 

Background: 

Discussion Notes:

  • Summary of the problem:
    • With the current approach, it's possible to use Kafka ACLs to improve security.
      • See Kafka Temporary Security Proposal <add link>
    • Comments were made in the RFC suggesting that the proposal would prevent that from being an option if accepted/adopted.
  • Marc Johnson (via chat):  We don't have a quorum... so an official decision won't happen in this meeting
  • Managing ACLs is currently the responsibility of system operators
    • This is currently a manual and cumbersome process...  Need to restart brokers, need to be aware of when topics are created, requires credential management, etc.
  • Olamide Kolawole: The proposed changes are optional; essentially an opt-in.
  • Marc Johnson: If I understand correctly, Julian Ladisch indicates in the RFC that it's currently possible to implement the temporary Kafka security using ACLs  and Olamide Kolawole suggests it isn't.
    • Olamide Kolawole: It's not possible because code changes are required (provide credentials to authenticate with Kafka.
  • Jeremy Huff: would adopting the proposal in the RFC paint us into a corner (security-wise)?
  • Olamide Kolawole:  I don't think so.  It would be possible to use message encryption for instance, but that needs to be thought through and formally proposed.
  • Julian Ladisch: if the modules automatically create the topics, then using ACLs is indeed challenging, but if you have some external process which creates the topics, it is more feasible.
  • Julian Ladisch:  It might be sufficient to document the issue in the RFC.  It doesn't necessarily need to be a deal breaker for acceptance of the RFC.
  • The problematic statement in the RFC is that the one around ACLs being out of scope.  Julian Ladisch feels it should be in-scope.
  • Julian Ladisch maybe "in-scope" is a poor choice of words, but it is related or at least of note.
  • Olamide Kolawole will work with Julian Ladisch to get this sufficiently documented.
  • Marc Johnson tried to gain a better understanding of what documentation changes are required, and to which section.  
  • After discussion, Julian Ladisch indicated that he accepts the RFC in its current form.

Action Items

  •