Elasticsearch (UXPROD-2591)

[UXPROD-2935] NFR: Increase security of Kafka for mod-search Created: 03/Mar/21  Updated: 14/Jun/21  Resolved: 14/Jun/21

Status: Closed
Project: UX Product
Components: None
Affects versions: None
Fix versions: R2 2021
Parent: Elasticsearch

Type: New Feature Priority: P2
Reporter: Ann-Marie Breaux (Inactive) Assignee: Magda Zacharska
Resolution: Done Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original estimate: Not Specified

Issue links:
Cloners
clones UXPROD-2929 NFR: Increase security of Kafka for D... Closed
Relates
relates to MSEARCH-105 Use TLS for Kafka connection Closed
Epic Link: Elasticsearch
Front-End Confidence factor: Medium
Back End Estimate: Small < 3 days
Back End Estimator: Bohdan Suprun (Inactive)
Estimation Notes and Assumptions: Assuming Kafka configuration on reference/rancher environments out of the scope.
Development Team: Falcon
PO Rank: 0
Cap Plan Fix Version (DO NOT CHANGE): R2 2021

 Description   

Current situation or problem:
There were some concerns raised in the community regarding how secure the direct connection will be. To address these concerns, the new solution was designed: https://folio-org.atlassian.net/wiki/display/DD/Temporary+Kafka+security+solution.
The solution was reviewed and approved by the Security group and Tech Council.

Multi-tenancy on Kafka's side is implemented for the modules differently, so it will take time to make the changes in them that unify the multi-tenancy approach.
However, the direct Kafka connections should be secured in R1, so a simplified version of the solution is proposed for now.

In scope

  • Add module-level Kafka user credentials support to mod-search. The credentials should be provided to all producers and consumers of a module with other Kafka client settings.
  • Add TLS support to the same modules. Same here, the settings should be provided to all producers and consumers of a module with other Kafka client settings.

Out of scope
This work is also needed for Data Import and Remote Storage, but those applications/modules are managed by other dev teams

Proposed solution/How it could be implemented:

  • ModuleDescriptor should be updated to include the new Kafka settings: TLS, and, for now, user credentials (the credentials later could be injected to container a different way, for instance, as EnvironmentVariables)
  • Update a class that represents Kafka config
  • Update a class(es) that creates and assigns the config to Kafka producers and consumers
  • Test the updates

Links to additional info
https://folio-org.atlassian.net/wiki/display/DD/Temporary+Kafka+security+solution.

Additional information:
On TC meeting on March 3rd, 2021 it was decided that this work will be in scope for R2.



 Comments   
Comment by Khalilah Gambrell [ 02/Jun/21 ]

Magda Zacharska, will this be done for Juniper?

Generated at Fri Feb 09 00:28:02 UTC 2024 using Jira 1001.0.0-SNAPSHOT#100246-sha1:7a5c50119eb0633d306e14180817ddef5e80c75d.