Connect to a Multi-Broker Kafka Cluster in a Kubernetes Production Environment

Bitnami provides a Kafka Helm chart ready to run in demanding production environments. It is completely up-to-date and configured to provide high performance and fault tolerance. 

This chart includes a values-production.yaml file which lets you set numerous parameters oriented to production configuration, such as the number of Kafka nodes, default replication factors for automatically created topics, Kafka authentication, and so on.

To support production scenarios that use external clients to connect to the Kafka clusters, a new feature has been added: the option to include and configure an additional listener and advertiser listener for your deployment.

How can I connect to my cluster externally if I am running more than one Kafka broker?  

That was the question that some of our customers asked. As a result, we have enabled connections from outside the Kubernetes cluster for multi-broker Kafka deployments. 

The Bitnami Kafka Helm chart now includes new parameters in its values-production.yaml and values.yaml files that support the configuration of an additional listener and advertised listener to enable accessing Kafka brokers using external clients. 

There are two ways to configure external access to the Kafka brokers: using LoadBalancer or NodePort services.
In both cases, the Helm chart will create an independent service for each Kafka broker (a service per pod). The next step is to add the corresponding parameter values in each broker for the service selected: 

• LoadBalancer: loadBalancerIP 
• NodePort: nodePort, and optionally, the domain   

Check out the Bitnami Kafka Helm chart documentation to learn how to configure the external connection parameters depending on the approach you prefer for your deployment. 

With this new feature, the Bitnami Kafka Helm chart covers user needs in real production scenarios. 
Do you want to learn more about how to use the Bitnami Kafka Helm chart in production? Follow this step-by-step guide to deploy a scalable Apache Kafka/Zookeeper cluster on Kubernetes.