Mastering Strimzi Kafka Operator: Architecture, Deployment & Tuning on K8s

Background

Kafka applications built with the Kubernetes Operator pattern offer full lifecycle automation, strong scalability, and self‑healing capabilities, making them a key approach for cloud‑native deployments. The two dominant Kafka Operator solutions are the community‑driven Strimzi Kafka Operator and Banzai Cloud’s koperator . Compared with koperator, Strimzi provides richer upstream Kafka ecosystem support, extensible custom operators, and higher configurability.

Strimzi Kafka Operator Overview

Strimzi is implemented on top of the fabric8 Kubernetes client and enables automated deployment, management, and monitoring of multiple Kafka clusters on K8s. Hosted by CNCF since August 2019 and primarily maintained by the Red Hat community, it is one of the few Java‑based Operators in the CNCF landscape. Its key capabilities include one‑click cluster provisioning, generic Kafka management, cross‑cluster replication, integration with big‑data sources, unified monitoring and alerting, fault handling, and data rebalancing.

Operator Architecture

Strimzi encapsulates four top‑level operators:

Cluster Operator – manages the Kafka cluster and surrounding ecosystem tools.

Entity Operator – creates User Operator and Topic Operator instances for a given cluster.

User Operator – handles authentication, authorization, and access control for different user types.

Topic Operator – manages Kafka topics.

Installation and Usage

1. Deploy Cluster Operator

Create a dedicated namespace and install the operator using the official YAML:

kubectl create ns kafka-operator

curl -L https://strimzi.io/install/latest?namespace=kafka-operator > strimzi-kafka-operator.yaml

Adjust the STRIMZI_NAMESPACE environment variable to * so the Cluster Operator can manage Kafka clusters across all namespaces, then create the required ClusterRoleBindings:

kubectl create clusterrolebinding strimzi-cluster-operator-namespaced --clusterrole=strimzi-cluster-operator-namespaced --serviceaccount kafka-operator:strimzi-cluster-operator
kubectl create clusterrolebinding strimzi-cluster-operator-watched --clusterrole=strimzi-cluster-operator-watched --serviceaccount kafka-operator:strimzi-cluster-operator
kubectl create clusterrolebinding strimzi-cluster-operator-entity-operator-delegation --clusterrole=strimzi-entity-operator --serviceaccount kafka-operator:strimzi-cluster-operator

Verify that the Cluster Operator pod reaches the Running state:

kubectl -n kafka-operator get pods -w

2. Create a Kafka Instance

Apply an example manifest that creates an ephemeral single‑node Kafka cluster with an embedded Zookeeper:

kubectl apply -f https://strimzi.io/examples/latest/kafka/kafka-ephemeral-single.yaml -n kafka

3. Produce and Consume Messages

Run temporary producer and consumer pods using the Strimzi Kafka Docker image:

kubectl -n kafka run kafka-producer -ti --image=quay.io/strimzi/kafka:kafka:0.38.0-kafka-3.6.0 --rm=true --restart=Never -- bin/kafka-console-producer.sh --bootstrap-server my-cluster-kafka-bootstrap:9092 --topic test
kubectl -n kafka run kafka-consumer -ti --image=quay.io/strimzi/kafka:kafka:0.38.0-kafka-3.6.0 --rm=true --restart=Never -- bin/kafka-console-consumer.sh --bootstrap-server my-cluster-kafka-bootstrap:9092 --topic test --from-beginning

Cluster Operator Technical Implementation

Overall Architecture

The Cluster Operator is the core module of Strimzi. It defines several Custom Resource Definitions (CRDs) and uses the Kubernetes informer mechanism to watch events. Reconciliation logic is delegated to six AssemblyOperator implementations:

KafkaAssemblyOperator – creates the Kafka cluster, handles persistent storage, and configures external access.

KafkaConnectAssemblyOperator – runs Kafka Connect services for data import/export.

KafkaBridgeAssemblyOperator – starts the Kafka Bridge, exposing a REST API.

KafkaMirrorMaker/2AssemblyOperator – replicates data between clusters.

KafkaRebalanceAssemblyOperator – interacts with Cruise Control to trigger and monitor rebalancing.

When the operator pod starts, it creates the necessary ClusterRoles, enables leader election for high availability, and registers event handlers for each CRD. The operator processes work‑queue items, translates the desired state from CRD specs into Kubernetes resources (Pods, Deployments, Services, ConfigMaps, PVCs, Secrets, etc.), and updates the resource status until all components report Ready.

Kafka Cluster Reconcile Flow

The KafkaAssemblyOperator’s reconciliation covers several sub‑steps:

Storage tuning – supports dynamic creation and expansion of EmptyDir, JBOD, or PVC storage.

Service tuning – creates Services of type Route, Ingress, Internal, LoadBalancer, NodePort, or ClusterIP based on listener definitions.

Secret handling – generates self‑signed CA certificates for broker‑to‑broker encryption.

ConfigMap updates – injects broker configuration, metrics, and logging settings.

Pod creation – assembles pod specs with initContainers, containers, ports, volumes, imagePullSecrets, liveness/readiness probes, etc.

Status checks – continuously verifies that Pods, Services, and listeners reach the Ready state.

Conclusion

Compared with traditional on‑premise Kafka deployments, a Strimzi‑based containerized Kafka service offers automated lifecycle management, high scalability, fault recovery, and integrated monitoring, logging, and alerting. This article introduced Strimzi’s core functions, provided step‑by‑step installation and usage guidance, and dissected its architecture and reconciliation process to help readers adopt and operate the operator effectively.