Apache Kafka 2.8 Introduces KRaft: Running Without ZooKeeper

In the upcoming Kafka 2.8 release, the distributed publish‑subscribe system Apache Kafka will use an internal Quorum controller to replace ZooKeeper, allowing users to run Kafka without any ZooKeeper dependency. This saves compute resources, improves performance, and supports larger clusters.

Historically, Apache ZooKeeper acted as a coordination service for distributed systems like Kafka; every broker connected to ZooKeeper for registration and state storage. While powerful, ZooKeeper added complexity as a separate component, prompting the move to an internal Quorum controller.

The effort began in April of the previous year, and partial results are now available: in version 2.8 users can run Kafka in KRaft (Kafka Raft metadata) mode, where metadata previously managed by the controller and ZooKeeper is merged into the new Quorum controller and executed inside the Kafka cluster, though it can also be deployed on dedicated hardware for special scenarios.

KRaft employs an event‑driven mechanism to track cluster metadata, replacing RPC‑based tasks with event‑driven log transport, which enables Kafka to support more partitions.

Removing ZooKeeper makes Kafka lighter and more suitable for small‑scale workloads, edge deployments, and lightweight hardware solutions.

It is important to note that the early‑access version does not yet support features such as ACLs, security, transactions, partition reassignment, or JBOD, and the vendor advises against using it in production until these features are added in later releases.

Reference:

[1] Apache Kafka Made Simple: A First Glimpse of a Kafka Without ZooKeeper – https://www.confluent.io/blog/kafka-without-zookeeper-a-sneak-peek/

[2] Apache Kafka 2.8 No Longer Requires ZooKeeper – https://www.ithome.com.tw/news/143569