Kafka 2.8.0 Release: Say Goodbye to ZooKeeper with Raft Metadata Mode

Kafka enthusiasts should note that Kafka 2.8.0 was officially released on April 19, 2021.

The most eye‑catching change is that Kafka now replaces ZooKeeper with a self‑managed arbitration mechanism, meaning Kafka can run without ZooKeeper for the first time.

This shift, known as Kafka Raft Metadata mode , marks a historic milestone for the platform.

Previously, ZooKeeper provided authoritative metadata storage for partitions, leaders, and other critical information, making the system complex to operate because administrators had to manage two separate logging and networking layers.

Removing ZooKeeper may cause short‑term pain, but the long‑term benefits for Kafka’s evolution outweigh the drawbacks.

In addition to the ZooKeeper separation, version 2.8.0 adds several new features:

KAFKA-10500 – API to start and stop stream threads

KAFKA-10700 – Support for mutual TLS authentication using SASL_SSL listeners

KAFKA-10749 – IP rate‑limiting via connection throttling

KAFKA-5488 – KStream.branch no longer returns an array that must be accessed by known indices

KAFKA-6687 – Allow multiple reads of a topic

KAFKA-6943 – Clean shutdown of Kafka Streams when any or all threads crash

KAFKA-9023 – Producer network exception responses now log more information

KAFKA-12327 – Remove MethodHandle usage from CompressionType

KAFKA-12365 – Proxy/controller does not support block API (currently)

KAFKA-12394 – Consider topic ID existence and authorization errors

KAFKA-4748 – Provide a way to shut down all worker processes in a Streams application

KAFKA-10722 – Store timestamps even when not required

KAFKA-10723 – LogManager leaks internal thread‑pool activity during shutdown

Hundreds of bug fixes and optimizations are also included in this release.

For a complete list of changes, see the official release notes: https://downloads.apache.org/kafka/2.8.0/RELEASE_NOTES.html