Apache Kafka 3.0.0 Release: New Features, API Changes, and KRaft Improvements

Kafka was originally designed as a message queue and, since its open‑source release by LinkedIn in 2011, has evolved into a mature event‑stream processing platform.

It provides four core APIs that enable two primary use cases: building reliable real‑time data pipelines for data transfer between systems, and creating real‑time streaming applications that react to data streams.

Apache Kafka 3.0.0 was officially released with a host of new features and significant API changes.

Support for Java 8 and Scala 2.12 is deprecated and will be removed in version 4.0, giving users time to migrate.

KRaft (Kafka Raft) now supports metadata‑topic snapshots and other self‑managed quorum improvements.

Message formats v0 and v1 are deprecated.

Producers enable the strongest delivery guarantees (acks=all, idempotence) by default.

OffsetFetch and FindCoordinator requests have been optimized.

MirrorMaker 2 gains more flexible configuration, while MirrorMaker 1 is deprecated.

Kafka Connect can restart connector tasks in a single call.

Connector client overrides and connector log context are enabled by default.

Kafka Streams receives timestamp‑synchronization enhancements and API clean‑ups.

The release also introduces extensive updates to the Kafka Raft controller and broker metadata handling, with new KIP‑based changes such as KIP‑750, KIP‑751, KIP‑630, KIP‑746, KIP‑730, KIP‑679, KIP‑735, KIP‑709, KIP‑699, KIP‑724, KIP‑707, KIP‑466, KIP‑734, and many others that affect producers, consumers, AdminClient, Connect, Streams, and MirrorMaker.

Key highlights include:

From version 3.0, producers default to the strongest delivery guarantees, ensuring ordering and durability.

Kafka Connect now supports restarting all or only failed connectors and tasks with a single API call (KIP‑745).

Kafka Streams adds new methods to TaskMetadata, improves timestamp handling (KIP‑695), and updates default SerDe behavior (KIP‑741).

MirrorMaker 2 can configure the location of offset‑sync internal topics, allowing read‑only source clusters and separate storage clusters (KIP‑716).

Although KRaft is not yet recommended for production, the release brings many improvements to its metadata and APIs, including exactly‑once semantics support and partition reassignment enhancements.

Overall, Kafka 3.0.0 delivers a comprehensive set of feature upgrades, API refinements, and deprecations that advance the platform’s streaming capabilities and prepare it for future major releases.