Apache Kafka 3.0 Release Highlights and New Features
The article provides a comprehensive overview of Apache Kafka 3.0, detailing its core APIs, two main use‑cases, major feature additions, deprecations, KRaft consensus improvements, enhanced producer guarantees, and numerous KIP‑driven changes across the broker, client, Connect, Streams, and MirrorMaker components.
Apache Kafka is a distributed open‑source streaming platform originally designed as a message queue and has evolved into a mature event‑stream processing system since its open‑source release by LinkedIn in 2011.
Kafka offers four core APIs that enable two primary application categories: building reliable real‑time data pipelines between systems and creating real‑time streaming applications that react to data streams.
Version 3.0.0 introduces many new features, including the deprecation of Java 8 and Scala 2.12 support, KRaft‑based metadata snapshots, removal of message formats v0/v1, stronger default producer delivery guarantees, and optimizations to OffsetFetch and FindCoordinator requests.
KRaft (Kafka Raft) becomes the built‑in consensus mechanism, replacing ZooKeeper, with extensive metadata and API improvements such as snapshot support, updated metadata record types, and producer ID generation handled by the controller.
The default producer configuration now enables idempotence and acks=all, providing stronger ordering and durability out of the box.
Kafka Connect gains a unified restart API for connectors and tasks, removal of internal converter properties, default enablement of connector client overrides, and inclusion of connector log context in Log4j configurations.
Kafka Streams receives numerous updates: enhanced timestamp synchronization, new TaskMetadata methods for committed offsets, API refactoring of TaskId, migration to new metadata interfaces, addition of Instant‑based methods to session stores, new system and stream time APIs, default SerDe changes, replication factor defaults, and deprecation of older EOS configurations in favor of eos‑v2.
MirrorMaker 2 receives configuration enhancements for offset‑sync topic placement and deprecation of MirrorMaker 1, signaling future focus on MM2.
The article concludes with a brief invitation to join the author’s architecture community for further learning.
Java Architect Essentials
Committed to sharing quality articles and tutorials to help Java programmers progress from junior to mid-level to senior architect. We curate high-quality learning resources, interview questions, videos, and projects from across the internet to help you systematically improve your Java architecture skills. Follow and reply '1024' to get Java programming resources. Learn together, grow together.
How this landed with the community
Was this worth your time?
0 Comments
Thoughtful readers leave field notes, pushback, and hard-won operational detail here.