The Apache Kafka 3.3 release introduces production‑ready KRaft mode and a series of KIP enhancements across brokers, streams, and connect, detailing new metrics, API changes, and operational improvements while encouraging users to review the full release notes and try the updated platform.
This article provides a thorough overview of Apache Kafka, covering its core streaming concepts, key components such as topics, partitions, producers and consumers, common use cases, step‑by‑step installation and multi‑broker configuration, fault‑tolerance testing, and an introduction to Kafka Connect for data import/export.
This article provides a comprehensive guide to Apache Kafka, covering its core concepts, architecture, key APIs, topics and partitions, deployment steps, multi‑broker clustering, fault tolerance, and data integration using Kafka Connect, with detailed command‑line examples.
This article introduces Kafka Connect, explaining its purpose as a scalable and reliable tool for moving data between Apache Kafka and external systems, detailing its core concepts, architecture, deployment modes, configuration files, and a step‑by‑step example that streams data from a file source to a file sink.
This article explains how Debezium's MongoDB Source Connector captures change events from replica sets or sharded clusters, streams them to Kafka topics, and provides detailed configuration, deployment, monitoring, and troubleshooting steps for building reliable change‑data‑capture pipelines.
This article chronicles the four‑year evolution of Chehaoduo Group’s Kafka ecosystem—from its initial role as a simple data‑ingestion layer to becoming the core of the company’s large‑scale data pipeline—detailing cluster management, upgrade strategies, multi‑cluster deployment, AVRO schema handling, SDK development, and operational lessons learned.
Debezium is an open‑source distributed platform built on Apache Kafka that captures row‑level changes from databases via change data capture, providing source connectors, an optional embedded engine, and features like low‑latency streaming, snapshots, filtering, masking, and integration with various sink systems.
Apache Kafka 2.3 introduced static members and incremental cooperative rebalancing to reduce disruptive global rebalances, allowing workers to retain assignments during failures, schedule delayed rebalances, and improve scalability for Kafka Connect clusters, balancing availability and fault tolerance.
This article introduces Kafka Connect, a framework for building scalable data pipelines between Kafka and other systems, covering its architecture, key concepts like connectors and tasks, and practical deployment examples.
Debezium is an open‑source distributed platform built on Apache Kafka that turns existing databases into real‑time event streams by capturing row‑level changes via change data capture, offering source and embedded connectors, flexible topic routing, and features such as snapshots, filtering, masking, and monitoring.
This article compares the main Kafka distributions—Apache Kafka, Confluent Kafka, and CDH/HDP Kafka—examining their origins, feature sets, ecosystem support, and trade‑offs to help you choose the most suitable version for your streaming workloads.
This step‑by‑step guide walks you through installing Kafka, starting a single‑node broker, producing and consuming messages, configuring a multi‑node cluster with replication, testing fault tolerance, and using Kafka Connect to import and export data between files.
