BestHub
Sign in
Big Data Technology Architecture
Big Data Technology Architecture
May 24, 2020 · Big Data

Kafka Core Concepts: Architecture, Replication, Controllers, High Watermark, and Message Production/Consumption Mechanisms

This article provides a comprehensive overview of Kafka's internal architecture, including its controller role, replication strategy, high‑watermark mechanism, message serialization, partitioning, batching, consumer pull model, log storage format, and retention policies, supplemented by a detailed mind‑map illustration.

High Watermarkmessage-consumption
0 likes · 7 min read
Kafka Core Concepts: Architecture, Replication, Controllers, High Watermark, and Message Production/Consumption Mechanisms
Big Data Technology & Architecture
Big Data Technology & Architecture
Dec 21, 2019 · Big Data

Kafka Offset Management and Replication Mechanisms Explained

This article provides a comprehensive technical overview of Kafka's offset handling, covering the request entry point, in‑memory offset sources, offset commit and fetch implementations, file storage layout, and the leader‑follower synchronization process that ensures data replication and high‑watermark updates.

Big DataHigh WatermarkKafka
0 likes · 16 min read
Kafka Offset Management and Replication Mechanisms Explained