Implementing Log Snapshotting in Raft: A Step‑by‑Step Guide

This article provides a comprehensive tutorial on adding log snapshotting (snapshotting) to a Raft‑based distributed key‑value store, explaining the motivation, the snapshot mechanism, and detailed Go code for generating, transferring, applying, and persisting snapshots to reduce log size and improve performance.

ConsensusDistributed SystemsGo

0 likes · 15 min read

Implementing Log Snapshotting in Raft: A Step‑by‑Step Guide

dbaplus Community

Nov 3, 2022 · Big Data

Why Kafka Stores Data the Way It Does: A Deep Dive into Its Log Architecture

This article thoroughly examines Kafka's storage system, explaining why it uses sequential log writes combined with sparse indexing, how different log formats evolved, and the mechanisms for log retention and compaction that enable high‑throughput, fault‑tolerant streaming at massive scale.

Big DataDistributed SystemsKafka

0 likes · 22 min read

Why Kafka Stores Data the Way It Does: A Deep Dive into Its Log Architecture

Qunar Tech Salon

Feb 27, 2018 · Big Data

Root Cause Analysis of Kafka Consumer Group Coordinator Failure and __consumer_offsets Compaction Issues

The article investigates a Kafka cluster outage where several brokers became unavailable and consumers could not join groups, explains the role of __consumer_offsets, analyzes the coordinator selection logic, identifies a stuck loadGroupsForPartition operation and compact thread failure, and documents the recovery steps taken.

Log CompactionOFFSETconsumer-group

0 likes · 11 min read

Root Cause Analysis of Kafka Consumer Group Coordinator Failure and __consumer_offsets Compaction Issues