Big Data Technology Architecture

Jun 29, 2020 · Fundamentals

Kafka Storage Mechanism and Reliability Guarantees

This article explains Kafka's internal storage architecture—including topics, partitions, segments, .log and .index files—how data is read, and the various reliability mechanisms such as ISR/OSR, LEO/HW, producer acknowledgment levels, leader election strategies, and delivery semantics.