Jan 14, 2021 · Big Data

Design and Implementation of an SSD‑Based Application‑Layer Cache Architecture for Kafka in Meituan Data Platform

Meituan built an SSD‑based application‑layer cache for Kafka that bypasses PageCache contention between real‑time and delayed jobs, classifies log segments across SSD and HDD, limits flush rates, and achieves up to 80% latency reduction while guaranteeing stable real‑time consumption.

Big DataKafkaLogSegment

0 likes · 19 min read

Design and Implementation of an SSD‑Based Application‑Layer Cache Architecture for Kafka in Meituan Data Platform

ITFLY8 Architecture Home

Nov 29, 2017 · Backend Development

How Kafka Stores Messages: Partitions, Segments, and Sparse Indexes Explained

This article explains Kafka's internal message storage mechanism, detailing how topics are divided into partitions, how partitions are segmented into LogSegments with data and index files, and how sparse indexing enables efficient offset lookups.

KafkaLogSegmentMessage Storage

0 likes · 9 min read

How Kafka Stores Messages: Partitions, Segments, and Sparse Indexes Explained