Lakehouse Research Base

Oct 18, 2026 · Big Data

Migrating Core Real-Time Indexing from Kafka to Apache Fluss: Xiaohongshu's Engineering Journey

This article details Xiaohongshu's migration of its core real-time indexing pipeline from Kafka to Apache Fluss, covering architectural drivers like read amplification and historical replay contention, Fluss's columnar storage and Lakestream integration with Paimon, and a phased migration approach including baseline testing, shadow dual-run, and gradual cutover.