BestHub
Sign in
Lakehouse Research Base
Author

Lakehouse Research Base

Focused on technical sharing in the data field, covering a tech stack that includes Hadoop, Spark, Flink, Kafka, Fluss, Paimon, Iceberg, StarRocks, ClickHouse, ES, Milvus, and more. Welcome to follow.

90
Articles
0
Likes
310
Views
0
Comments
Recent Articles

Latest from Lakehouse Research Base

90 recent articles
Lakehouse Research Base
Lakehouse Research Base
Sep 29, 2026 · Big Data

Inside Flink Checkpoint: Barrier Alignment, Unaligned Snapshots & RocksDB Recovery

This article provides a deep technical analysis of Apache Flink's checkpoint mechanism, covering barrier alignment and unaligned snapshots, heap vs RocksDB state backends with incremental checkpoints, savepoint vs checkpoint differences, failure recovery flows, and practical troubleshooting for checkpoint timeouts, state bloat, and recovery slowness.

Apache FlinkBarrier AlignmentCheckpoint
0 likes · 26 min read
Inside Flink Checkpoint: Barrier Alignment, Unaligned Snapshots & RocksDB Recovery
Lakehouse Research Base
Lakehouse Research Base
Sep 28, 2026 · Big Data

Apache Paimon Internals: LSM Write Path, Compaction Scheduling & Changelog Production

This article deep-dives into Apache Paimon's kernel source code, explaining the LSM write path from memory buffer to Sorted Runs, compaction scheduling trade-offs, four changelog production modes, and two-phase commit with Flink checkpoints, providing troubleshooting guidance for compaction backlog, changelog latency, and snapshot bloat.

Apache PaimonCompactionFlink
0 likes · 33 min read
Apache Paimon Internals: LSM Write Path, Compaction Scheduling & Changelog Production
Lakehouse Research Base
Lakehouse Research Base
Sep 26, 2026 · Big Data

Real-Time Lakehouse Implementation Methodology: From Requirements to Production

This article presents a comprehensive methodology for implementing real-time lakehouse architectures, covering requirements quantification, architecture selection, capacity planning, canary deployment, rollback strategies, and SLA decomposition with a six-stage pipeline, e-commerce case study, troubleshooting guides, and interview frameworks.

Apache FlinkApache PaimonReal-time Lakehouse
0 likes · 31 min read
Real-Time Lakehouse Implementation Methodology: From Requirements to Production
Lakehouse Research Base
Lakehouse Research Base
Sep 23, 2026 · Databases

Milvus 3.x: From Vector Database to Lake-Native AI Retrieval Engine

This article provides a comprehensive analysis of Milvus 3.x's evolution from a vector database to a Lake-Native retrieval engine, covering its architecture, External Collection for zero-copy indexing on data lakes, Storage V3, ANN indexes (HNSW, IVF, DiskANN), hybrid search, and production RAG architectures with performance tuning and troubleshooting guidance.

ANN IndexDiskANNHNSW
0 likes · 37 min read
Milvus 3.x: From Vector Database to Lake-Native AI Retrieval Engine
Lakehouse Research Base
Lakehouse Research Base
Sep 10, 2026 · Databases

Apache Fluss Virtual Tables: $changelog vs $binlog for Queryable Change Streams

This article deep-dives into Apache Fluss virtual tables, explaining how $changelog and $binlog suffixes expose change logs as read-only queryable tables, detailing their schemas, use cases, differences between PrimaryKey and Log tables, startup modes, limitations, and the design philosophy enabling native CDC consumption via SQL without external tools.

Apache FlussCDCChange Log
0 likes · 9 min read
Apache Fluss Virtual Tables: $changelog vs $binlog for Queryable Change Streams
Lakehouse Research Base
Lakehouse Research Base
Sep 6, 2026 · Big Data

StarRocks Query Acceleration on Paimon: Production Tuning & Best Practices

This article details production practices for accelerating StarRocks queries on Paimon external tables, covering architecture, table design (partitioning, bucketing, compaction), dirty data handling, catalog configuration, predicate pushdown verification, SQL optimization with partition/bucket pruning, query method selection (direct, async materialized views, hot/cold tiering), and BE-level tuning.

Bucket PruningCompactionLakehouse
0 likes · 22 min read
StarRocks Query Acceleration on Paimon: Production Tuning & Best Practices
Lakehouse Research Base
Lakehouse Research Base
Aug 31, 2026 · Big Data

Factory Lines & Data Pipelines: The Shared Orchestration DNA

This article reveals the deep structural isomorphism between manufacturing assembly lines and data pipeline orchestration, mapping five shared design philosophies, a 15-dimension logical correspondence, bidirectional lessons, and six practical mental models for data engineers to adopt factory-floor thinking.

Data EngineeringIdempotencyassembly line
0 likes · 22 min read
Factory Lines & Data Pipelines: The Shared Orchestration DNA