This article explains Paimon's LSM-based storage model, detailing its four-tier architecture (MemTable, Immutable MemTable, L0, L1+), write/read paths, Merge-On-Read with loser-tree K-way merge, four merge engines, compaction triggers and modes, changelog production, and performance tuning practices for lakehouse workloads.
This article explains Apache Hudi's architecture, including its timeline mechanism, file layout, indexing strategies, table types (COW and MOR), query options, storage format versioning, backward compatibility, and key configuration settings for managing data lake tables.
This article provides a detailed technical comparison of Apache Hudi, Delta Lake, and Apache Iceberg, covering core features such as incremental pipelines, concurrency control, merge‑on‑read, partition evolution, multi‑mode indexing, ingestion tools, and real‑world use cases, and explains why Hudi often leads for heavy update workloads.
The article explains how Apache Iceberg v3 replaces the scalable‑limited positional‑delete mechanism in Merge‑on‑Read tables with compact Deletion Vectors, detailing the performance, I/O and metadata drawbacks of positional deletes and showing how the new bitmap‑based approach resolves them.
This article examines the architecture and core design of lakehouse storage systems, compares the metadata handling and Merge‑On‑Read mechanisms of Delta Lake, Apache Hudi, and Apache Iceberg, and presents practical performance‑optimization techniques and real‑world case studies on Alibaba Cloud EMR.
This article explains the fundamentals of data lakes and data warehouses, compares their architectures, outlines the challenges of data lakes, and then dives deep into Delta Lake's core features, storage model, ACID guarantees, concurrency handling, and provides step‑by‑step Spark code examples for practical use.
This article provides a comprehensive overview of Apache Hudi, covering its storage reliance on HDFS, core design principles, table architecture, timeline management, file and index structures, as well as detailed read and write workflows for both Copy‑On‑Write and Merge‑On‑Read table types.
This article explains the evolution from traditional data warehouses to modern lakehouse architectures, introduces the Arctic system’s dynamic hash tree for fast update/delete, describes file splitting with sequence/offset ordering, and compares copy‑on‑write versus merge‑on‑read techniques for achieving low‑latency analytics.
The article explains how Hudi enables efficient record‑level updates in data lakes by adapting database update concepts such as copy‑on‑write and merge‑on‑read, contrasting them with traditional RDBMS and NoSQL storage mechanisms and their trade‑offs.
The article outlines the essential requirements for constructing petabyte‑scale data lakes—such as incremental CDC ingestion, log deduplication, storage management, ACID transactions, fast ETL, and compliance—and explains how Apache Hudi’s COW and Merge‑on‑Read architectures, async compaction, and advanced features address each need.
