Apache Hudi from Zero to One: Introduction to Hudi’s Storage Format (Part 1)

Apache Hudi is a transactional data‑lake platform that brings database and data‑warehouse capabilities to object storage. This article introduces Hudi’s storage format, describing the table layout, metadata files, and data files.

Hudi stores table metadata under <base_path>/.hoodie/ , including hoodie.properties and Timeline files that record actions such as commit and deltacommit . The naming convention for Timeline files is <action_timestamp>.<action_type>[.<action_state>] , where the state can be requested , inflight , or completed .

# an example of deltacommit actions on Timeline
20230827233828740.deltacommit.requested
20230827233828740.deltacommit.inflight
20230827233828740.deltacommit

<action timestamp>.<action type>[.<action state>]

Data files are divided into Base Files (columnar, e.g., Apache Parquet) that store the main records and Log Files (row‑oriented, e.g., Apache Avro) that capture incremental changes. A Base File together with its associated Log Files forms a File Slice; multiple slices compose a File Group, enabling efficient reads and writes.

Hudi supports two table types: Copy‑on‑Write (CoW) and Merge‑on‑Read (MoR). CoW rewrites entire files on each write, offering low‑latency reads at the cost of higher write amplification, while MoR appends changes to Log Files, reducing write amplification but requiring merge‑on‑read for the latest view.

The article concludes with a brief recap and points to future posts that will cover other aspects of Hudi, such as compaction, cleaning, and query modes.