In‑Depth Exploration of OceanBase Hierarchical Dump and Compaction Mechanisms

The author, a test engineer at iKangSheng responsible for DMP testing, presents an original investigation into OceanBase's hierarchical dump (compaction) mechanisms, focusing on the relationship between Mini and Minor compactions and the configuration parameters that control them.

1. LSM‑Tree Overview

LSM‑Tree (Log‑Structured Merge Tree) is widely used in modern databases such as TiDB, Cassandra, and OceanBase because it enables sequential writes and high write throughput.

It consists of two main components:

MemTable : an in‑memory KV lookup tree combined with an unordered WAL file.

SSTable (Sorted String Table) : a set of immutable on‑disk files that store ordered key‑value pairs.

Write Flow

1) Synchronously write MemTable : data is first appended to the WAL and then applied to an in‑memory AVL tree, resulting in at most one disk I/O per write.

2) Asynchronously write SSTable : when an Immutable MemTable accumulates, a background merge thread flushes it to disk as a Level‑0 SSTable; subsequent levels trigger compaction when file counts exceed thresholds.

Compaction Strategies

Common strategies include Classic Leveled, Tiered, Tiered & Leveled, and FIFO. The first three are described briefly:

Classic Leveled : each level contains non‑overlapping sorted runs; compaction moves data from level L n to L n+1 , causing significant write amplification.

Tiered : files within a level may overlap; multiple sorted runs per level reduce write amplification but increase read amplification and space overhead.

Tiered & Leveled : small, recent levels use Tiered to limit write amplification, while larger, older levels use Leveled to limit space amplification.

2. OceanBase Hierarchical Dump

OceanBase’s storage engine is built on an LSM‑Tree and divides SSTables into three layers, employing a Tiered & Leveled compaction strategy: Tiered for L0, Leveled for L1 and L2.

Compaction types in OceanBase:

Mini Compaction (Mini Merge) – a Tiered compaction that converts a frozen MemTable into a Mini SSTable; essentially a single dump.

Minor Compaction – merges multiple Mini SSTables to reduce read amplification; can be either L0 → L0 (Mini Minor Merge) or L0 → L1 (Minor Merge).

Major Compaction – large‑scale merges (not covered in this article).

Experiment 1: Observing L0/L1 Dump Timing Under Continuous Writes

Setup (OceanBase Community 4.0.0.0, tenant ob_bench , 2 GB memory) with key parameters:

memstore_limit_percentage = 50
freeze_trigger_percentage = 20
minor_compact_trigger = 2
_minor_compaction_amplification_factor = 25
major_compact_trigger = 9999  // disable major compaction

Steps:

Create a sysbench database and a table sbtest1 with 1 000 000 rows.

Insert data continuously for 10 minutes while monitoring GV$OB_TABLET_COMPACTION_HISTORY and DBA_OB_TABLE_LOCATIONS .

Results showed 4 × MINI_MERGE and 1 × MINI_MINOR_MERGE during the initial data load, followed by additional Mini and Minor compactions as the write workload continued.

Experiment 2: Does alter system minor freeze Trigger a L1‑level Minor Merge?

Using the same configuration as Experiment 1 (minor_compact_trigger = 2):

Update a row in sbtest1 to ensure the MemTable contains changes.

Execute alter system minor freeze; repeatedly and observe the compaction history.

Findings:

The first minor freeze produced only a MINI_MERGE (no L1 merge).

Thus, minor freeze always creates a Mini SSTable; whether it subsequently triggers a Mini‑Minor or Minor merge depends on minor_compact_trigger and _minor_compaction_amplification_factor .

When minor_compact_trigger was set to 0, every MemTable flush immediately caused a Mini SSTable to be pushed to L1, confirming the official documentation’s description.

Test Summary

The experiments confirm that alter system minor freeze performs a MINI_MERGE, and any further compaction is governed by the two configuration parameters mentioned above; the command name is somewhat misleading because it does not directly target L1.

References

https://www.cnblogs.com/buttercup/p/12991585.html

https://www.modb.pro/db/608302

https://www.oceanbase.com/docs/community-developer-advance-0000000000634015

https://www.modb.pro/db/610025

https://ask.oceanbase.com/t/topic/31400011

About SQLE (Promotional Note)

SQLE is an open‑source SQL audit tool from the iKangSheng community, supporting MySQL and extensible to other databases. Repository: https://github.com/actiontech/sqle .