How Apache Doris 2.0 Cuts Storage Costs with Hot‑Cold Data Tiering

In real‑world data analysis scenarios, hot and cold data have different query frequencies and latency requirements, leading to high storage costs when all data is kept on local disks. To address this, Apache Doris implements hot‑cold data tiering, storing hot data on high‑performance SSDs and cold data on lower‑cost HDDs or object storage.

Supported Scenarios

Long‑term data retention increases storage expenses.

Hot data requires fast response; cold data tolerates slower access.

Backup and recovery of massive data consume significant resources.

Dynamic Partition Lifecycle Management (Doris 0.12+)

Doris 0.12 adds dynamic partitioning, allowing users to set storage_cooldown_time or dynamic_partition.hot_partition_num to control when data moves from SSD to HDD.

Cold‑Hot Tiering in Doris 2.0

Doris 2.0 extends tiering to object storage, reducing cold‑data storage to a single replica and cutting costs to roughly one‑third of HDD storage. The three‑level storage hierarchy (SSD → HDD → object storage) can be configured via table properties.

Assuming cloud‑disk prices are 5‑10× higher than object storage, moving 80% of cold data to object storage can lower storage costs by at least 70%.

Example cost calculation for 100 TB of data:

OSS standard: 120 CNY / TB / month

普通云盘: 300 CNY / TB / month

SSD 云盘: 1000 CNY / TB / month

With 80% cold data stored in OSS, total cost drops from 30 000 CNY to 15 600 CNY (48% saving). Using SSDs for hot data, savings exceed 70%.

Step‑by‑Step Usage

1. Create a Resource that points to an object‑storage bucket (AWS, Azure, Alibaba Cloud, Huawei Cloud, Tencent Cloud, Baidu Cloud, etc.).

CREATE RESOURCE IF NOT EXISTS "${resource_name}" 
PROPERTIES(
  "type"="s3",
  "s3.endpoint"="${S3Endpoint}",
  "s3.region"="${S3Region}",
  "s3.root.path"="path/to/root",
  "s3.access_key"="${S3AK}",
  "s3.secret_key"="${S3SK}",
  "s3.connection.maximum"="50",
  "s3.connection.request.timeout"="3000",
  "s3.connection.timeout"="1000",
  "s3.bucket"="${S3BucketName}"
);

2. Create a Storage Policy to define the cooldown TTL or absolute datetime.

CREATE STORAGE POLICY testPolicy 
PROPERTIES(
  "storage_resource"="remote_s3",
  "cooldown_ttl"="1d"
);

Or with an absolute timestamp:

CREATE STORAGE POLICY testPolicyForTTLDateTime 
PROPERTIES(
  "storage_resource"="remote_s3",
  "cooldown_datetime"="2023-06-07 21:00:00"
);

3. Assign the Storage Policy to a table or a specific partition.

CREATE TABLE IF NOT EXISTS lineitem1 (
  L_ORDERKEY INT NOT NULL,
  L_PARTKEY INT NOT NULL,
  L_SUPPKEY INT NOT NULL,
  L_LINENUMBER INT NOT NULL,
  L_QUANTITY DECIMAL(15,2) NOT NULL,
  L_EXTENDEDPRICE DECIMAL(15,2) NOT NULL,
  L_DISCOUNT DECIMAL(15,2) NOT NULL,
  L_TAX DECIMAL(15,2) NOT NULL,
  L_RETURNFLAG CHAR(1) NOT NULL,
  L_LINESTATUS CHAR(1) NOT NULL,
  L_SHIPDATE DATEV2 NOT NULL,
  L_COMMITDATE DATEV2 NOT NULL,
  L_RECEIPTDATE DATEV2 NOT NULL,
  L_SHIPINSTRUCT CHAR(25) NOT NULL,
  L_SHIPMODE CHAR(10) NOT NULL,
  L_COMMENT VARCHAR(44) NOT NULL
)
DUPLICATE KEY(L_ORDERKEY, L_PARTKEY, L_SUPPKEY, L_LINENUMBER)
PARTITION BY RANGE(`L_SHIPDATE`)(
  PARTITION p202301 VALUES LESS THAN ("2017-02-01"),
  PARTITION p202302 VALUES LESS THAN ("2017-03-01")
)
DISTRIBUTED BY HASH(L_ORDERKEY) BUCKETS 3
PROPERTIES(
  "replication_num"="3",
  "storage_policy"="${policy_name}"
);

4. Verify the tiering using show tablets or ADMIN SHOW REPLICA STATUS FROM TABLE PARTITION(...). Tablets with non‑‑1 CooldownReplicaId and a populated CooldownMetaId are using the cold storage policy.

TabletId: 2749703
BackendId: 10090
LocalDataSize: 73001235
RemoteDataSize: 0   // before cooldown
...
LocalDataSize: 0
RemoteDataSize: 73001235   // after cooldown

5. Querying a table that has cold data triggers Doris to download the required remote rowsets to a local block cache; subsequent queries hit the cache, resulting in query latency comparable to hot‑only tables.

Performance Evaluation

A benchmark using the SSB SF100 dataset on a 3‑node (1 FE + 3 BE) cluster shows that after warm‑up, the hot‑cold tiered table takes 5.799 s versus 5.822 s for a non‑tiered table, confirming negligible performance impact.

Storage Optimization Details

In earlier Doris versions, cold data still kept multiple replicas on local disks. Doris 2.0 stores a single replica of cold data in object storage while keeping metadata and hot data locally, allowing all local replicas to share the same cold data and drastically reducing storage consumption.

Cold Data Compaction

Cold data can be compacted on object storage via the ColdDataCompaction process, which merges fragmented cold rowsets into larger parts, further saving space and reducing I/O.

Cold Data Cache

When a query accesses cold data, Doris downloads the needed blocks to a local block cache (LRU‑based). The cache granularity is block‑level, ensuring that cached cold data performs as fast as hot data after the first access.