Mastering HBase RowKey Design: Principles, Use Cases, and Architecture

First answer: why use HBase? As business grows and data volume increases, MySQL faces issues:

MySQL supports TB‑level data and cannot retain all historical data, while HBase handles PB‑level data, suitable for long‑term cold data.

Adding columns in MySQL is costly and time‑consuming as data grows; HBase allows arbitrary column addition, empty columns consume no space, enabling flexible data models.

The most critical aspect of using HBase is rowkey design; a poor design incurs high future modification costs.

HBase RowKey Design Principles

Key principles for designing HBase rowkeys include:

Composite key : concatenate multiple business fields; queries must include those fields as part of the rowkey.

Field order : for one‑to‑many relationships, place the “one” side first (e.g., userId:orderId) to enable efficient scans.

Business field length alignment : because rowkeys are sorted lexicographically, pad fields to a fixed length (e.g., 12‑digit IDs padded with leading zeros) to maintain expected ordering.

Salting to avoid hotspots : sequential IDs can cause read/write hotspots; prepend a prefix such as a hash of the business ID modulo the number of regions to distribute load.

HBase Application Examples

Cold‑Hot Data Separation

HBase is suitable for cold data storage, handling massive historical records.

MySQL serves as hot storage, supporting read/write and transactional operations.

Archive infrequently updated historical data to HBase and delete corresponding MySQL rows.

Transaction Logs

Transaction logs can add fields at any time.

Ideal for storing massive log records.

Brief Review of HBase Architecture

Region : rows are ordered by rowkey; a region is a shard that resides on a single region server.

Region Server : hosts one or more regions and uses HDFS client APIs for read/write.

WAL : Write‑Ahead Log; data is written to WAL before memstore, providing recovery safety.

Store : each column family maps to a store; a store contains a memstore and multiple HFiles. Limit column families to improve performance.

Memstore : after WAL, data is kept in memory for sorting before flushing to HFile.

HFile : the persistent storage file; memstore flushes to HFile when full.

Region auto‑splits and merges when size thresholds are reached.

Compaction : after deletions, HFiles are merged to reduce file count and improve lookup efficiency.