An Introduction to HBase: Architecture, Data Model, Storage Engine, Indexing, Features, and Use Cases

1. Storage Engine

HBase is the open‑source implementation of Google’s BigTable and uses an LSM‑Tree based storage engine. Writes are first recorded in a WAL log, then placed in an in‑memory MemStore; when the MemStore reaches a threshold it flushes to disk, creating new HFile files. Over time, many HFiles accumulate, so HBase periodically runs compaction to merge them and improve read performance. The read path leverages a BlockCache, MemStore, and HFiles, along with Bloom filters and indexes, to achieve high performance.

2. Data Model

HBase’s data model resembles a relational model with namespaces, tables, rows, columns, column families, qualifiers, cells, and timestamps, but data is stored as ordered key‑value pairs. The key consists of rowkey, column‑family:qualifier, timestamp, and type (Put/Delete). Rows are sparse, and null columns consume no storage.

3. Column‑Family Storage

HBase is a column‑family‑oriented store: data within the same column family is stored sequentially, giving it row‑store characteristics, while a single‑column family behaves like a column store. Thus HBase is best described as a column‑family storage system.

4. Indexing

By default HBase only provides a single‑column index on the rowkey, enabling efficient point lookups and range scans. Non‑rowkey queries are slower unless a secondary index is built, commonly via Phoenix or custom coprocessors.

For secondary indexing, Phoenix is a mature solution that adds SQL support and secondary indexes to HBase.

5. Main Features

Advantages:

Massive capacity: a single table can be extremely large, suitable for permanent storage of huge datasets.

High performance: LSM‑Tree design yields strong write throughput and millisecond‑level read latency.

High reliability: WAL logging and multi‑replica storage ensure data safety.

Native Hadoop integration: stores data on HDFS and works with MapReduce for offline processing.

Schema‑free: columns are added dynamically at write time.

Sparse storage: null columns occupy no space.

Multi‑version: each cell stores a timestamp, enabling versioned reads.

Disadvantages:

Weak analytical capabilities: lacks built‑in aggregation, multi‑dimensional queries, or joins; external tools like Phoenix or Spark are needed.

No native secondary index: only rowkey indexing is provided.

No native SQL support: requires a layer such as Phoenix for SQL queries.

6. Application Scenarios

HBase is commonly used for order/message storage, user profiling, recommendation feeds, social streams, security risk control, and IoT time‑series data. It is ideal when you need to store massive data with high concurrent reads/writes and do not require complex analytical queries.

If your scenario demands large‑scale storage with high throughput and modest analysis, HBase is a strong candidate.