Horizontal Comparison of HBase, Kudu, and ClickHouse (V2.0)

Introduction

The Hadoop ecosystem contains many storage solutions. HBase is a NoSQL core component offering massive storage and random read/write, while ClickHouse is a column‑oriented OLAP database that provides fast SQL analytics. Apache Kudu, released by Cloudera in 2016, combines random read/write with SQL analytics, complementing HDFS and HBase.

Installation & Dependency Comparison

All three systems require external components. HBase depends on HDFS for storage and ZooKeeper for metadata. Kudu relies on Impala for analytical queries and optionally on a CDH cluster for management. ClickHouse needs ZooKeeper, a log service, and a catalog service for table metadata.

Architecture Comparison

Both HBase and Kudu follow a master‑slave (master‑regionserver / master‑tserver) model, whereas ClickHouse adopts a multi‑master architecture where every server is equal. HBase and ClickHouse also use ZooKeeper for auxiliary metadata, while Kudu’s metadata is managed by its master.

Basic Operations

Read/write flows differ: HBase performs random reads/writes with versioned timestamps; updates are implemented as inserts with newer timestamps. Kudu supports row‑level insert, update, delete with near‑Parquet scan performance. ClickHouse is read‑optimized; it does not support standard UPDATE/DELETE but provides asynchronous MUTATION operations.

Query Capabilities

HBase requires Phoenix for SQL‑like queries and does not support full scans efficiently. Kudu queries are executed via Impala. ClickHouse offers native high‑performance SQL analytics, making it ideal for large‑scale reporting.

Didi Use Cases (HBase)

Didi stores four major data types in HBase: statistical reports, raw fact data (orders, GPS traces), intermediate model data, and backup copies. RowKey designs such as reverse(order_id)+(MAX_LONG‑TS) enable efficient scans for order status, history, and trajectory queries. Geo‑hash based RowKeys support geographic range queries.

ETA Service Architecture

ETA (estimated time of arrival) uses Spark jobs to train models every 30 minutes, reads raw data from HBase, writes results back to HBase, and periodically persists data to HDFS for further feature extraction.

NetEase Kaola Real‑Time Data Warehouse (Kudu)

Kudu provides row‑level CRUD APIs and near‑Parquet scan speed. Real‑time traffic logs are ingested via Spark Streaming using KafkaUtils.createDirectStream, processed into DataFrames, and upserted into Kudu tables. Performance tests show 75 % of batches finish within 1 s.

ClickHouse Log Analysis (Ctrip)

Log data is first formatted to JSON, then ingested into ClickHouse using gohangout. Recommendations include round‑robin writes across servers, batching writes to reduce part count, avoiding distributed tables for bulk inserts, and using daily partitions. Query optimisations (two‑step time‑range selection, materialised views) reduce execution time by up to 1/60.

ClickHouse Operations

Typical operational tasks include new log ingestion, partition cleanup, monitoring via ClickHouse‑exporter + VictoriaMetrics + Grafana, and data migration using ClickHouse‑copier. Common issues such as slow queries, “Too many parts” errors, and startup failures are addressed with query killing, proper partitioning, and filesystem repairs.

Summary

HBase excels at unstructured data storage with strong random read/write. Kudu + Impala is suitable for workloads requiring both low‑latency writes and real‑time analytics. ClickHouse delivers the fastest analytical queries for static data but lacks full update/delete support. Selecting the right technology depends on workload characteristics such as write intensity, query latency, and data structure.