How ClickHouse Achieves Billion‑Row Queries in Seconds: Architecture & Cloud Deployment

With the rise of big data and IoT, enterprises need databases that can handle massive data volumes efficiently. ClickHouse, developed by Yandex, stands out for delivering sub‑second query responses on billions of rows, making it popular among major tech companies.

OLTP vs OLAP

Traditional databases like Oracle and MySQL are OLTP (On‑Line Transaction Processing) systems focused on low latency, data integrity, and 24/7 availability for transactional workloads. OLAP (On‑Line Analytical Processing) databases, such as ClickHouse, store massive, rarely‑updated datasets for multidimensional analytical queries, prioritizing high query throughput over transaction speed.

Key Feature: Columnar Storage

ClickHouse uses a column‑oriented storage engine. Unlike row‑based storage that reads entire rows even when only a few columns are needed, columnar storage groups each column into separate files, allowing queries to read only the relevant data and dramatically reducing I/O.

B+Tree vs MergeTree

Traditional InnoDB stores data in a B+Tree structure, where each node holds primary key values and data pages are accessed via leaf nodes. ClickHouse replaces B+Tree with the MergeTree engine, which organizes data into parts and granules, enabling efficient bulk inserts and fast reads.

MergeTree Storage Process

Data is partitioned (e.g., by birth date) and each column is written to its own file. ClickHouse creates a Data part for each batch, assigning a min and max block number. An asynchronous background task merges small parts within the same partition, while a .mrk file records offsets for fast lookup.

Why JD Cloud Uses Kubernetes for ClickHouse

Kubernetes abstracts underlying infrastructure differences, letting developers focus on database logic.

The platform can be deployed unchanged on public clouds, private clouds, or hybrid environments.

Deployment Process on JD Cloud

JD Cloud extends the open‑source ClickHouse Operator, adding custom APIs, pod‑affinity rules, and security labels to avoid placing primary and replica pods on the same physical node. Helm charts submit configuration forms to Kubernetes, which then creates ClickHouse instances, Zookeeper clusters, Prometheus monitoring, and Grafana dashboards.

Monitoring, Scaling, and High Availability

Pods run on up to 64‑core CPUs with 512 GB memory, providing high performance.

Kubernetes automatically replaces failed pods and re‑attaches persistent volumes via StatefulSets.

Hot‑scaling of CPU, memory, and storage is supported without downtime.

Integrated Prometheus and Grafana give real‑time visibility into pod resource usage and enable multi‑dimensional alerts via email, SMS, or WeChat.

ClickHouse’s powerful SQL‑compatible engine, columnar storage, and parallel processing enable enterprises to unlock greater value from massive data workloads, provided they adapt their data‑warehouse design and pipelines to fully exploit these capabilities.