Core Features and Architecture of ClickHouse: An In‑Depth Overview

ClickHouse, the column‑oriented OLAP database powering Yandex.Metrica, handles over 200 billion events daily and stores more than 20 trillion rows, delivering 90% of custom queries within one second across a cluster of over 400 servers.

Core Features

Complete DBMS functionality: DDL, DML, fine‑grained permission control, backup/restore, and distributed management.

Columnar storage with LZ4 compression (up to 8:1 ratio in production), reducing I/O and enabling vectorized processing.

Vectorized execution using SIMD (SSE4.2), allowing single‑instruction multiple‑data operations on whole columns.

Full relational model with standard SQL (GROUP BY, ORDER BY, JOIN, IN, case‑sensitive identifiers).

Multiple table engines (MergeTree, TinyLog, etc.) that can be chosen per workload.

Multi‑threading and distributed query processing, with sharding and replication for horizontal scaling.

Multi‑master architecture eliminating single‑point‑of‑failure and simplifying multi‑data‑center deployments.

Online query performance comparable to commercial solutions but at open‑source cost.

Example of column‑based query reduction:

SELECT A1, A2, A3, A4, A5 FROM A

Simple compression illustration:

压缩前：abcdefghi_bcdefghi

压缩后：abcdefghi_(9,8)

Architecture Design

The system is built around several core abstractions:

Column & Field : Columns store whole vectors in memory; Fields represent single scalar values.

DataType : Handles serialization/deserialization for many formats (binary, JSON, CSV, Protobuf).

Block & Block Streams : A Block bundles Columns, DataTypes, and column names; IBlockInputStream reads data, IBlockOutputStream writes data.

Table (IStorage) : Table engines implement DDL, read, and write operations; examples include StorageMergeTree and StorageTinyLog.

Parser & Interpreter : Parsers build ASTs from SQL; Interpreters turn ASTs into executable pipelines.

Functions & Aggregate Functions : Stateless functions operate column‑wise; stateful aggregates (e.g., COUNT) support serialization for distributed execution.

Cluster & Replication : Clusters consist of shards; each shard has replicas. ClickHouse’s design requires one replica per shard, and logical sharding is represented by local tables while distributed tables act as query routers.

Cluster configuration example (custom cluster ch_cluster with one shard and one replica):

Why ClickHouse Is So Fast

Hardware‑first design: group‑by performed in memory using hash tables, careful cache‑level considerations.

Algorithm‑first approach: selecting the fastest algorithm per use‑case (e.g., Volnitsky for constant strings, SIMD‑accelerated brute force for variable strings, re2/hyperscan for regex).

Continuous experimentation: new algorithms are adopted only after rigorous benchmarking.

Special‑case optimizations: different implementations for uniqCombined based on data size (array, hash set, HyperLogLog).

Code generation and loop unrolling to reduce overhead, combined with SIMD for text conversion, filtering, decompression, and JSON handling.

Rapid release cycle and real‑world testing using Yandex’s production data ensure constant improvement.

The combination of these low‑level optimizations, flexible architecture, and relentless testing explains ClickHouse’s superior performance in large‑scale analytical scenarios.

Author: Zhu Kai, ClickHouse contributor and senior architect at YuanGuang Software; source: "ClickHouse Principles and Practice" (Mechanical Industry Press).