Automating Data Balancing for ClickHouse Clusters on Tencent Cloud

ClickHouse is a column‑oriented DBMS for online analytical processing (OLAP). Open‑sourced in 2016 under the Apache 2.0 license, it is popular among big‑data engineers for its excellent query performance.

In April 2020, Tencent Cloud launched a managed ClickHouse service. Since its release, the service has quickly gained internal and external customers, leading to rapid growth in the number of workloads. Consequently, operational and control pressure increased, and users began demanding elastic scaling capabilities.

ClickHouse follows a classic Share‑Nothing architecture, which naturally supports elastic scaling. Adding nodes or increasing the number of shard replicas is straightforward.

However, after scaling out a ClickHouse cluster, data does not automatically rebalance across the new nodes; manual intervention is required to ensure data distribution. The same applies when removing nodes – data must be migrated off the node before decommissioning.

In production environments, the operational workload grows sharply with the number of tables and data volume. Automating data balancing for ClickHouse clusters on the cloud is therefore highly valuable.

The article explains how Tencent Cloud ClickHouse implements an unattended data‑balancing service.

1. Missing Data‑Balancing Feature in ClickHouse Clusters

In production, ClickHouse is typically deployed in cluster mode. Users divide cluster nodes into logical groups (clusters) that store datasets. Queries are performed via Distributed Engine tables. A cluster defines the number of shards and replicas per shard. For example, a cluster named cluster‑dataset may have 4 shards, each with 2 replicas, distributing data across 4 shards with 2 copies each.

Adding a new shard is easy—just allocate machines and modify the configuration. Yet existing data remains on the original shards, causing resource waste.

To address this, several manual solutions exist:

Delete all data and re‑import from a backup source.

Increase the weight of new nodes, let the system rebalance, then readjust weights.

Manually move data to the new nodes.

All these approaches have drawbacks: they may be infeasible without a backup, incur long downtime, require repeated permission changes, or be error‑prone for large tables.

2. Cloud‑Based ClickHouse Solution

Tencent Cloud ClickHouse provides an automatic data‑balancing feature. After obtaining user authorization, administrators configure a bandwidth limit for data migration in the console and start a balancing task.

The backend control system evaluates each machine’s available disk capacity, creates a migration plan, and executes it respecting the bandwidth limit, ultimately achieving a balanced data distribution.

Example workflow:

Select the target cluster and the ClickHouse instance, then choose the “Data Migration” menu and pick the “Data Balancing” mode.

Select source and destination nodes for migration.

Choose the tables to be migrated.

Confirm the migration details and submit the task.

During migration, the selected tables are unavailable to business queries. After the task completes, the data distribution across nodes is balanced, as shown by the post‑migration statistics.

3. Conclusion

The cloud data‑migration feature solves the elastic‑scaling data‑movement problem for ClickHouse. Typical use cases include:

After expanding a cluster, migrate part of the data to new nodes to achieve balance.

Before shrinking a cluster, migrate data off the node to be decommissioned to avoid data loss.

This functionality greatly reduces operational pressure for ClickHouse clusters, though it is important to note that tables under migration cannot be accessed by business workloads.