Alibaba Cloud BDS Service for Non‑Stop HBase Cluster Migration

Background Continuous, non‑stop migration of HBase clusters is a common demand, but existing open‑source tools and native HBase solutions often suffer from performance, stability, and usability issues. Alibaba Cloud therefore offers the BDS migration service to handle TB‑scale data migrations without downtime.

Supported Scenarios

HBase major version upgrades (e.g., 1.x to 2.x)

Cluster hardware upgrades (e.g., 8‑core/16 GB to 16‑core/32 GB)

Network environment changes (classic network to VPC)

Cross‑region or cross‑data‑center migrations

HBase business splitting

Solution Overview

Open‑source approaches typically involve three parts: schema migration, real‑time data sync, and full‑data migration. They lack automated schema tools, require manual table creation (risking partition mismatches), and rely on client‑side dual writes or HBase Replication, which can introduce latency, hotspot issues, and heavy load on RegionServers.

Full‑data migration with tools like DataX, CopyTable, or snapshot export either consumes excessive API traffic or impacts source cluster stability due to snapshot creation and compaction.

Alibaba Cloud BDS solution addresses these gaps by providing automated schema creation with consistent partitioning, file‑level full‑data transfer that saves over 50% of network traffic, automatic Region Split and HFile compaction handling, and scalable worker nodes that can achieve 100 MB/s per node, supporting TB‑to‑PB scale migrations.

Stability Guarantees

Distributed architecture with ZooKeeper‑monitored master/worker failover.

Comprehensive monitoring, alerting, and checkpoint‑based task resumption.

Decoupled from HBase clusters to avoid CPU/memory contention.

Full‑data and incremental sync access only HDFS, leaving source HBase untouched.

Migration Steps

Purchase BDS service and prepare network connectivity.

Submit migration tasks via the BDS console: configure source/target clusters, enable incremental sync, and launch historical data migration.

Validate business functionality and perform gray‑release testing.

Switch traffic to the new cluster during low‑traffic windows, with optional reverse sync to ensure data completeness.

After stable operation, terminate BDS resources.

Case Study

A migration from a self‑managed HBase 1.x cluster (20 RegionServers, >30 TB data, hundreds of GB daily growth) to Alibaba Cloud HBase was performed within the same VPC. BDS worker nodes (4 cores / 8 GB) were sized to handle five RegionServer logs per node, requiring at least four workers to sustain the workload.

Task Configuration

After completing prerequisite setup, users create a real‑time sync channel, select source and target clusters, batch‑submit tables, and let BDS auto‑create missing target tables with matching partitions. Then they submit a full‑data migration job, monitor progress, and finally validate and cut over traffic.

The article concludes with a reminder to like, share, and collect the post.