Migrating 130 MySQL Instances to Tencent Cloud CDB: Key Lessons and Tips

Test Cases and Process

A large domestic game developer operated more than 130 IDC‑deployed MySQL instances totaling 20 TB. To meet business needs they migrated all instances to Tencent Cloud CDB for MySQL. The team investigated migration workflows, identified four major problems, and prepared solutions before the production migration.

Migration Tool Basics

The migration tool uses a high‑privilege account on the source instance to fetch its configuration and synchronizes that configuration to the target CDB instance. It dumps the source database with mysqldump, transfers the dump to CDB, and imports it. Afterwards a master‑slave relationship is established so that new data continues to replicate. Communication between the source IDC and the CDB instance is performed via NAT through an external server.

Tool Functions and Usage

Migration tasks are created via the console at http://console.qcloud.com/migrate/migrate/cdb following the wizard, then monitored through task logs and CDB backend operations. After the task starts, the console displays proxy traffic, CDB write metrics, and other performance indicators.

Master‑Slave Synchronization

The developer used an external IDC slave to replicate data to CDB. The basic topology is shown below.

MySQL master configuration used in the test:

server_id = 98
log_bin = binlog
binlog_format = ROW
innodb_stats_on_metadata = off

Multi‑Instance Large Binlog Concurrency

After initial tests, the developer planned to run 15 concurrent migration tasks, generating roughly 100 GB of binlog per day. Fifteen MySQL instances were launched on a single server, each mapped to a different port, and each instance established a master‑slave link with a distinct CDB instance.

To create multiple MySQL instances on one host, the built‑in mysqld_multi Perl script was used. The /etc/my.conf file can contain several sections, each defining a separate instance (memory size determines how many can run). Example configuration excerpt:

# /etc/my.conf example
[mysqld1]
port=3307
socket=/tmp/mysql1.sock

[mysqld2]
port=3308
socket=/tmp/mysql2.sock
# … add more sections as needed

Instances are started with mysqld_multi start 1-4. Screenshots show the launched instances.

To generate heavy binlog traffic, a simple update script runs every two hours, producing ~700 MB of binlog per run. Over a day, 15 instances generate about 126 GB of binlog.

# Pseudo‑code for binlog generation
while true; do
  mysql -e "UPDATE big_table SET col=col+1 WHERE id BETWEEN 1 AND 1000000;"
  sleep 7200  # 2 hours
done

The migration console was used to create 15 parallel migration tasks, all of which completed successfully according to both console and backend logs.

A verification script periodically compared data between the IDC source and the CDB target to ensure consistency. Logs confirmed that all updates were applied correctly.

Test Data Records

The internal test was followed by three production migrations. Results showed stable bandwidth usage (≈500 Mbps limited by the developer’s outbound link) and no data‑loss incidents. All 130+ MySQL instances have been successfully migrated and are now in production.

Encountered Problems and Solutions

Initial master‑slave connection failure Symptom: “Can't connect to MySQL server on 10.*.*.*”. Cause: Timestamp mismatch between the CDB proxy (NAT) and the IDC server, combined with connection reuse on the IDC side, caused packets to be dropped. Solution: Disable TCP timestamps and TIME_WAIT recycling on the IDC server:

net.ipv4.tcp_timestamps = 0
net.ipv4.tcp_tw_recycle = 0

Stored‑procedure migration failure across versions Symptom: ERROR: Can't load from mysql.proc – table probably corrupted. Cause: Schema differences in the proc table between source MySQL version and CDB. Solution: Use the CDB version that skips the proc table during migration.

Binlog import failure due to new database creation Symptom: errno:1049:Error 'Unknown database 'xxxx' on query. Cause: The migration was set to transfer a specific database, but a new database was created during the process and its binlog was captured, causing a mismatch. Solution: Avoid any DDL operations (such as creating new databases) while a migration task is running.

Conclusion

Thorough pre‑migration testing—covering functionality, performance, and edge‑case scenarios—prevented data inconsistency and operational failures during the actual cut‑over. To date, all 130+ MySQL instances have been migrated smoothly, providing a reliable reference for large‑scale database migrations to Tencent Cloud CDB.