Mastering Multi-Cloud Data Migration: MySQL, File & Object Storage Strategies

Introduction

As internet services grow, multi‑cloud deployment and cross‑cloud migration become essential, but data migration and synchronization remain the biggest pain points for operations and development teams.

Common Challenges

Data integrity and consistency

Limited migration window

Application dependencies and call relationships

Resource Categories in Cross‑Cloud Migration

Network services such as EIP, VPC, load balancers, and NAT gateways are stateless and can be aligned manually.

Compute instances are migrated with UCloud Server Migration Tool (USMC), keeping IP consistency and supporting minute‑level incremental sync.

Storage services (databases, file storage, object storage) are migrated with the UDTS data‑transfer tool – the focus of this article.

Migration Phases

The process is divided into three stages: data‑sync, data‑clean‑up, and cut‑over. The data‑sync stage involves copying data to the new platform and testing applications with real data.

MySQL Synchronization

Consistency is achieved when both full and incremental data are identical. Two common approaches:

MySQL master‑slave using mysqldump to capture binlog position, export full data, restore a standby, then replay incremental binlog.

UDTS tool, which also separates full and incremental phases, relying on binlog for incremental sync and verifying consistency after migration.

Case Study: A company with dozens of MySQL instances switched from manual mysqldump to UDTS, reducing DBA workload and ensuring data consistency.

Data Validation for MySQL

Tools like pt-table-checksum create a temporary checksum table and compare results between master and replica, but require a master‑slave relationship and indexed tables.

When that relationship is broken, sync_diff_inspector (TiDB team) can compare arbitrary source and target databases without requiring replication or indexes, pinpointing differences via binary search.

Both tools can be time‑consuming for large datasets (e.g., 500 GB takes ~28 hours), so simple row counts are often used during cut‑over windows.

File Storage Synchronization

File migration typically uses rsync‑like copying and MD5 checksum verification. Detecting changes is challenging, especially with NFS where inotify does not work across hosts.

Business logs are used to record changed files; during cut‑over, only those files are incrementally synced.

FastDFS provides a binlog‑like mechanism to track file operations (create, append, delete) across replicas.

File Validation

To detect silent disk errors, full‑link checksum verification is required: generate checksums before upload, store them with the file, periodically recompute and compare, and verify on download.

For large file sets, use recursive md5sum commands:

find ./ -type f -print0 | xargs -0 md5sum > ./my.md5

md5sum -c ./my.md5

Object Storage Synchronization

Object storage sync sits between database and file storage in complexity. US3SYNC migrates S3‑compatible buckets to UCloud US3, optionally using Redis to track progress and support resumable transfers.

Initial sync should avoid enabling US3 as the front‑end endpoint to prevent massive cache‑misses and latency spikes.

US3SYNC can operate in comparison mode, using HEAD requests to sync only missing or size‑mismatched objects, reducing memory pressure when dealing with billions of keys.

Object Storage Validation

ETag headers (often MD5 of the object) are used for integrity checks. US3SYNC calculates the expected ETag before upload and compares it with the returned value to confirm successful transfer.

Conclusion

Multi‑cloud deployment is now a trend. UCloud’s USMC and UDTS tools help customers migrate massive data within limited windows while preserving integrity and consistency. This article covered MySQL, file storage, and object storage sync; future articles will address data‑clean‑up and cut‑over phases.