Practices for Data Cleaning and Cutover Consistency in Cross‑Cloud Migration
This article explains the technical details of data cleaning, dirty‑data handling, and three methods—database read‑only, application termination, and network ACL isolation—to ensure data consistency during the data‑regulation and cutover phases of cross‑cloud migration, illustrated with real‑world case studies.
Data Cleaning Phase
During migration testing, real data is used to replay traffic and verify the new platform. After testing, the new platform contains dirty data that must be handled. Two approaches are common: rolling back using MySQL binlog backups or storage change logs, or discarding all dirty data and re‑synchronizing it, which is slower but idempotent.
Ensuring Data Consistency
Three practical methods are discussed:
Database read‑only per user : Create a dedicated sync account with full privileges for data‑sync tools, and a read‑only account for business applications, preventing accidental writes during cutover.
Terminate application processes : Stop all application, OS, and scheduled tasks that could modify data, guaranteeing no external changes during the cutover window.
Network ACL isolation : Configure network ACLs to allow only data‑sync services to reach storage components, blocking all other hosts; this works across all storage types but requires clear subnet segmentation.
Each method has advantages and limitations; they can be combined for stronger protection.
Case Study
In a migration project for an unnamed company, numerous database instances (MySQL, MongoDB, SQL Server, etc.) and storage services were involved. The team applied both application termination and ACL isolation, as the applications lacked a read‑only switch and reconnection logic.
Data Cutover Phase
The cutover window is limited, so data validation should start after the cleaning phase. Validation includes entity‑level checks with sync_diff_inspector and metadata checks (e.g., SELECT MAX(id) ). In a real case, a 500 GB database required over a day for entity validation, while metadata checks completed quickly.
Rollback considerations are essential; if inconsistencies are detected during the cutover window, a rollback to the pre‑cutover state is performed, relying on the clean data ensured in the previous phase.
Summary
The article shares practical experiences and recommendations for maintaining data consistency during cross‑cloud migration, covering dirty‑data handling, read‑only strategies, process termination, ACL isolation, validation timing, and rollback procedures, applicable also to IDC‑to‑cloud migrations.
Architects' Tech Alliance
Sharing project experiences, insights into cutting-edge architectures, focusing on cloud computing, microservices, big data, hyper-convergence, storage, data protection, artificial intelligence, industry practices and solutions.
How this landed with the community
Was this worth your time?
0 Comments
Thoughtful readers leave field notes, pushback, and hard-won operational detail here.