How to Seamlessly Migrate a Legacy MongoDB System to JimKV, MySQL, and ES

Current Situation

This article introduces a complete migration plan for a legacy system whose core data storage relies on MongoDB in a 1‑primary‑1‑replica configuration, serving seven applications with one database and two tables.

Migration Plan

2.1 Migration Rhythm

The overall rhythm includes:

Identify the scope: both MongoDB and MySQL data sources exist, and all business functions using MongoDB must be listed.

Determine the target storage medium and standards to cover all business requirements, ensuring read/write performance.

Refactor the DAO layer of the existing data structures.

Implement dual‑write and data migration.

Perform R2 traffic verification, test regression, and data comparison.

Gradually increase traffic.

2.2 Code Refactor / Data Heterogeneity

Adopt the decorator pattern to uniformly control dual‑write logic (primary and secondary), traffic switching, and offline logic. Extract existing direct MongoDB API calls into the DAO layer without altering business logic, facilitating later traffic‑switch adaptation.

Data source selection criteria (summarized from the comparison table):

JimKV offers multi‑storage engines, Raft‑based strong consistency, multi‑datacenter disaster recovery, and robust monitoring and scaling, but its write performance is lower due to Raft. HBase provides petabyte‑scale storage, cloud‑native architecture, and high write throughput, yet its failover time is longer (1‑2 minutes). Based on these factors, JimKV, MySQL, and Elasticsearch are chosen as replacements for MongoDB.

2.3 Existing Data Migration

Feasibility assessment:

Big‑data extraction task – feasible and easy.

Asynchronous code task – feasible and easy.

DRC synchronization – not supported from MongoDB to other databases.

Given a moderate data volume (approximately 300 w rows per table), a code‑driven approach provides flexibility to adjust migration speed and scope, handling approved applications (static) and in‑process applications (dynamic) differently.

2.4 Incremental Data Sync

When creating or updating an application without a status field, write to MongoDB first; if MySQL write fails, compensate asynchronously via MQ.

3 Online Three‑Pronged Strategy (Gray Release / Monitoring / Rollback)

3.1 Monitoring (Data Comparison Read Logic)

Incremental data comparison: after dual‑write, send an MQ message, query both new and old databases, compare results in real time, log mismatches, trigger alerts, and write to log files for analysis.

Full data comparison: traverse all data in the old database, fetch corresponding data from the new database, normalize objects, compare for consistency, and log any anomalies.

3.2 Monitoring (Comparison Read Logic)

Introduce R2 traffic replay to accelerate the comparison process.

3.3 Gray Release (Gradual Traffic Switching)

Read traffic is switched based on vendor/whitelist and a percentage allocation; ThreadLocal is used to store and retrieve merchant information per request.

3.4 Rollback (Gray Write Switching)

Write switching consists of four steps:

Validate writes to the new database (gray test); if issues arise, revert.

After validation, backfill missing data in the new database.

Switch primary writes to the new database.

Once reads and writes are stable, decommission the old database.

Conclusion

The article thoroughly outlines the end‑to‑end process for migrating a production environment, covering migration rhythm, data source selection, code refactoring, data heterogeneity, bulk and incremental migration, and ensuring monitoring, rollback, and gray release capabilities for a smooth cut‑over.