Why and How to Split Monolithic Applications: A Practical Guide to Service and DB Refactoring

1. Why Split?

The article starts with a dialogue illustrating four main pain points of monolithic systems: severe inter‑application coupling, poor business extensibility, outdated and hard‑to‑maintain code, and limited system scalability. These issues lead to high maintenance cost and risk of system failure.

2. Preparation Before Splitting

2.1 Understand Business Complexity – Analyze the domain model of each existing application, discuss with product and development teams, and practice by implementing or refactoring features to grasp real‑world details.

2.2 Define Service Boundaries – Follow the principles of high cohesion, low coupling, and single responsibility. Use analogies such as the “Huluwa brothers” to illustrate independent yet composable services.

2.3 Set Goals for Each Split – Determine concrete objectives (e.g., separate DB and application in the first phase, redesign data model in the second). Avoid overly fine granularity; start with larger boundaries and refine later.

2.4 Assess Current Architecture – Review codebase, dependencies, and potential failure scenarios before proceeding.

3. Practical DB Splitting

3.1 Vertical and Horizontal Splitting – Vertical splitting moves tables to separate databases based on functional domains; horizontal splitting shards large tables (e.g., a message table with millions of rows) across multiple databases.

3.1.1 Global ID Generation – Replace auto‑increment primary keys with a globally unique ID generator to avoid key collisions during migration. Options include:

Twitter Snowflake (non‑sequential)

MySQL table using auto_increment (sequential)

Dual‑table approach generating odd/even IDs

Alibaba’s internal tddl‑sequence (memory‑backed, non‑sequential)

Using global IDs prevents conflicts when rolling back or merging data after a split.

3.1.2 Migration Steps

Create new tables with utf8mb4 charset and required indexes.

Perform full data migration during low‑traffic windows, controlling concurrency.

Synchronize incremental changes via binlog tools (e.g., Alibaba Canal or Otter). Ensure the binlog start point is before the full migration to avoid data loss.

3.1.3 Join Query Refactoring – Cross‑database joins are unsupported; therefore, all join queries must be rewritten. Strategies include:

Business decoupling to eliminate joins.

Duplicating tables (global tables) – not ideal for large schemas.

Adding redundant foreign‑key fields.

In‑memory composition via RPC calls (suitable for low‑QPS jobs).

Local caching of remote data for high‑QPS read‑heavy scenarios.

3.1.4 Cut‑over Strategies

DB Stop‑Write – Quickly switch traffic by stopping writes to the old DB. Low cost but risky if rollback is needed during peak periods.

Dual‑Write – Write to both old and new tables for a short window, handling exceptions explicitly. Safer rollback, but adds latency and complexity.

3.1.5 Feature Switch Management – Feature switches must default to null and be carefully initialized to avoid default‑value‑induced dirty reads after a restart.

4. Consistency After Splitting

When data moves to separate databases, traditional transactions no longer work. Options are:

Distributed transactions – generally avoided due to performance impact.

Message‑based compensation – use a message queue to achieve eventual consistency.

Scheduled compensation jobs – periodic tasks that reconcile data differences.

5. Post‑Split Stability

5.1 Defensive Programming – Implement fallback strategies, set timeouts, and convert strong dependencies to weak ones (e.g., async processing).

5.2 Interface Design for Consumers – Follow the principle of minimal exposure, provide batch APIs to avoid N+1 RPC calls, and avoid long‑running endpoints.

5.3 Capacity Control – Apply flow control per application, enforce quotas, and limit tenant usage in SaaS scenarios.

5.4 Operational SOPs – Maintain clear runbooks for incident response, clean up legacy technical debt, and monitor resource usage (CPU, memory, network, disk). Predictable resource consumption can be achieved by adding local caches (e.g., Guava) to limit burst traffic to Redis/Tair and the database.

6. Summary

Successful application splitting requires thorough preparation, incremental and testable steps, robust rollback plans, and well‑defined SOPs. Anticipate Murphy’s Law: the problems you fear will happen quickly, so invest in defensive design, clear boundaries, and automated remediation.