Why and How to Split Applications and Databases: Practical Practices

1 Why Split?

From the dialogue shown, the reasons for splitting are:

Severe coupling between applications : the same functionality is duplicated across many apps, so a change requires modifications in all of them.

Poor business extensibility : the original data model only supports a single business type, making new types cause re‑coding and delays.

Old, hard‑to‑maintain code : scattered if‑else logic and hard‑coded rules create many pitfalls.

Poor system scalability : both applications and databases cannot bear rapid business growth.

Increasing technical debt : without change the system will eventually become unsustainable.

2 Preparation Before Splitting

2.1 Grasp Business Complexity from Multiple Dimensions

Understanding the relationship between system and business is crucial; the ideal is a loose coupling like a car and driver, but reality often resembles a pacemaker and patient – tightly bound and not easily replaceable.

Technical analysis (domain models, pros/cons) combined with hands‑on practice (requirements, refactoring, optimization) is needed to fully understand complexity.

From a system perspective, building a platform‑type product requires centralized control, breaking small business loops and unifying functions, which demands consensus among developers, product owners, and business teams.

2.2 Define Service Boundaries – High Cohesion, Low Coupling, Single Responsibility

Good boundaries are like the brothers in "Huluwa": each application has an independent skill (single responsibility) yet can combine into a larger whole.

Granularity is hard to pin down; start with larger boundaries and let evolution naturally drive further splitting.

2.3 Set Goals for the Split Application

After drawing a macro split diagram, decide concrete targets for each iteration, e.g., first split DB and application, then redesign the data model in the next phase.

2.4 Assess Current Architecture, Code, Dependencies and Anticipate Exceptions

Pre‑implementation thinking costs far less than solving problems after work begins; unexpected constraints can derail the schedule and morale.

2.5 Keep a “Preparedness” Pocket Guide

Write down multiple solution options, decomposition of complex problems, and contingency plans to increase success probability and confidence.

2.6 Relax and Reduce Stress

Clear your mind and start working.

3 Practice

3.1 Database Splitting Practice

DB splitting is the most complex part and includes vertical and horizontal splitting.

Vertical splitting : move tables with different responsibilities to separate databases (e.g., messages vs. organization structures).

Horizontal splitting : split a massive table (e.g., messages) into multiple databases/tables.

3.1.1 Global ID Generator for Primary Keys

Replace auto‑increment IDs with globally unique IDs before migration to avoid primary‑key conflicts during rollbacks.

Common solutions include:

Snowflake (Twitter)

Dedicated MySQL table using AUTO_INCREMENT

Dual‑table odd/even strategy or multiple tables with different step ranges

Alibaba’s tddl‑sequence (non‑incremental global uniqueness)

Using tddl‑sequence requires adjusting order‑by clauses and handling occasional primary‑key conflicts caused by incomplete code changes.

3.1.2 Create New Tables, Migrate Data, Binlog Sync

Recommendations:

Use utf8mb4 charset and ensure all indexes are created.

Perform full data migration during low‑traffic periods with controlled concurrency.

After full migration, use binlog incremental sync tools (e.g., Alibaba Canal or Otter) to keep data consistent.

Make sure the binlog start position is set before the full migration to avoid data loss.

3.1.3 Refactor Cross‑Database Join Queries

Since cross‑database joins are unsupported, rewrite hundreds of join SQLs using strategies such as:

Business decoupling to eliminate joins.

Duplicating global tables (though this reduces the benefit of splitting).

Adding redundant fields.

Fetching remote data via RPC and assembling in memory (suitable for low‑volume jobs).

Local caching of remote tables for high‑volume, stable queries.

3.1.4 Cut‑over Schemes

a) DB Write‑Stop Scheme – stop writes, switch quickly, low cost, but high rollback risk and limited to a single verification point.

b) Dual‑Write Scheme – write to old and new tables simultaneously for a short period, then stop binlog sync; higher safety and easier rollback, but longer duration and increased latency.

3.1.5 Proper Switch Handling

Initialize feature switches to null; using a default value can cause stale reads after a restart, leading to dirty data.

3.2 Ensuring Consistency After Splitting

Options include:

Distributed transactions (poor performance, rarely used).

Message‑based compensation.

Scheduled‑task compensation (both data‑add and data‑delete types).

3.3 Maintaining Stability After Splitting

Key principles:

Distrust third‑party services – use defensive programming, timeouts, and fallback strategies.

Guard against misuse by consumers – design minimal, well‑documented interfaces and apply flow control per application priority.

Maintain your own service – enforce single responsibility, clean historical technical debt, SOP‑ize operations, and predict resource usage (CPU, memory, network, disk).

Examples illustrate how local caching, rate limiting, and periodic data flushing can make resource consumption predictable.

4 Summary

1) Be prepared for pressure.

2) Decompose complex problems into testable, rollback‑able steps.

3) Murphy’s law: the things you fear will happen quickly, so have SOPs ready.

4) “Borrow false to achieve true” – take on challenging refactoring opportunities to grow.

Final Note (Support the Author)

If this article helped you, please like, view, share, and bookmark; your support motivates the author to keep writing.

The author also offers a knowledge‑sharing community with paid content such as Spring full‑stack practice, billion‑scale sharding, DDD microservice series, and source‑code deep dives.