Ensuring Data Consistency Across Microservices: Strategies and Design Principles

Why Data Consistency Between Services Matters

In large‑scale microservice architectures, business operations often require reading and writing data across multiple services. Scenarios such as e‑commerce order processing, marketing points, and coupon handling expose consistency problems when disparate services update related data independently.

Core Terminology

Business‑side system : the consumer that initiates a business operation.

Platform‑side system : the provider of foundational capabilities that execute the operation.

Business operation : a data‑read/write task that spans multiple services.

Operation identifier : a tag that classifies the type of business operation.

Operation unique ID : a globally unique identifier for a specific execution of a business operation.

Operation log table : a persistent log that records each operation’s identifier and unique ID.

Solution 1 – Business‑Side Guarantees Final Consistency

Core Idea

The business‑side service records a log for every operation and triggers a consistency‑check process for any changes that did not succeed completely.

Design Principles

Provide an interface for the platform‑side to execute business operations, ensuring idempotency based on the operation identifier and unique ID.

Expose a result‑query interface that allows the business‑side to verify outcomes using the same identifiers.

Maintain an operation log table to track identifiers and unique IDs.

The business‑side periodically or synchronously validates the log against platform data, using real‑time sync, near‑real‑time async checks, or scheduled batch jobs.

Workflow Diagram

Solution 2 – Platform‑Side Guarantees Final Consistency

Core Idea

The platform‑side service actively confirms each data change with the business‑side, ensuring the result matches expectations.

Design Principles

Platform interfaces must be idempotent, using operation identifier and unique ID.

Provide a callback SPI for the business‑side to confirm execution results.

Business‑side implements a verification callback based on the same identifiers.

Workflow Diagram

Practical Design Details

Data Structure Design

Idempotency relies on the operation identifier (Tag) and unique ID. Unique IDs can be auto‑increment keys, distributed ID generators, or meaningful business fields such as userId, orderId, skuId. Using business‑meaningful IDs simplifies integration without extra dependencies.

State Storage Design

MySQL is recommended as the primary store for its strong consistency guarantees, often combined with a unique composite index on Tag + UniqueID. To alleviate MySQL bottlenecks, a Redis pre‑check can filter traffic before hitting MySQL, with asynchronous sync back to MySQL after successful validation. Archiving old data and sharding tables keep row counts manageable (≈5‑10 million rows per table).

Exception Handling

Identify exception categories (network, format, business logic) and apply strategies such as retry with back‑off, rate‑limited retries, and fallback handling for terminal failures.

Result Uniqueness

Ensure repeated calls return identical responses by using a state‑machine to control status codes or by validating core business parameters before responding.

Latency Tolerance

Assess acceptable delay for consistency checks (e.g., 1 minute batch, MQ‑driven async, or synchronous zero‑delay at the cost of throughput). Choose between full‑scan and incremental checks based on workload.

State‑Machine Design

Define initial, intermediate, and terminal states. Successful terminal state indicates consistency; failure states capture retryable or non‑retryable errors, enabling metrics on latency and failure ratios.

Concurrency Control

Use MySQL unique constraints to prevent duplicate operation logs. For high‑concurrency scenarios, consider locking, queuing, or MQ‑based buffering to protect throughput.

Availability and Observability

Implement circuit‑breaker, rate‑limiting, and graceful degradation.

Set up alerting for exception states, threshold breaches, and resource usage (MySQL, Redis, MQ).

Build monitoring dashboards for storage health, consistency check progress, and latency metrics.

Conclusion

The article compares two viable patterns for achieving cross‑service data consistency. The business‑side approach suits scenarios with strong coupling to operation outcomes, while the platform‑side approach fits cases where the final data state is the primary concern. Both require careful design of identifiers, idempotent interfaces, storage choices, latency handling, state‑machine modeling, concurrency safeguards, and observability to succeed in production.