Mastering Distributed Transactions: Reliable Messaging, TCC, and Flexible Solutions

1. Prelude to Distributed Transactions

Transaction: A reliable, independent unit of work composed of multiple operations, characterized by ACID properties (Atomicity, Consistency, Isolation, Durability).

Local Transaction: Managed by a single resource manager, offering strict ACID, high efficiency, and simple programming model, but lacking distributed capabilities.

Global Transaction: Managed by a global transaction manager (TM) that coordinates the state of all participating resources, ensuring consistent commit or rollback across them.

TX Protocol: Interface between applications (or application servers) and the transaction manager.

XA Protocol: Interface between the global transaction manager and resource managers (RM). Defined by the X/Open group, it enables two‑phase commit (2PC) across databases and other resources.

AP, RM, TM: Application, Resource Manager, and Transaction Manager respectively, each playing a role in coordinating distributed work.

Two‑phase commit (2PC) coordinates multiple resources via a prepare phase and a commit phase, but incurs high protocol cost, persistent state overhead, and vulnerability to failures.

2. Data Consistency Theories

BASE Theory: Basic Availability, Soft state, Eventual consistency – prioritizes availability and performance over strict consistency.

CAP Theorem: In a distributed system you can only simultaneously guarantee two of Consistency, Availability, and Partition tolerance; real systems often blend ACID and CAP characteristics.

3. Service Patterns in Flexible Transactions

Idempotent Operations: Repeated calls produce the same result, achieved via business logic or caching previous responses.

TCC Operations:

Try: Reserve resources and perform checks, ensuring provisional consistency.

Confirm: Execute the actual business logic using reserved resources; must be idempotent.

Cancel: Release reserved resources; also idempotent.

Compensating Operations: Consist of a Do phase (actual business execution) and a Compensate phase (undo or partial rollback), both required to be idempotent.

4. Flexible Transaction Solution Architecture

In high‑traffic e‑commerce scenarios, traditional DB transactions become bottlenecks. Flexible transactions aim for basic availability and eventual consistency, tolerating intermediate states such as replication lag.

(1) Reliable‑Message Based Eventual Consistency

Implementation: Before committing a business transaction, the service writes a message to a real‑time message system (which only stores the message). After the transaction commits, it confirms the send; only upon confirmation does the message get actually delivered.

Message Flow: If the business transaction rolls back, a cancel request is sent. The message system periodically checks for unconfirmed or rolled‑back messages and queries the business service for their validity.

Cost & Advantages: Requires two requests per message and a status‑query interface, but provides independent storage, scalability, reduced coupling, and near‑real‑time eventual consistency across any JMS‑compatible middleware.

(2) TCC Compensation Model

Implementation: A business activity consists of a primary service and multiple participant services offering TCC operations. The activity manager registers all operations, invokes Confirm on commit, and Cancel on rollback.

Cost & Scope: Higher development cost due to Confirm/Cancel logic; suited for short‑lived, strongly isolated operations such as account handling.

(3) Best‑Effort Notification

Implementation: The active side sends a notification that may be lost; the passive side periodically queries the active side to recover missing messages.

Use Cases: Low time‑sensitivity eventual consistency, cross‑enterprise interactions, e.g., bank or merchant notifications.

5. Detailed Reliable‑Message Consistency Scheme

(1) Message Send Consistency

Message middleware provides asynchronous communication, decoupling, and traffic buffering. In distributed environments, network uncertainty introduces partition tolerance challenges (CAP).

Message send consistency means that if a business operation succeeds, the corresponding message must be sent; otherwise data divergence occurs.

Approach 1

public void completeOrderService() {
  // process order
  order.process();
  // send accounting voucher message
  pipe.sendAccountingVouchetMessage();
}

If the service crashes after processing the order but before sending the message, inconsistency arises.

Approach 2

public void completeOrderService() {
  // send accounting voucher message first
  pipe.sendAccountingVouchetMessage();
  // then process order
  order.process();
}

Reordering can cause the message to be sent while the order fails, also leading to inconsistency.

JMS XA interfaces (e.g., XAConnection, XAConnectionFactory, XAQueueConnection, etc.) support distributed transactions, but they impose high costs, require XA‑compatible resources, and contradict the flexible‑transaction philosophy.

(2) Work‑around for Message Consistency

The active side marks the message as “pending confirmation”. The middleware persists it but does not deliver to the passive side yet. After the business operation succeeds, the active side confirms the message, which then changes to “ready to send” and is delivered.

If any step fails, appropriate rollback or discard actions are taken.

(3) Conventional MQ Processing Flow

Standard MQ queues cannot guarantee message‑level consistency.

(4) Duplicate Message Handling & Idempotent Design

When a message is re‑delivered, the consumer must ensure its business interface is idempotent to avoid duplicate effects.

(5) Local Message Service Pattern

Implementation stores a copy of the message locally in the active application and another copy in the real‑time message service. The passive side acknowledges receipt, after which the active side deletes its local copy.

Pros: High timeliness, lightweight, reduces reliance on MQ features.

Cons: Tight coupling to specific business scenarios, resource consumption, and DB concurrency limits.

(6) Independent Message Service Pattern

Messages are pre‑sent to a dedicated message‑service subsystem, which manages status transitions (pending → sendable → consumed) and handles retries, acknowledgments, and recovery independently of the business application.

Pros: Independent deployment, scalable storage options, reusable across scenarios, reduced coupling.

Cons: Requires two requests per send and additional status‑query interfaces.

(7) Message Service Subsystem Design

Typical message table schema (illustrated in the original diagram) stores message ID, payload, status, timestamps, and retry counters.

Source: https://www.cnblogs.com/bluemiaomiao/p/11216380.html