How to Achieve Distributed Transaction Consistency with MQs

Distributed Transactions

What Is a Distributed Transaction?

When a system grows from a single machine to multiple nodes, communication must rely on the network, making traditional reliable method calls and inter‑process communication infeasible; network instability leads to data‑synchronization problems, which is the classic distributed‑transaction issue.

In a distributed transaction, participants, transaction‑supporting servers, resource servers, and the transaction manager reside on different nodes, and the goal is to guarantee data consistency across those nodes.

Common Distributed‑Transaction Solutions

2PC (Two‑Phase Commit) – strong consistency

3PC (Three‑Phase Commit)

TCC (Try‑Confirm‑Cancel) – eventual consistency

Saga – eventual consistency

Local message table – eventual consistency

MQ transaction – eventual consistency

The following sections focus on achieving consistency using message queues.

MQ‑Based Distributed Transactions

Local Message Table – Eventual Consistency

The producer not only handles its business logic but also maintains a message table that records information to be synchronized with other services; each message has a status indicating whether it has been processed successfully.

The business logic and the insertion into the message table are executed within a single transaction, avoiding situations where business succeeds while the transaction message fails, or vice‑versa.

Example: an order service creates an order and sends a message to a cart service to clear the purchased items. The cart service listens to the queue, processes the message, and replies to the order service indicating success or the need for rollback. Retries and idempotency ensure reliability.

Key operations:

Both producer and consumer must be idempotent.

The producer uses a timer to retry unprocessed messages, preventing transaction breakage.

MQ Transaction – Eventual Consistency

Different MQs provide transaction support:

RocketMQ Transaction Handling

RocketMQ ensures that local transaction execution and message sending either both succeed or both fail, and adds a transaction‑check mechanism to improve success rates.

Normal commit flow:

Send a half‑message (invisible to consumers until commit).

MQ server records the half‑message and returns a result.

Based on the result, execute the local transaction; if successful, send a Commit, otherwise a Rollback.

On Commit, the message is delivered to downstream consumers; on Rollback, the message is discarded.

If the server does not receive Commit or Rollback, it initiates a compensation query to the producer to obtain the transaction status.

Kafka Transaction Handling

Kafka guarantees that all messages sent within a transaction are either all committed or all aborted, achieving exactly‑once semantics for the read‑process‑write pattern.

Transaction coordinator (part of the broker) records transaction IDs, and the producer interacts with the coordinator to begin, commit, or abort transactions.

RabbitMQ Transaction Handling

RabbitMQ provides a transaction API where the producer selects a transaction, sends messages, and either commits or rolls back based on server acknowledgment. This approach blocks the producer and reduces throughput.

RabbitMQ also offers publisher confirms (synchronous, batch, or asynchronous) to ensure messages reach the broker without using full transactions.

Message Loss Prevention

Message flow consists of production, storage, and consumption stages. Each stage can suffer loss, and various MQs implement safeguards:

Production Stage

RabbitMQ: use channel.txSelect, channel.txCommit, and channel.txRollback to wrap sends in a transaction.

RabbitMQ: enable publisher confirms to receive acknowledgments from the broker.

Kafka: require broker acknowledgment before considering a send successful.

RocketMQ: use synchronous send or configure the broker for synchronous disk flush and replication to multiple nodes.

Storage Stage

Persisting messages (durable exchanges, durable queues, and persistent messages) reduces loss risk; replication and mirroring further improve availability.

Consumption Stage

Consumers should acknowledge messages only after business processing completes, ensuring that unprocessed messages can be redelivered.

Message Duplicate Handling

Message delivery semantics include:

At most once – may lose data.

At least once – no loss but possible duplicates.

Exactly once – no loss and no duplicates (the highest guarantee).

Most MQs provide at‑least‑once delivery; consumers must ensure idempotency, e.g., using unique database keys, conditional updates, or attaching a unique ID to each message and tracking it in a ledger table.