How to Ensure Distributed Transaction Consistency Using Message Queues

What Is a Distributed Transaction?

When a system grows from a single machine to a cluster of nodes, inter‑process communication must rely on the network, which is unreliable; consequently, keeping data consistent across machines becomes a classic distributed‑transaction problem. Participants, transaction managers, and resource servers reside on different nodes, and the goal is to guarantee data consistency among them.

Common Distributed‑Transaction Solutions

2PC (Two‑Phase Commit) – strong consistency

3PC (Three‑Phase Commit)

TCC (Try‑Confirm‑Cancel) – eventual consistency

Saga – eventual consistency

Local message table – eventual consistency

MQ transaction – eventual consistency

The focus of this article is the MQ‑based approach; the other designs are referenced in the final citation links.

MQ‑Based Distributed Transaction (Local Message Table)

The producer (e.g., the Order service) maintains a local message table alongside its business logic. Each row records the information that must be synchronized to another service (e.g., the Cart service) and includes a status flag indicating whether the message has been processed successfully.

The business operation and the insertion of the message record are performed within a single database transaction, preventing the following inconsistencies:

business processing succeeds + message send fails
business processing fails + message send succeeds

Example Scenario :

The Order service completes the order creation and publishes a message to the MQ for the Cart service to clear the purchased items.

The Cart service listens to the queue; upon receiving the message, it performs a local transaction to delete the items and replies to the Order service.

If the Cart service processes the message successfully, it sends an acknowledgment; otherwise, it signals a rollback.

If the acknowledgment is lost, the Order service periodically retries unsent messages from its local table.

Two critical operations are required:

Both producer and consumer must implement idempotent processing to handle duplicate deliveries.

The producer must run a scheduled task that scans the message table for unprocessed entries and retries them, preventing transaction gaps.

Pros :

Reliability is achieved at the design level without depending on MQ features.

Implementation is simple and straightforward.

Cons :

High coupling with business data; the same database stores both business and message data, consuming additional resources.

MQ Transaction (RocketMQ)

RocketMQ solves the problem of ensuring that a local transaction and a message send either both succeed or both fail. It adds a transaction‑check mechanism to improve success rates and consistency.

The process consists of two phases:

Send a half message (invisible to consumers until commit).

The broker records the half message and returns a response.

Based on the broker’s response, the producer decides whether to execute the local transaction.

According to the local transaction outcome, the broker receives a Commit (delivers the message) or Rollback (discards the half message).

If the broker does not receive a commit/rollback, it initiates a compensation flow: it queries the producer for the transaction status, the producer replies, and the broker proceeds accordingly.

Kafka Transaction

Kafka guarantees atomicity across multiple partitions: either all messages in a transaction are written successfully, or none are. It achieves exactly‑once semantics by combining the transaction coordinator, a special transaction‑log topic, and the idempotent producer.

Key steps:

The producer sends an OpenTransaction request; the coordinator records a transaction ID in the log.

The producer sends transactional messages; unlike RocketMQ, uncommitted messages are stored like normal messages and filtered client‑side.

After sending, the producer commits or aborts the transaction with the coordinator.

Commit flow:

The coordinator sets the transaction state to PrepareCommit and writes it to the log.

Each partition writes a commit marker; consumers can now read the previously hidden messages.

Rollback flow:

The coordinator sets the state to PrepareAbort and writes it to the log.

Partitions write an abort marker, causing the half messages to be discarded.

RabbitMQ Transaction

RabbitMQ provides a transaction API to guarantee that a message reaches the server. The workflow is:

channel.txSelect   // start transaction
... send messages ...
if (send fails) {
    channel.txRollback   // rollback and retry
} else {
    channel.txCommit     // commit
}

Because the transaction is synchronous, the producer blocks until the server acknowledges, which severely reduces throughput.

RabbitMQ also offers a confirm mode where each published message receives a unique delivery tag; the broker sends an Basic.Ack once the message is safely stored. Confirm mode can be:

Synchronous confirm (low efficiency)

Batch confirm (higher efficiency but duplicate risk on failure)

Asynchronous confirm (high efficiency, no blocking)

Message Loss Prevention

The lifecycle of a message includes production, storage, and consumption stages. Each stage has specific safeguards:

Production Stage

Network failures can cause loss. Producers should retry on detectable errors and, for MQs that support transactions, wrap the send in a transaction.

Storage Stage

Broker crashes can lose messages. Strategies include:

Persisting messages (setting delivery_mode=2 in RabbitMQ).

Persisting queues and exchanges (RabbitMQ durable=true).

Using replicated brokers (Kafka, RocketMQ) and waiting for acknowledgments from multiple replicas before confirming.

Consumption Stage

Consumers should acknowledge only after business processing completes, ensuring at‑least‑once delivery without premature deletion.

Message Duplication Handling

Delivery guarantees: At most once: possible loss. At least once: no loss, possible duplicates (most MQs). Exactly once: no loss, no duplicates (Kafka’s transactional mode).

Idempotency techniques:

Leverage a unique database key (e.g., a流水表) and perform business logic and key insertion in one transaction.

Use conditional updates (e.g., UPDATE ... WHERE version=?).

Attach a unique ID to each message and deduplicate using the database unique constraint.

Overall, combining idempotent processing, persistent storage, and appropriate acknowledgment strategies enables reliable distributed transactions across heterogeneous message‑queue systems.