Understanding Transactional Messages in Distributed Systems: RocketMQ and Kafka

When we talk about transaction messages in a message queue, the concept of ACID immediately comes to mind. In monolithic systems, strict ACID constraints are rarely enforced, and in distributed systems we often have to compromise to achieve eventual consistency because availability is paramount.

Distributed Transaction

Common distributed transaction models include 2PC, TCC, and transactional messages. This article focuses on transactional messages while briefly mentioning 2PC and TCC.

2PC

Two‑Phase Commit (2PC) involves a coordinator and participants. In the prepare phase the coordinator asks participants to prepare; in the commit phase the coordinator decides to commit if all participants are ready, otherwise it rolls back.

2PC is limited to database‑level transactions and enforces strong consistency through synchronous blocking, which can cause long‑lasting resource locks and single‑point‑of‑failure risks.

TCC

Try‑Confirm‑Cancel (TCC) guarantees business‑level transactions. Each business operation must implement three methods: try (reserve resources), confirm (perform the actual operation), and cancel (rollback if any try fails). TCC introduces high coupling with business logic and requires idempotent confirm/cancel steps.

Transactional Message

Transactional messages are suited for asynchronous update scenarios where real‑time data consistency is not critical. Their purpose is to ensure that the producer and consumer see consistent data, e.g., deleting a shopping‑cart entry only after an order is successfully placed.

RocketMQ Transactional Message

RocketMQ implements transactional messages as a two‑phase process. When a transaction starts, a half‑message is sent to the broker; this message is invisible to consumers and stored in a special queue.

After the half‑message is sent, the local transaction is executed. Depending on the transaction result, the producer sends either a commit or a rollback message.

If the commit/rollback message fails to reach the broker, the broker periodically checks the transaction status by invoking a callback interface on the producer.

Below is a simplified usage example (code omitted for brevity).

RocketMQ Transactional Message Source Code Analysis

The core method is sendMessageInTransaction , which constructs a half‑message, sends it to the broker, executes the local transaction, and reports the transaction state back to the broker.

On the broker side, SendMessageProcessor#sendMessage detects the MessageConst.PROPERTY_TRANSACTION_PREPARED flag, stores the half‑message under the internal topic RMQ_SYS_TRANS_HALF_TOPIC , and prevents consumers from reading it.

The broker later processes commit or rollback requests via EndTransactionProcessor#processRequest . A commit rewrites the message to the real topic/queue, while a rollback records the half‑message in a half_op topic for later cleanup.

A background service TransactionalMessageCheckService periodically scans half‑messages and invokes TransactionalMessageServiceImpl#check to query the producer for the transaction outcome.

Kafka Transactional Message

Kafka’s transactional messages differ: they ensure that multiple messages sent within a single transaction are either all committed or all aborted, leveraging Kafka’s idempotent producer to achieve exactly‑once semantics for the producer side.

Kafka uses a transaction coordinator (part of the broker) to log transaction states. The producer sends a begin request, writes messages, then sends a commit or abort request. Consumers filter out transactional messages until the transaction is completed.

Kafka’s exactly‑once guarantee applies only to specific scenarios where the same Kafka cluster is both the source and the sink of data; it does not provide the same end‑to‑end transactional guarantees as RocketMQ.

Conclusion

Both RocketMQ and Kafka support transactional messaging, but RocketMQ’s model aligns with the typical need to coordinate a local business operation with a message send, while Kafka’s model is suited for stream‑processing pipelines that require exactly‑once delivery across multiple messages.