How to Reach Millisecond Consistency for Million‑Scale Transactions with RocketMQ

Background and Challenge

In modern internet applications, transaction systems handling millions or even tens of millions of requests per second are common. A core difficulty is guaranteeing cross‑service and cross‑database data consistency within sub‑second latency under extreme concurrency. Traditional two‑phase commit (2PC) or TCC solutions are often rejected because of their performance overhead, complexity, and high development cost.

Why Choose RocketMQ Transactional Messages?

Distributed systems suffer from network uncertainty and data inconsistency. For example, a payment service updates the local payment status to "success" and must notify the order service to change the order status. If the message fails to send or the local transaction and message send cannot be made atomic, data inconsistency occurs.

RocketMQ’s transactional message design splits a distributed transaction into a local transaction and an asynchronous message, using a two‑phase commit and transaction‑status check to ensure eventual consistency.

Working Principle (Two‑Phase Commit)

Phase 1 – Send Half Message

The producer sends a “half‑transaction message” that is stored but not deliverable to consumers.

The MQ server persists the message and returns a success ACK to the producer.

Phase 2 – Execute Local Transaction and Commit

After receiving the ACK, the producer executes the local DB transaction (e.g., deduct inventory, generate payment record).

Based on the local transaction result, the producer sends a Commit or Rollback command to the MQ. Commit: the half message becomes deliverable; consumers can consume it. Rollback: the half message is deleted and never visible to consumers.

Fallback – Transaction Status Check

If the producer does not send Commit or Rollback, RocketMQ periodically initiates a status check.

The producer implements a check interface, queries the local transaction status using a unique business key (e.g., order ID), and returns Commit or Rollback.

Guarantee: As long as the local transaction succeeds, the message is eventually sent; if the local transaction fails, the message is rolled back, providing a foundation for error rates as low as 10⁻⁹.

Atomic‑Level Wrapper Design

The goal is to make distributed transaction usage as simple as adding @Transactional on a method. The wrapper provides:

Annotation‑Driven: @TransactionalMessage processed by AOP to handle message sending and transaction listening automatically.

Context Management: Uses ThreadLocal or TransmittableThreadLocal to pass message content and business ID through the transaction context.

Automatic Status Check: A generic TransactionCheckListener queries business status based on the ID.

Idempotence & Exception Handling: Consumer side integrates idempotent checks and provides retry and degradation strategies.

Architecture Diagram

Key Code Implementation (Simplified)

1. Define Annotation

@Target({ElementType.METHOD})
@Retention(RetentionPolicy.RUNTIME)
public @interface TransactionalMessage {
    String topic();
    String tag() default "";
    String bizId(); // business unique ID, e.g., "#order.id"
}

2. AOP Aspect

@Aspect
@Component
@Slf4j
public class TransactionalMessageAspect {
    @Autowired
    private RocketMQTemplate rocketMQTemplate;

    @Around("@annotation(transactionalMessage)")
    public Object executeInTransaction(ProceedingJoinPoint joinPoint, TransactionalMessage transactionalMessage) throws Throwable {
        Message message = buildMessage(joinPoint, transactionalMessage);
        String bizId = parseBizId(joinPoint, transactionalMessage.bizId());
        TransactionSendResult sendResult = rocketMQTemplate.sendMessageInTransaction(
                transactionalMessage.topic() + ":" + transactionalMessage.tag(),
                message,
                bizId);
        if (!sendResult.getSendStatus().equals(SendStatus.SEND_OK)) {
            throw new RuntimeException("Half message send failed.");
        }
        return joinPoint.proceed(); // execute local transaction
    }
}

3. Generic Transaction Listener & Check

@Component
@RocketMQTransactionListener
public class CoreTransactionListener implements RocketMQLocalTransactionListener {
    @Autowired
    private BizStatusCheckService statusCheckService;

    @Override
    public RocketMQLocalTransactionState executeLocalTransaction(Message msg, Object arg) {
        try {
            return RocketMQLocalTransactionState.COMMIT;
        } catch (Exception e) {
            return RocketMQLocalTransactionState.ROLLBACK;
        }
    }

    @Override
    public RocketMQLocalTransactionState checkLocalTransaction(Message msg) {
        String bizId = /* extract from msg */;
        boolean isSuccess = statusCheckService.checkStatusById(bizId);
        return isSuccess ? RocketMQLocalTransactionState.COMMIT : RocketMQLocalTransactionState.ROLLBACK;
    }
}

4. Business Usage Example

@Service
public class OrderPaymentService {
    @Autowired
    private OrderMapper orderMapper;

    @Transactional
    @TransactionalMessage(topic = "ORDER_PAY_SUCCESS_TOPIC", bizId = "#order.id")
    public void payOrder(Order order) {
        orderMapper.updateStatus(order.getId(), OrderStatus.PAID);
    }
}

Performance Optimizations & Capacity Considerations

Million‑QPS Strategies

Batch message sending vs. single message sending.

Broker memory and disk tuning.

Timely cleanup of half messages.

Transaction Timeout & Long‑Running Transactions

Split long local transactions or handle them asynchronously.

The status‑check mechanism ensures eventual consistency for long‑running transactions.

Failure Scenarios and Recovery Strategies

Producer crash: fallback to status‑check mechanism.

Network jitter causing lost Commit: idempotent sending to avoid duplicates.

Consumer duplicate consumption: idempotent consumption with unique ID deduplication.

Broker temporarily unavailable: message persistence with multiple replicas ensures reliability.

Extensions and Monitoring

Link tracing: integrate SkyWalking or Zipkin for full‑chain transaction tracing.

Metric monitoring: half‑message backlog, send failures, check failures, consumption latency.

Degradation strategy: when MQ or transaction errors surge, business can switch to a degraded path.

Applicable Scenarios

E‑commerce payment systems

Order processing systems

Inventory / points / coupon management

Financial settlement systems

Transactional messaging fits scenarios demanding extremely high data consistency while also requiring high performance and low latency.

Achieving 10⁻⁹‑Level Ultra‑Low Error Rate

MQ High Availability: multi‑master, multi‑slave, data persistence, multi‑replica synchronization.

Producer Guarantees: half‑message mechanism, automatic status check, idempotent sending.

Consumer Guarantees: idempotent consumption using Redis or DB unique keys.

Monitoring & Alerts: real‑time alerts for message backlog, send failures, and check failures.

By stacking these safeguards, the overall link error rate can be reduced to one in a billion.

Advantages

High performance: asynchronous messaging decouples systems, easily handling million‑QPS workloads.

Extreme simplicity: business code only focuses on the local transaction; annotation handles the rest.

Ultra‑reliability: RocketMQ + status check + idempotent consumption guarantees consistency.

Broad applicability: can serve as a company‑wide foundational component for any final‑consistency scenario.

Conclusion

RocketMQ transactional messages are a powerful tool; by wrapping them at the atomic level, they become a precise, convenient, and safe “launch pad” for building million‑scale transaction systems with millisecond‑level data consistency, turning a traditionally complex problem into a standardized development task.