Preventing Message Loss in RocketMQ: Scenarios and Solutions

When using RocketMQ in a project, the risk of message loss becomes critical, especially in financial transaction scenarios. The article first outlines three typical loss scenarios: (1) network glitches during producer-to‑RocketMQ transmission, (2) broker failures before asynchronous disk flush or lack of data backup, and (3) consumer crashes after reporting successful consumption.

To guarantee zero loss, three corresponding solutions are proposed:

Scenario 1: Use RocketMQ’s built‑in transaction mechanism. The producer first sends a half‑message, executes its business logic, and then either commits or rolls back the message based on the outcome, ensuring the message is persisted only after the core transaction succeeds.

Scenario 2: Switch the broker’s flushDiskType to SYNC_FLUSH for synchronous disk flushing, and deploy RocketMQ in a master‑slave cluster so that each message is replicated across multiple followers, protecting against disk failures.

Scenario 3: Ensure the consumer only acknowledges a message after the business processing completes. The provided Java code registers a MessageListenerConcurrently that returns ConsumeConcurrentlyStatus.CONSUME_SUCCESS after processing; if the consumer crashes before this return, RocketMQ will re‑deliver the message to another consumer in the group.

Code examples illustrate both a straightforward listener and a problematic pattern that spawns a new thread for asynchronous processing, which can cause premature acknowledgments and potential loss.

// Register message listener
consumer.registerMessageListener(new MessageListenerConcurrently() {
    @Override
    public ConsumeConcurrentlyStatus consumeMessage(List<MessageExt> msgs, ConsumeConcurrentlyContext context) {
        // Process the message
        return ConsumeConcurrentlyStatus.CONSUME_SUCCESS;
    }
});

The article also notes that while these measures eliminate loss, they introduce performance overhead: transactional messaging adds extra steps, synchronous flushing is slower than asynchronous, replication requires leader‑follower synchronization, and consumers must wait for processing before acknowledging.

In summary, achieving zero‑message loss in RocketMQ involves careful selection of the appropriate strategy based on business requirements, balancing reliability against throughput.