Preventing Message Loss in RocketMQ: Scenarios, Solutions, and Trade‑offs

When a project adopts a message queue like RocketMQ, preventing message loss becomes critical, especially in financial transaction scenarios.

The article identifies three fundamental loss scenarios: (1) the producer fails to deliver the message to RocketMQ due to network instability; (2) the broker stores the message only in OS cache and crashes before it is flushed to disk; (3) the consumer reports successful consumption before actually processing the message and then crashes.

To achieve zero loss, the author proposes specific measures for each scenario. For scenario 1, RocketMQ’s built‑in transaction mechanism should be used, where a half‑message is first sent, the business logic is executed, and only upon success is the message committed; otherwise it is rolled back.

For scenario 2, the broker’s flushDiskType should be set to SYNC_FLUSH to force synchronous disk writes, and a master‑slave cluster should be deployed so that data is replicated across multiple brokers, protecting against disk failures.

For scenario 3, the consumer must acknowledge the message only after the business processing finishes. The article provides a Java example that registers a MessageListenerConcurrently and returns ConsumeConcurrentlyStatus.CONSUME_SUCCESS only after processing is complete, ensuring that a crash before acknowledgment will trigger automatic failover to another consumer.

// Register a message listener to process messages
consumer.registerMessageListener(new MessageListenerConcurrently() {
    @Override
    public ConsumeConcurrentlyStatus consumeMessage(List<MessageExt> msgs, ConsumeConcurrentlyContext context) {
        // Process the message
        return ConsumeConcurrentlyStatus.CONSUME_SUCCESS;
    }
});

The author warns that spawning a separate thread to handle the message asynchronously can re‑introduce loss, because the consumer may report success before the background thread finishes.

While the presented solutions guarantee message reliability, they significantly increase latency and reduce throughput due to extra transaction steps, synchronous disk I/O, and the need for replication.

In conclusion, achieving zero message loss is a trade‑off that must be balanced against performance requirements and the specific business context.