Understanding RocketMQ: Architecture, Key Features, and Best Practices

RocketMQ is a high‑performance, high‑throughput distributed messaging middleware that addresses two core challenges in distributed systems: message ordering and message duplication.

Key Features and Implementation

1. Ordered Messages

To guarantee order, messages must be sent to the same server and consumed one‑by‑one; the article illustrates this with diagrams and explains the pitfalls of network latency and server failures. A simple solution is a one‑to‑one producer‑MQServer‑consumer relationship, though this reduces parallelism.

Ensure producer - MQServer - consumer is a one‑to‑one relationship.

2. Duplicate Messages

Duplication arises from network unreliability. The recommended mitigation is to make consumer logic idempotent and optionally maintain a de‑duplication log keyed by a unique message ID.

Keep consumer processing idempotent. Record processed message IDs in a log table.

3. Transaction Messages

RocketMQ supports transactional messages through a three‑phase process: send a Prepared message, execute a local transaction, then commit or rollback based on the transaction outcome. The broker periodically checks unresolved transactions and invokes the producer’s TransactionCheckListener to decide the final state.

TransactionCheckListener transactionCheckListener = new TransactionCheckListenerImpl();
TransactionMQProducer producer = new TransactionMQProducer("groupName");
producer.setTransactionCheckListener(transactionCheckListener);
TransactionExecuterImpl tranExecuter = new TransactionExecuterImpl();
producer.start();
Message msg = new Message(...);
SendResult sendResult = producer.sendMessageInTransaction(msg, tranExecuter, null);
producer.shutdown();

4. Producer Message Sending

Producers initialize once, retrieve topic routing info, select a queue via round‑robin, and send messages. The code shows how DefaultMQProducer builds a Message and sends it, handling retries and load balancing.

DefaultMQProducer producer = new DefaultMQProducer("ProducerGroupName");
producer.start();
Message msg = new Message("TopicTest1", "TagA", "OrderID188", "Hello MetaQ".getBytes());
SendResult sendResult = producer.send(msg);
producer.shutdown();

5. Message Storage

RocketMQ stores messages in two files: ConsumeQueue (logical index) and CommitLog (physical data). The ConsumeQueue holds fixed‑length entries pointing to offsets in the CommitLog, while the CommitLog stores variable‑length message records.

msg.setStoreTimestamp(System.currentTimeMillis());
msg.setBodyCRC(UtilAll.crc32(msg.getBody()));
MapedFile mapedFile = this.mapedFileQueue.getLastMapedFile();
AppendMessageResult result = mapedFile.appendMessage(msg, this.appendMessageCallback);
DispatchRequest dispatchRequest = new DispatchRequest(topic, queueId, result.getWroteOffset(), result.getWroteBytes(), tagsCode, msg.getStoreTimestamp(), result.getLogicsOffset(), msg.getKeys(), msg.getSysFlag(), msg.getPreparedTransactionOffset());
this.defaultMessageStore.putDispatchRequest(dispatchRequest);

6. Message Subscription

Consumers can use push or pull modes, but both are implemented as long‑polling pulls. A rebalance service periodically distributes topic queues among consumers using strategies like average allocation to achieve load balancing.

7. Best Practices

Recommended practices include using a single topic per application with tags for sub‑types, setting unique keys for traceability, logging send results, implementing retry mechanisms, ensuring idempotent consumer processing, and disabling automatic topic creation in production.

Overall, the article provides a deep dive into RocketMQ’s design principles, code‑level implementations, and operational guidelines for building robust, scalable messaging solutions.