Why RocketMQ Beats Other Message Queues: A Deep Dive into Architecture, Models, and Best Practices

1. Why Use a Message Queue?

Message queues serve three major purposes, illustrated with an e‑commerce order flow:

Decoupling : Before a queue, the order service must synchronously call inventory, marketing, etc.; after introducing a queue, the order service simply publishes an "order completed" event and downstream services consume it independently.

Asynchrony : After payment, operations such as inventory deduction, point addition, and notifications form a long chain that increases latency. By moving everything except update order status to asynchronous processing, response time is reduced.

Peak‑shaving : In flash‑sale scenarios, traffic spikes overwhelm Redis or MySQL. By throttling consumption to the service's processing capacity, the system can absorb short‑term bursts without crashing.

2. Why Choose RocketMQ?

A market comparison of four popular MQs (ActiveMQ, RabbitMQ, RocketMQ, Kafka) highlights reliability, performance, features, operability, and community activity. RocketMQ is selected for a high‑concurrency C‑end e‑commerce system because it offers low latency, high throughput, and strong availability.

3. RocketMQ Pros and Cons

Advantages :

Single‑machine throughput reaches 100k messages/s.

Highly available distributed architecture.

Zero‑loss reliability after parameter tuning.

Rich feature set and good extensibility.

Supports billions of messages in backlog without performance degradation.

Java source code eases custom business integration.

Designed for financial‑grade reliability, ideal for order payment and flash‑sale scenarios.

Disadvantages :

Limited client language support (only Java and immature C++).

Lacks native JMS implementation, requiring code changes for migration.

4. Message Queue Models

Two models exist:

Queue Model : One producer sends to a queue; multiple consumers compete, each message consumed by only one consumer.

Publish/Subscribe Model : A publisher sends to a topic; every subscriber receives a copy of each message.

5. RocketMQ’s Message Model

RocketMQ adopts the standard publish/subscribe model. Its core components are:

Message : The payload; must belong to a Topic (the address).

Topic : First‑level classification (e.g., transaction, logistics).

Tag : Optional second‑level label for finer filtering.

Consumer Group : Each group receives a full copy of a topic; within a group, consumers compete.

Message Queue : A topic can contain multiple queues; consumers pull from assigned queues.

Offset : The index of the last consumed message in a queue (conceptually an array index).

6. Consumption Modes

Two modes are supported:

Clustering (load‑balanced) : One consumer in a group consumes each queue; if a consumer fails, others take over.

Broadcasting : Every consumer in the group receives every message.

7. RocketMQ Basic Architecture

Four components form the system:

NameServer : Stateless discovery service, similar to Zookeeper but lighter.

Broker : Stores and forwards messages.

Producer : Sends messages, supports synchronous, asynchronous, and one‑way modes.

Consumer : Pulls or pushes messages, supports clustering and broadcasting.

7.1 NameServer

Each NameServer runs independently, maintains long‑lived connections with brokers, and periodically registers broker routes. It provides two functions: keeping broker connections alive and maintaining topic routing information.

7.2 Broker

Broker stores messages in a CommitLog and maintains ConsumerQueue indexes for fast lookup. It synchronizes topic metadata with NameServers and uses Netty for communication.

7.3 Producer

Producers are user‑deployed and send messages via load‑balanced broker selection. Three sending modes are available:

Synchronous : Waits for broker acknowledgment before proceeding.

Asynchronous : Returns immediately after sending; a callback handles success/failure.

One‑way : Sends without waiting for any response, suitable for low‑reliability logs.

7.4 Consumer

Consumers can be PUSH or PULL. PUSH wraps PULL with a listener; both support clustering and broadcasting. Pull consumers can be Pull (active) or Push (passive callback).

8. Reliability Guarantees

Message loss can occur during production, storage, or consumption. Guarantees are achieved by:

Production : Synchronous send must check response OK; asynchronous send must verify callbacks; timeouts trigger log queries.

Storage : Use broker parameters that prioritize reliability (synchronous flush), persist to CommitLog, and employ master‑slave replication.

Consumption : Acknowledge only after business logic succeeds; otherwise the message remains unacknowledged and will be redelivered.

9. Handling Duplicate Messages

Two strategies are used:

Idempotent Business Logic : Ensure repeated processing yields the same result.

Message De‑duplication : Assign a unique key (e.g., order ID), store it in a DB with a primary‑key constraint, and skip processing if the key already exists.

10. Dealing with Backlog

Two approaches:

Scale out consumers when the number of queues exceeds consumer count.

Create a temporary topic with more queues, migrate messages there, consume, then switch back.

11. Ordered Messages

Two types:

Partial Order : Send messages with the same key to the same queue; consumers process that queue sequentially.

Global Order : Set topic’s read/write queue count to 1 and run producer/consumer in single‑thread mode, sacrificing throughput.

DefaultMQPushConsumer consumer = new DefaultMQPushConsumer("CID_EXAMPLE");
consumer.subscribe("TOPIC", "TAGA || TAGB || TAGC");

DefaultMQPushConsumer consumer = new DefaultMQPushConsumer("please_rename_unique_group_name_4");
consumer.subscribe("TopicTest", MessageSelector.bySql("a between 0 and 3"));
consumer.registerMessageListener(new MessageListenerConcurrently() {
    @Override
    public ConsumeConcurrentlyStatus consumeMessage(List<MessageExt> msgs, ConsumeConcurrentlyContext context) {
        return ConsumeConcurrentlyStatus.CONSUME_SUCCESS;
    }
});
consumer.start();

12. Delayed Messages

Use RocketMQ’s built‑in delay levels (e.g., 1s, 5s, 10s, up to 2h). Example:

DefaultMQProducer producer = new DefaultMQProducer("ExampleProducerGroup");
producer.start();
for (int i = 0; i < 100; i++) {
    Message msg = new Message("TestTopic", ("Hello scheduled message " + i).getBytes());
    msg.setDelayTimeLevel(3); // 10 seconds
    producer.send(msg);
}

Delay levels are defined in a string such as "1s 5s 10s 30s 1m 2m … 2h".

13. Transactional (Half) Messages

Half messages are stored as "not deliverable" until the producer confirms the local transaction. The flow:

Producer sends a half message.

Broker stores it with a flag.

Producer executes local transaction.

If commit, broker marks the message as normal; if rollback, the message is discarded.

If the broker does not receive a commit/rollback within a timeout, it queries the producer for the transaction status and acts accordingly.

14. Dead‑Letter Queue

When a message exceeds the maximum retry count, RocketMQ moves it to a special dead‑letter queue associated with the consumer group. The queue retains messages for three days before automatic deletion.

15. High Availability

NameServers are stateless and clustered, guaranteeing availability. Brokers achieve HA via clustering and master‑slave replication: producers write to masters, consumers can read from either master or slave, and automatic failover occurs when a master becomes unavailable.

16. Storage Mechanics

RocketMQ uses three file types:

CommitLog : Append‑only log of raw messages (default 1 GB per file).

ConsumeQueue : Fixed‑length index file (20 bytes per entry) mapping topic‑queue offsets to CommitLog positions.

IndexFile : Hash‑based index for fast key or time‑range lookups (≈400 MB per file, 20 M entries).

The design separates data (CommitLog) from indexes (ConsumeQueue, IndexFile), enabling high write throughput and fast consumption.

17. File I/O Strategy

RocketMQ leverages OS page cache, sequential reads, and zero‑copy via MappedByteBuffer. CommitLog writes are memory‑mapped; flushing uses MappedByteBuffer.force(). Synchronous flush waits for the disk write; asynchronous flush triggers a background thread.

18. Load Balancing

Client‑side load balancing works as follows:

Producer : After obtaining TopicPublishInfo from the NameServer, the client selects a MessageQueue via selectOneMessageQueue(). An optional latency‑fault tolerance switch avoids brokers with recent failures.

Consumer : A RebalanceService thread runs every 20 s. It retrieves the list of queues for a topic, the list of consumer IDs in the group, sorts both, and applies an average‑allocation algorithm to assign queues to consumers. The assignment updates ProcessQueue structures, creates pull requests, and dispatches them to the PullMessageService.

19. Long‑Polling Pull

If a pull request finds no data, the broker can suspend the request (long‑polling) until new messages arrive or a timeout expires. The PullRequestHoldService periodically checks held requests and wakes them when data becomes available.

References

《RocketMQ实战与原理解析》

《RocketMQ技术内幕》

Juejin article on RocketMQ interview questions

艾小仙《我要进大厂》

http://dreamcat.ink/java-interview/docs/knows/classify/dis/RocketMQ/

《浅入浅出》‑ RocketMQ

十二张图，踹开消息队列的大门

mq的那些破事儿，你不好奇吗?

消息幂等（去重）方案

七万字，151张图，通宵整理消息队列核心知识点

极客时间《消息队列高手课》

RocketMQ 官方文档