RocketMQ Overview: Introduction, Evolution, Architecture, Key Features, and Application Scenarios

01 RocketMQ Introduction

RocketMQ is a pure‑Java, distributed, queue‑model open‑source messaging middleware originally known as MetaQ, developed by Alibaba and later donated to the Apache Foundation as a top‑level project, offering high performance, high reliability, real‑time processing, and distributed capabilities.

02 RocketMQ Evolution

RocketMQ has undergone three generations of evolution:

1. First generation – Push model : data stored in relational databases (e.g., Notify, Napoli).

2. Second generation – Pull model : proprietary message store inspired by Kafka, represented by MetaQ.

3. Third generation – Pull‑dominant with optional push : low‑latency engine adding reliable retry, file‑based distributed transactions, and supporting trillion‑level message flows in Alibaba’s e‑commerce scenarios.

03 RocketMQ Architecture Design

Core components :

RocketMQ consists of four main parts: NameServer, Broker, Producer, and Consumer.

1) NameServer : manages topics and routing information; lightweight compared with Zookeeper, with independent nodes that do not exchange state.

2) Producer : generates messages, typically deployed by business systems; supports multiple load‑balancing strategies, low‑latency sending, and fast failure.

3) Broker : stores and forwards messages; each broker maintains long‑lived connections with all NameServers, registers its topics, uses Netty for communication, supports thousands of queues per instance, and guarantees ordering and high throughput.

4) Consumer : consumes messages, usually by backend services; supports both PUSH and PULL modes, clustering, broadcasting, and real‑time subscription.

Message flow : on startup, brokers register with NameServers and send heartbeats; producers fetch broker addresses from NameServers and send messages to the appropriate broker.

04 RocketMQ Message Domain Model

The model includes Message, Topic, Tag, Queue, Offset, and Group.

Topic : first‑level classification of messages (e.g., transaction, logistics); a message must belong to a topic.

Tag : second‑level classification within a topic (e.g., order created, order completed).

Group : a consumer group can subscribe to multiple topics.

Message Queue : physical storage unit; a topic can have many queues, enabling horizontal scaling. Queues are persisted, virtually infinite in length, accessed via a 64‑bit offset.

05 Key Features of RocketMQ

1. Ordered messages : messages with the same ID are sent to the same queue, and a single consumer processes that queue, ensuring consumption order.

2. Message duplication : RocketMQ follows QoS models – At most once, At least once, Exactly once. Most MQs provide at‑least‑once delivery, which can cause duplicates under network failures.

Deduplication is not built‑in; it relies on application‑level idempotency, unique identifiers, and database constraints (e.g., primary key conflicts) to filter repeated messages.

06 Application Scenarios

1. Peak shaving (削峰填谷) : handling traffic spikes such as flash sales by buffering requests in the high‑throughput MQ.

2. Asynchronous decoupling : isolating downstream services (e.g., points system) from upstream transaction systems to maintain core stability.

3. Ordered messages (FIFO) : ensuring sequential processing for order creation, payment, refund, etc.

4. Distributed transaction messages : splitting large transactions into smaller MQ‑driven steps, leveraging at‑least‑once semantics and multi‑replica reliability to achieve eventual consistency.

—