Mastering RocketMQ: Core Concepts of Topics, Queues, and Messaging

1 Introduction

RocketMQ is a distributed messaging middleware whose core technology involves several key concepts, including topics, queues, messages, producers, consumers, consumer groups, and subscription relationships.

2 Core Technology Implementation

2.1 Topic

Definition: A topic is the top‑level logical container for message transmission and storage in RocketMQ, used to identify messages belonging to the same business logic type. It is a logical concept, not a physical container.

Purpose:

Data classification isolation: RocketMQ recommends separating different business data into different topics for isolated storage and subscription.

Data identity and permission management: Messages are anonymous; topics are used to identify and manage permissions for messages of the same classification.

Name: The topic name uniquely identifies the topic within the cluster.

Queue list: A topic consists of one or more queues, which are the actual containers where messages are stored.

Message types: When creating a topic, you can specify the message types it stores, such as Normal, FIFO, Delay, and Transaction.

2.2 Queue (MessageQueue)

Definition: A queue is the actual container for message transmission and storage in RocketMQ, and the smallest storage unit for messages.

Features:

All topics are composed of multiple queues, enabling horizontal sharding and streaming storage.

Queues inherently preserve order: messages are written to storage in the order they enter the queue, establishing a natural sequence.

The position of a message and its order are managed via an offset.

2.3 Message

Definition: A message is the smallest data transmission unit in RocketMQ; producers package data into messages and send them to the RocketMQ server.

Characteristics:

Immutability: Once created, a message’s content cannot be changed.

Persistence: RocketMQ persists messages to server storage by default, ensuring traceability and recoverability in failure scenarios.

2.4 Producer

Definition: A producer is responsible for sending messages to a specified topic and queue in RocketMQ.

Sending modes:

Synchronous sending: Returns a result and waits for the send to succeed.

Asynchronous sending: Returns immediately after sending, while the send status is received asynchronously.

One‑way sending: Sends the message without waiting for any response.

2.5 Consumer

Definition: A consumer subscribes to and consumes messages from a specified topic in RocketMQ.

Consumption process:

The consumer pulls messages from the broker to the client, then starts consumer threads to process them.

PullMessageService continuously pulls messages from the broker, while ConsumeMessageService processes the messages.

2.6 Consumer Group

Definition: A consumer group manages a set of consumers that share the load of consuming messages.

Purpose:

Enables load balancing and fault tolerance for message consumption.

All consumers within the same group must maintain consistent consumption behavior.

2.7 Subscription

Definition: A subscription defines the relationship between a consumer group and a topic, including filtering rules for message consumption.

Purpose:

Allows consumers to specify which topics and message types to consume.

RocketMQ uses subscription relationships to filter and dispatch messages.

3 Summary

In summary, RocketMQ’s core technology comprises multiple interrelated concepts that collaborate to deliver high performance, high availability, and high scalability.