RocketMQ Core Concepts, Windows Installation, Configuration, and Best Practices

Introduction

Hello, I am Wukong. This edition focuses on the core knowledge of RocketMQ. Please bookmark and study it gradually.

Windows Installation

Download URL:
https://www.apache.org/dyn/closer.cgi?path=rocketmq/4.5.2/rocketmq-all-4.5.2-bin-release.zip

Select the Binary package to download and unzip the downloaded archive.

Configuration

Add system variables: ROCKETMQ_HOME → F:\RocketMQ\rocketmq-4.5.2 JAVA_HOME → F:\Java_JDK\JDK1.8 Update Path to include Maven/bin directory Note: RocketMQ stores messages in C:\Users\Administrator\store ; the files can be large, delete unnecessary data.

Startup

Run start mqnamesrv.cmd Run start mqbroker.cmd -n 127.0.0.1:9876 autoCreateTopicEnable=true

Integrating the Visual Monitoring Plugin

git clone https://github.com/apache/rocketmq-externals.git
cd rocketmq-externals\rocketmq-console\src\main\resources
# Edit application.properties, set server.port=8100 and rocketmq.config.namesrvAddr=127.0.0.1:9876
cd ..\rocketmq-externals\rocketmq-console
mvn clean package -Dmaven.test.skip=true
java -jar rocketmq-console-ng-1.0.1.jar
# Access http://127.0.0.1:8100

Rocket visual monitoring plugin adds Topic | Auto‑create Topic (4.5.2 version)

SpringBoot Integration

<!-- RocketMQ dependency -->
<dependency>
    <groupId>org.apache.rocketmq</groupId>
    <artifactId>rocketmq-client</artifactId>
    <version>4.5.2</version>
</dependency>

RocketMQ Basic Concepts

Overview

A RocketMQ‑based distributed system typically consists of four clusters: NameServer, Broker, Producer, and Consumer.

NameServer – lightweight service discovery and routing.

Broker – provides topic/queue storage, supports push/pull, high availability via replication, and handles billions of messages.

Producer – distributes messages with load‑balancing, fast failure, low latency.

Consumer – supports push and pull, cluster or broadcast consumption.

Core Modules

NameServer

Provides broker management and routing services.

The NameServer is a stateless discovery service; multiple instances do not communicate. All producers, consumers, and brokers interact with every NameServer via one‑way requests, which makes horizontal scaling easy.

Broker

Offers remoting, client management, storage, HA, and indexing.

Stores messages for topics and queues.

Supports push and pull modes.

High availability through 2‑ or 3‑copy replication.

Can hold billions of pending messages.

Provides fault recovery, statistics, and alerting.

Producer

Distributed deployment with load‑balancing.

Fast failure handling.

Low latency.

Consumer

Supports push and pull.

Supports cluster and broadcast consumption.

Core Roles

Producer

Produces messages to brokers; supports synchronous, asynchronous, and one‑way sending.

Producer Group

Producers with the same role are grouped; if one instance crashes, others in the group can continue the transaction.

Consumer

Consumers pull messages from brokers. Two types exist: pull‑type (client initiates) and push‑type (broker pushes via callback).

Consumer Group

Consumers with the same role are grouped to achieve load balancing and fault tolerance. All instances in a group subscribe to the same topic.

Topic

A logical channel where producers publish and consumers subscribe. Multiple consumers can read the same topic.

Message

Each message belongs to a topic and may contain optional tags and key‑value pairs for filtering and debugging.

Message Queue

Topics are split into one or more queues. Queues can be synchronous master, asynchronous master, or slave. For strict reliability, use synchronous master + slave.

Tag

Sub‑topic label that provides additional filtering flexibility.

Broker (Queue)

Stores messages, metadata, consumer offsets, and topic/queue information.

Name Service

Provides routing information for producers and consumers to locate topics and queues.

Message Ordering

When using DefaultMQPushConsumer , you must decide between ordered consumption and concurrent consumption.

Ordered consumption: messages are processed in the same order they were stored; requires a single queue per topic.

Concurrent consumption: multiple instances consume in parallel; ordering is not guaranteed.

Best Practices

Producer Best Practices

Use a single topic per application; differentiate sub‑types with tags. Only messages with tags can be filtered by consumers.

Set a unique business identifier in the keys field for traceability; ensure high uniqueness to avoid hash collisions.

Log send results and keys; for critical data, implement retry mechanisms on failure.

When delivery confirmation is not required, use sendOneWay for fire‑and‑forget sending.

Consumer Best Practices

Ensure idempotent processing to avoid duplicate effects.

Prefer batch consumption to improve throughput.

Optimize the processing logic of each message.

MQ Core Issues

1. Problems Message Queues Solve

Asynchrony, decoupling, and traffic‑shaping (peak‑shaving). Introducing a queue also adds complexity such as duplicate delivery, ordering, loss, and retry handling.

2. Message Models: Topic vs Queue

Early queues followed a strict FIFO model. Publish‑Subscribe introduced topics, allowing multiple consumers to receive the same message without the producer needing to know the consumer count.

3. Message Loss and Reliability

Reliability is ensured through a three‑stage acknowledgment mechanism: producer receives a send‑ack, broker writes to disk (sync flush for high reliability), and consumer sends a consume‑ack after successful processing. Configuring flushDiskType=SYNCHRONOUS_FLUSH and multi‑node replication reduces loss risk.

4. Handling Duplicate Messages

Duplicate messages arise from retries. Use idempotent operations, database unique constraints, conditional updates, version numbers, or token/GUID checks to achieve exactly‑once semantics.

5. Transactional Messages for Distributed Transactions

Transactional messages ensure consistency between producer and consumer actions. A producer sends a “half‑message”, executes a local transaction, then commits or rolls back the message. RocketMQ provides a transaction‑status‑check mechanism for network failures.

6. Ordering in Message Queues

Ordering can be defined by absolute timestamps or causal relationships. In distributed systems, same‑thread events have a clear order; cross‑thread ordering must be inferred from causality. RocketMQ guarantees order by routing related messages to the same partition and ensuring a single consumer thread processes each partition.

FIFO (ordered) messages consist of ordered publishing and ordered consumption. Two types exist: partition‑level order and global order.

Producer Side Ordering

Route messages to a specific partition using MessageQueueSelector (e.g., orderId % mqs.size() ).

Consumer Side Ordering

PushConsumer obtains a lock on the MessageQueue , processes messages from the ProcessQueue in FIFO order, and ensures only one thread consumes a given partition.

Limitations

Ordered messages cannot leverage fail‑over retries across partitions.

Hot‑spot partitions may cause load imbalance.

Parallelism is limited by the number of partitions.

Failed ordered messages cannot be skipped without risking downstream inconsistency.