Mastering RocketMQ: 7 Core Techniques for Reliable Messaging

1 Introduction

ArchitePlus has been writing about RocketMQ for a while, and this series contains a wealth of technical knowledge that helps us use message middleware effectively.

Reference articles:

RocketMQ Series: Message Order (Lecture 6)

RocketMQ Series 5: Delayed Message Detailed (Graphic Summary)

How to Solve MQ Message Backlog (Three Solutions)

RocketMQ Transactional Messages: Ultimate Consistency Solution (Lecture 8)

Deep Dive into RocketMQ Retry Mechanism: Zero Message Loss (Lecture 9)

RocketMQ Message Storage Mechanism (Reliability Tool)

MQ Series 16: Implementing Message Filtering

2 Seven Core Knowledge Points

2.1 Message Order Guarantee

Apache RocketMQ Message Lifecycle States:

1. Initialization: Producer creates the message, not yet sent to the server. 2. Pending Consumption: Message reaches the server and is visible for pull. 3. Consuming: Consumer fetches the message and processes it; the server waits for the result (timeout triggers retry). 4. Consumption Commit: Consumer reports the result; the server marks the message as success/failure while retaining the data for possible replay. 5. Message Deletion: Expired messages are periodically cleaned according to storage policies.

2.2 Delayed / Scheduled Messages

Sending delayed messages in RocketMQ is straightforward: set a delay level (1 s to 2 h) or specify an exact timestamp (RocketMQ 5.0+).

Message message = newMessage("OrderTopic", "订单超时提醒".getBytes());
message.setDelayTimeLevel(3); // level 3 = 10 seconds delay
producer.send(message);

All 18 delay levels:

1s 5s 10s 30s 1m 2m 3m 4m 5m 6m 7m 8m 9m 10m 20m 30m 1h 2h

Custom timestamp example (10 minutes later):

long delayTime = System.currentTimeMillis() + 10 * 60 * 1000; // 10 minutes later
message.setDeliverTimestamp(delayTime);

Consumer simply subscribes to the target topic:

@RocketMQMessageListener(topic = "OrderTopic", consumerGroup = "order_group")
public class OrderConsumer implements RocketMQListener<String> {
    @Override
    public void onMessage(String message) {
        System.out.println("Received delayed message: " + message);
        // handle order timeout logic...
    }
}

Illustrates a 30‑minute unpaid order cancellation scenario.

2.3 Message Backlog Analysis and Optimization

Producer‑side optimizations: 🍯 Apply rate limiting (token bucket, leaky bucket) during traffic spikes. 🍯 Off‑load non‑time‑critical requests to delayed queues.

MQ‑side cluster optimizations: 🍪 Increase the number of Brokers and evenly distribute topics/partitions. 🍪 Upgrade hardware: network bandwidth, SSDs, memory, CPU.

Consumer‑side optimizations: 🧉 Reduce per‑message processing time, avoid heavy computation and frequent DB access. 🧉 Use caching and asynchronous handling. 🧉 Employ multithreaded consumption where appropriate.

2.4 Transactional Message Guarantees

RocketMQ supports distributed transactional messages with a two‑phase commit and rollback mechanism to ensure eventual consistency.

The process includes:

Message sending by the producer.

Half‑transaction (prepare) phase: broker stores the half message in RMQ_SYS_TRANS_HALF_TOPIC and returns an ACK.

Local transaction execution on the producer side.

Second‑phase confirmation: commit moves the message to a normal topic; rollback deletes the half message.

Message check (retry) when the state is unknown, triggered by network issues or restarts.

Transaction check handling by the producer.

Repeat second‑phase confirmation if needed.

2.5 Retry Mechanism: Zero Message Loss

When sending fails due to network jitter, broker failure, or storage errors, RocketMQ automatically retries. The strategy includes a retry matrix, broker selection policy, and flow‑control handling.

Scenario

Synchronous Send

Asynchronous Send

Default retry count

2 (configurable up to 3)

0 (manual enable)

Broker selection

Switch to another broker on failure

Retry on the same broker

Flow control

Exponential backoff (initial 1 s)

Fast fail + downgrade

Code example:

// Synchronous send: 3 retries
producer.setRetryTimesWhenSendFailed(3);
// Asynchronous send: enable retry
producer.setRetryTimesWhenSendAsyncFailed(2);
// Switch broker on storage failure
producer.setRetryAnotherBrokerWhenNotStoreOK(true);

Retry interval follows a step‑wise increase (e.g., 10 s, 30 s, 1 min, … up to 2 h for >16 attempts). Sequential messages use a fixed 3 s interval to preserve order.

2.6 Message Storage Mechanism

RocketMQ retains messages based on a configured storage duration, independent of consumption status. After the retention period expires, messages are cleaned up.

Key points:

Retention is managed per storage node.

Time‑based criteria determine whether a message stays.

Consumption state does not affect storage.

Illustrates the storage lifecycle of messages in a queue.

2.7 Message Filtering

2.7.1 Requirement

In real‑world scenarios, multiple downstream services consume the same topic but need only a subset of messages. Filtering reduces unnecessary traffic and processing load.

2.7.2 Implementation

Filtering works via three steps:

Producer adds tags or properties to messages.

Consumer registers filter criteria when subscribing.

Broker dynamically matches messages against the criteria and delivers only matching ones.

2.7.3 Filtering Types

Comparison

Tag Filtering

SQL Property Filtering

Target

Message Tag

Message attributes (including tags)

Capability

Exact match

SQL‑like expression

Use case

Simple, lightweight filtering

Complex, expressive filtering

3 Summary

The seven core techniques described above enable developers to handle complex messaging scenarios with confidence and efficiency.