Unveiling RocketMQ: How Messages Journey Through Storage, Delivery, and Expiration

When using RocketMQ we often focus on whether messages can be sent or consumed, but rarely consider these questions:

What stages does a message go through inside RocketMQ?

Who is responsible for storing, pushing, and cleaning up these messages?

How is message expiration determined? Does a sent message stay forever in the queue?

What happens to messages if a broker node crashes?

This article does not discuss APIs; instead it dissects RocketMQ messages from six key dimensions: message roles, storage, push, expiration, retry, and cluster switching.

Message Roles

1. Broker – receives and sends messages, i.e., the queue itself.

2. Nameserver – a registration center responsible for service discovery and registration of producers, brokers, and consumers; it can be deployed in a cluster where brokers do not communicate directly.

3. Producer – obtains topic routing info from the nameserver and establishes a long connection with the master broker to write data (only the master can write).

4. Consumer – obtains topic routing info from the nameserver and connects to master/slave brokers to pull messages.

How Messages Are Stored

RocketMQ stores all messages on disk.

1. Messages are first written to a commitlog file located at $HOME/store/commitlog, stored sequentially.

2. To enable fast lookup, an index file called consumequeue is created per topic and queue, e.g., ../rocketmq-master-1/store/consumequeue/pay-test/2/, containing offset, size, and hash.

3. For key‑based or timestamp queries, an index file (similar to a database index) is maintained under ./rocketmq-master-1/store/index.

$HOME/store/consumequeue/{Topic}/{QueueId}/{file}

How Messages Are Pushed

RocketMQ supports both pull and push consumption modes.

1. Pull mode – the consumer actively pulls messages, suitable for batch processing or rate control, but the consumer must manage offsets.

2. Push mode – the broker actively pushes messages, ideal for low‑latency scenarios like order data, though it may overload the consumer.

SpringBoot registers consumers via annotations, which internally use a push‑style implementation based on long polling:

while(true){
    log.info("hello");
    // simple long‑polling via thread sleep
    Thread.sleep(1000);
}

In reality the push mode is a wrapper around pull requests; the broker holds the request thread (long polling) until a message arrives or a timeout occurs.

Message Expiration

All messages reside on disk, so expired messages must be periodically deleted, or deletion is triggered when disk space is low. Two strategies are used:

At 4 AM daily, a task deletes old commitlog files.

Every 10 seconds, a scheduled task removes stale consumequeue and index files.

private boolean isTimeToDelete(){
    String when = this.getMessageStoreConfig().getDeleteWhen();
    SimpleDateFormat sdf = new SimpleDateFormat("HH");
    String now = sdf.format(new Date());
    return now.equals(when);
}

Message Retry Queue

If a consumer fails to acknowledge a message (throws an exception), the message enters a retry queue and the broker attempts redelivery up to 16 times with increasing back‑off intervals (1 s, 5 s, 10 s, …, 30 m). After the final retry the message moves to the dead‑letter queue (DLQ) for manual handling.

1s, 5s, 10s, 30s, 1m, 2m, 3m, 4m, 5m, 6m, 7m, 8m, 9m, 10m, 20m, 30m

It is recommended to include original metadata (topic, tag, business type) in the message body to facilitate recovery from the DLQ, or to forward failed messages to a secondary DLQ or persistent storage.

Cluster Switching Mechanism

1. Message loss prevention – configure the flush disk type. flushDiskType=ASYNC_FLUSH # or SYNC_FLUSH SYNC_FLUSH writes to disk immediately, providing high reliability at the cost of latency. ASYNC_FLUSH returns ACK instantly and writes to disk asynchronously, which may lose messages if the broker crashes before the async write completes.

2. High‑availability – set the broker role. brokerRole = ASYNC_MASTER # or SYNC_MASTER SYNC_MASTER synchronously replicates messages to slave brokers before acknowledging the producer; ASYNC_MASTER acknowledges immediately and replicates asynchronously, risking minor loss on failure.

3. Automatic master‑slave failover – from RocketMQ 4.5 onward, DLedger (Raft‑based) mode provides automatic leader election and high availability.

If you master these mechanisms, you have moved beyond basic usage to reliable, high‑performance operation of RocketMQ.