7 Real-World Message Queue Patterns Every Backend Engineer Should Know

In the author's view, message queues, caching, and sharding/partitioning form the three swords for solving high‑concurrency problems. The author has worked with ActiveMQ, RabbitMQ, Kafka, and RocketMQ, and now shares seven classic message‑queue application scenarios.

1. Asynchronous & Decoupling

A user‑service that handles registration, query, and update originally sent SMS directly after registration. If the SMS gateway was unstable, the registration call would block, degrading front‑end experience, and any change to the SMS API required core‑service modifications. By introducing a message queue, the service publishes a registration‑success event and returns immediately (asynchronous), while a separate consumer reads the event and invokes the SMS service (decoupling).

Asynchronous: The user service saves the user record, pushes a message to the queue, and immediately returns to the front‑end, avoiding long‑running SMS calls.

Decoupling: A dedicated task service consumes the message and calls the SMS provider, separating core business logic from auxiliary functions.

2. Peak Shaving

In high‑traffic spikes, sudden request bursts can overwhelm databases or exhaust CPU/IO. The author describes a ride‑hailing order‑modification flow where the service updates a cache, sends a message to MetaQ, and a persistence service consumes the message, validates order consistency, and finally writes to the database. Because consumer concurrency is bounded, the database load stays smooth and the front‑end experiences stable response times.

3. Message Bus

A message bus works like a motherboard data bus, providing a standard communication interface so components do not talk to each other directly. In a lottery order system, a scheduling center service maintains order state and interacts with downstream services (ticketing gateway, prize‑calculation) via a message queue, preventing each subsystem from directly updating the central order table and keeping the architecture loosely coupled.

4. Delayed Tasks

When a user places an order in a food‑delivery app but does not pay within a timeout, the order should be automatically cancelled. Using delayed messages, the order service publishes a delayed message to the queue; when the delay expires, the consumer checks the order status and, if unpaid, triggers cancellation logic.

RocketMQ 4.x producer code for a delayed message:

Message msg = new Message();
msg.setTopic("TopicA");
msg.setTags("Tag");
msg.setBody("this is a delay message".getBytes());
// set delay level 5, which corresponds to 1 minute
msg.setDelayTimeLevel(5);
producer.send(msg);

RocketMQ 4.x supports 18 predefined delay levels, configurable via messageDelayLevel on the broker. RocketMQ 5.x adds arbitrary timestamp‑based delays with three new APIs.

5. Broadcast Consumption

Broadcast consumption delivers each message to every consumer in a cluster, guaranteeing that each consumer processes the message at least once. It is commonly used for message push and cache synchronization.

5.1 Message Push

In a ride‑hailing platform, after a user places an order, the dispatch system assigns the order to a driver and pushes a notification to the driver’s app via a TCP‑based push service. The push service acts as a consumer of a broadcast‑consumed topic, maintaining long‑lived TCP connections for each driver.

5.2 Cache Synchronization

In high‑concurrency scenarios, applications often keep frequently accessed dictionaries in local caches (e.g., HashMap, Guava, Caffeine). When a dictionary changes, a message is broadcast via RocketMQ; each node consumes the message and refreshes its local cache, ensuring consistency without direct database queries.

6. Distributed Transactions

In an e‑commerce checkout, a user’s payment triggers downstream actions such as logistics, points update, and cart clearing. Traditional XA transactions guarantee consistency but suffer from low concurrency and high lock contention. Plain message approaches can lead to inconsistencies (e.g., message sent but order failed, or vice‑versa). RocketMQ’s transactional messages add a two‑phase commit: the producer sends a “half‑message”, executes a local transaction, then commits or rolls back the message based on the local outcome. The broker delivers the message only after a commit, achieving eventual consistency while preserving high throughput.

Key steps:

Producer sends message to broker; broker stores it and returns ACK, marking the message as “pending delivery” (half‑transaction).

Producer executes local business logic.

Producer sends a commit or rollback decision to the broker.

If committed, broker marks the message as deliverable and pushes it to consumers; if rolled back, the message is discarded.

When the broker does not receive a decision (e.g., network failure), it performs a message check after a timeout, prompting the producer to re‑evaluate the local transaction and resend the decision.

7. Data Hub (Kafka)

Over the past decade, specialized systems such as HBase (KV store), Elasticsearch (search), Storm/Spark/Samza (stream processing), and OpenTSDB (time‑series) have emerged. Often the same raw data must be ingested into multiple systems. Using Kafka as a data hub, a single log stream can be routed to all downstream stores.

The pipeline consists of:

Log‑collection client that batches and asynchronously sends logs to Kafka, minimizing impact on the source service.

Kafka persisting the logs in its log files.

Log‑processing applications (e.g., Logstash, custom consumers) that read from Kafka and forward data to search services, Hadoop, or other big‑data platforms.