Core Concepts and Common Patterns of RabbitMQ

Core Concepts of RabbitMQ

Hello, I am Wukong. This article introduces the core knowledge of RabbitMQ.

1. Why Message Queues Exist

Message middleware is mainly used for asynchronous processing, application decoupling, traffic shaping, and log handling.

1.1 Asynchronous Processing

When a user registers, three solutions exist: serial execution, parallel execution of email/SMS, or asynchronous processing via MQ. The recommended approach is to push the registration event to a queue and let email and SMS services consume it.

1.2 Application Decoupling

For example, an order system calls an inventory system. If the inventory service fails, the order service can still operate because the two are decoupled via a queue.

1.3 Traffic Shaping (Peak‑Smoothing)

During a flash‑sale event, a queue is inserted to absorb the burst of requests, preventing the service layer from being overwhelmed.

1.4 Log Processing

Web UI requests can be routed through a queue for centralized log handling.

1.5 Drawbacks of MQ

Reduced System Availability : Introducing a third‑party component adds dependency on its stability.

Increased System Complexity : Issues such as consistency, duplicate consumption, and reliable delivery must be handled.

2. Common Message Queues

Popular middleware includes ActiveMQ, RabbitMQ, RocketMQ, and Kafka.

ActiveMQ – legacy, suited for traditional enterprises.

RabbitMQ – high concurrency, high throughput, Erlang‑based, supports clustering.

RocketMQ – Alibaba’s Java‑based solution, high performance.

Kafka – ultra‑high throughput for real‑time log collection, often paired with Spark Streaming or Flink.

3. RabbitMQ Common Patterns

RabbitMQ is an open‑source AMQP implementation written in Erlang. It supports multiple clients and excels in usability, scalability, and high availability.

3.1 Basic Concepts

Broker : the server that hosts queues.

Exchange : routes messages to queues based on rules.

Queue : stores messages.

Binding : links an exchange to a queue with a routing rule.

Routing Key : the key used by an exchange to match a queue.

VHost : a virtual broker that isolates resources and permissions.

Producer : publishes messages.

Consumer : receives messages.

Channel : a virtual connection inside a TCP connection, allowing multiple independent sessions.

3.2 Working Modes

3.2.1 Simple Mode

One producer, one consumer. The consumer must acknowledge messages; otherwise the queue can fill up.

3.2.2 Work (Competing Consumers) Mode

Multiple consumers share a queue. Two distribution strategies:

Round‑Robin : messages are distributed evenly regardless of consumer load.

Fair Dispatch : a consumer receives the next message only after it acknowledges the previous one.

3.2.3 Fanout (Publish/Subscribe) Mode

Messages are broadcast to all bound queues; routing keys are ignored.

3.2.4 Direct Routing Mode

Messages are delivered only to queues whose binding key exactly matches the routing key.

3.2.5 Topic Mode

Routing keys support wildcards:

# matches one or more words.

* matches exactly one word.

This enables flexible pattern‑based routing.

4. Message Routing, Reliability, and Advanced Features

4.1 Routing Basics

Producers attach a routing key to a message; exchanges use the key to route to bound queues (fanout, direct, topic).

4.2 Transport Layer

Channels are virtual connections built on a single TCP socket, allowing thousands of concurrent logical streams while avoiding the overhead of many TCP connections.

4.3 Preventing Message Loss

Producer Side : use transactions (txSelect/txCommit) or confirm mode (channel.confirmSelect) to get ACK/NACK.

Queue Persistence : declare queues with durable=true and publish messages with deliveryMode=2 (persistent).

Consumer Side : disable auto‑ack and manually acknowledge after successful processing.

4.4 Maintaining Message Order

Order can be broken when multiple consumers share a queue or when a consumer processes messages concurrently. Solutions include:

Hashing related messages to the same queue.

Using a single consumer that internally dispatches to ordered in‑memory queues.

4.5 Clustering

RabbitMQ offers three deployment modes:

Standalone : single node, suitable only for demos.

Normal Cluster : metadata is shared; queues reside on a single node but can be accessed from any node.

Mirrored Cluster : queues and their data are replicated across multiple nodes for high availability, at the cost of network bandwidth.

4.6 Dead‑Letter and Delayed Queues

Messages become dead letters when they are rejected, expire, or exceed queue length limits. A dead‑letter exchange routes them to a designated dead‑letter queue.

TTL (Time‑To‑Live) can be set on queues or individual messages. Example code:

Map<String, Object> args = new HashMap<String, Object>();
args.put("x-message-ttl", 6000); // ms
channel.queueDeclare(queueName, durable, exclusive, autoDelete, args);

AMQP.BasicProperties.Builder builder = new AMQP.BasicProperties.Builder();
builder.expiration("6000");
AMQP.BasicProperties properties = builder.build();
channel.basicPublish(exchangeName, routingKey, mandatory, properties, "msg body".getBytes());

Delayed queues are implemented by combining a dead‑letter exchange with TTL.

4.7 Handling Message Backlog

Temporary scaling steps include fixing consumer issues, creating many parallel queues, deploying a fast‑consumer to redistribute messages, and adding more consumer instances to process the new queues.

4.8 Push vs Pull

Push (basicConsume) : the broker pushes messages to the consumer continuously; low latency but requires a buffer.

Pull (basicGet) : the consumer explicitly fetches a message; higher latency and lower throughput.

4.9 Designing a Message Queue System

Key considerations: scalability (adding partitions like Kafka), sequential disk writes for performance, high availability via replication and leader election, and guaranteeing zero message loss.

References

Message order consumption: https://www.jianshu.com/p/02fdcb9e8784

RabbitMQ dead‑letter queue: https://www.cnblogs.com/mfrank/category/1514703.html

RabbitMQ message push: https://blog.csdn.net/qq_40837310/article/details/109033000