Skip to content
Message Queues Explained: Principles, Architecture & Selection Guide

Message Queues Explained: Principles, Architecture & Selection Guide

This article comprehensively explains message queue fundamentals including decoupling, async communication, reliability, and buffering; details architecture roles, queue types, persistence, acknowledgment, and protocols; compares ActiveMQ, RabbitMQ, Kafka, RocketMQ, and Pulsar with performance metrics; and provides selection guidance for e-commerce, finance, and big data scenarios.

Mike Chen's Internet Architecture
Mike Chen's Internet Architecture
Mike Chen's Internet Architecture

What Is a Message Queue (MQ)?

MQ (Message Queue) is a container that stores messages, essentially a queue. It enables asynchronous communication between applications by introducing middleware between sender and receiver, achieving decoupling, async processing, and reliable message delivery.

Key Characteristics of MQ

Decoupling : Sender only sends to queue without knowing receiver's existence or state.

Asynchronous Communication : Sender continues after sending, improving response speed and throughput.

Reliability : Mechanisms like acknowledgment and retry ensure messages are not lost.

Buffering and Peak Shaving : Queue acts as buffer when production speed exceeds consumption, smoothing traffic spikes and preventing overload.

MQ Design Concepts

1. Overall Architecture

Three roles:

Producer : Creates and sends messages to Broker.

Broker : Message processing center handling storage, acknowledgment, retry; contains multiple queues.

Consumer : Receives and processes messages from queue.

Overall architecture diagram
Overall architecture diagram

2. Message Queue Types

Point-to-Point Model

Producer sends to a queue; one consumer receives each message. Roles: Queue, Sender, Receiver.

Point-to-point model diagram
Point-to-point model diagram

Publish-Subscribe Model

Publisher sends to a topic; multiple subscribers receive. Roles: Topic, Publisher, Subscriber.

Publish-subscribe model diagram
Publish-subscribe model diagram

Difference

Point-to-point: one message, one consumer. Pub/sub: message delivered to all subscribers of the topic.

3. Message Persistence

Queues support persistent storage to prevent message loss after restart or failure.

4. Message Acknowledgment

Consumer sends acknowledgment after successful processing; queue then deletes the message.

5. Message Transport Protocols

Protocols include AMQP, MQTT, STOMP.

Major Message Queue Products

1. ActiveMQ

Java-based open-source middleware implementing JMS spec. Apache project, early product, slower updates, performance ~10K msgs/sec.

ActiveMQ logo
ActiveMQ logo

Official site: activemq.apache.org

2. RabbitMQ

Open-source, implements AMQP standard. Written in Erlang, leveraging Erlang's concurrency; supports AMQP, XMPP, SMTP, STOMP. Performance ~10K msgs/sec.

RabbitMQ logo
RabbitMQ logo
RabbitMQ architecture diagram
RabbitMQ architecture diagram

Official site: www.rabbitmq.com

3. Kafka

High-throughput, distributed, persistent messaging system. Originally developed by LinkedIn, open-sourced 2011. Performance ~1M msgs/sec.

Kafka logo
Kafka logo
Kafka architecture diagram
Kafka architecture diagram

Official site: kafka.apache.org

4. RocketMQ

Alibaba open-source, pure Java. High throughput, high availability, suited for large-scale distributed systems. Performance ~100K msgs/sec. Designed with reference to Kafka, used in financial scenarios.

RocketMQ logo
RocketMQ logo
RocketMQ architecture diagram
RocketMQ architecture diagram

Official site: rocketmq.apache.org

5. Pulsar

Distributed messaging and streaming platform by Apache Foundation. Cloud-native, multi-tenant, persistent storage, scalable, high throughput for large-scale real-time data processing.

Pulsar logo
Pulsar logo
Pulsar architecture diagram
Pulsar architecture diagram

Official site: pulsar.apache.org

Message Queue Selection Guide

E-commerce, finance, high transaction requirements: consider RocketMQ.

Lower technical complexity: RabbitMQ is a good choice.

Big data real-time computing, log collection: consider Kafka.

Original Source

Signed-in readers can open the original source through BestHub's protected redirect.

Sign in to view source
Republication Notice

This article has been distilled and summarized from source material, then republished for learning and reference. If you believe it infringes your rights, please contactadmin@besthub.devand we will review it promptly.

KafkaMessage QueueMQRabbitMQRocketMQPulsarActiveMQSelection Guide
Reader feedback

How this landed with the community

Sign in to like

Rate this article

Was this worth your time?

Sign in to rate
Discussion

0 Comments

Thoughtful readers leave field notes, pushback, and hard-won operational detail here.