Comparison of Four Popular Open-Source Distributed Message Queues: Kafka, ActiveMQ, RabbitMQ, and RocketMQ

Message queue middleware is a crucial component in distributed systems, addressing application coupling, asynchronous messaging, and traffic shaping. It enables high performance, high availability, scalability, and eventual consistency, and is indispensable in large-scale architectures such as e‑commerce, logging, and communication platforms.

1. Kafka

Apache Kafka, originally developed by LinkedIn and donated to the Apache Foundation in 2010, is a distributed streaming platform.

Kafka Overview

Official definition: "Apache Kafka is a distributed streaming platform."

Core Capabilities

Publish and subscribe to record streams, similar to a messaging system.

Fault‑tolerant, persisting streams to storage.

Real‑time processing of streams as they are generated.

Typical Use Cases

Building reliable real‑time data pipelines between systems.

Developing real‑time stream processing applications.

Core APIs

Producer API – publish records to one or more topics.

Consumer API – subscribe to topics and process records.

Streams API – process input streams and produce output streams.

Connector API – integrate Kafka with external systems (e.g., databases).

Key Features

Fast persistence with O(1) overhead.

High throughput (up to 100k messages/s per server).

Fully distributed with automatic load balancing.

Supports synchronous and asynchronous replication.

Zero‑copy I/O, transparent scaling and expansion.

Deployment Requirements

Operating System

Windows – supported but not recommended.

Unix/Linux – fully supported.

Environment

JDK 1.8+ (Kafka 2.0.0+).

Zookeeper – required for cluster coordination (future versions aim to remove this dependency).

Architecture

A typical Kafka cluster consists of multiple Producers, Brokers, Consumers (grouped), and a Zookeeper ensemble that manages configuration, leader election, and consumer rebalancing. Producers use a push model, while Consumers pull messages.

High‑Availability

Kafka achieves HA through replication. Each topic is divided into partitions, each with multiple replicas. Zookeeper elects a leader for each partition; the leader handles client requests while followers replicate the data. If a follower fails, a new follower is elected; if the leader fails, a follower becomes the new leader.

Pros and Cons

Advantages

Rich client language support (Java, .NET, Python, Go, etc.).

Excellent performance (millions of messages/s).

Fully distributed with strong replication.

Batch operations, ordered consumption, and robust monitoring tools.

Disadvantages

Performance degrades with many partitions (>64 per broker).

Short‑polling can affect latency.

No built‑in retry for failed consumption.

Potential message ordering issues after broker failure.

Community updates can be slow.

2. ActiveMQ

Apache ActiveMQ is a popular open‑source messaging server supporting JMS 1.1 and J2EE 1.4.

Overview

Written in Java, the latest version is 5.1.5.6.

Features

Supports many languages (Java, C, C++, C#, Ruby, Perl, Python, PHP) and protocols (OpenWire, STOMP, REST, WS‑Notification, XMPP, AMQP).

Full JMS 1.1 compliance, persistence, XA transactions.

Enterprise integration patterns, message groups, virtual destinations.

Spring integration, JCA adapters, and extensive J2EE server support.

Multiple transport options (VM, TCP, SSL, NIO, UDP, Multicast, JGroups, JXTA).

High‑performance persistence via JDBC or journal.

Cluster modes: Network of Brokers, Master‑Slave.

REST API and AJAX support for web integration.

Deployment Requirements

Hardware

Binary installation requires ~60 MB; source build needs ~300 MB.

Operating System

Windows (XP SP2, 2000, Vista, 7) and Unix/Linux variants.

Environment

JRE 1.7+ (or JDK for source builds).

JAVA_HOME must be set.

Maven 3.0+ required for source compilation.

Architecture

High‑Availability Solutions

ActiveMQ provides several HA configurations based on the Master/Slave model:

Shared Nothing Master/Slave – simple but largely deprecated.

Shared Storage Master/Slave – uses shared databases or file systems.

Replicated LevelDB Store – newer HA approach (replaced by KahaDB).

Network Bridges extend HA to true distributed clusters, allowing multiple groups to interconnect via Network Connectors.

Pros and Cons

Advantages

Cross‑platform Java implementation.

JDBC persistence, JMS compatibility, automatic reconnection.

Security via Shiro, JAAS, and fine‑grained ACLs.

Rich monitoring (Web Console, JMX, Jolokia, Hawtio).

Disadvantages

Community activity is relatively low.

Occasional message loss issues.

Focus shifting to Apollo; limited support for thousands of queues.

3. RabbitMQ

RabbitMQ, developed by RabbitMQ Technologies Ltd (now part of VMware/Pivotal), is the reference implementation of the AMQP protocol.

Overview

Current version 3.7.8, written in Erlang, supporting many client languages (Python, Ruby, .NET, Java, JMS, C, PHP, etc.).

Key Features

Asynchronous messaging with flexible routing (direct, fanout, topic exchanges).

Excellent developer experience and cross‑platform support.

Pluggable authentication (TLS, LDAP) and robust clustering.

Rich management UI, HTTP API, and command‑line tools.

Deployment Requirements

Supported on Windows (NT to Windows 8) and Unix/Linux (Ubuntu, Fedora, openSUSE, Solaris, BSD, macOS).

Erlang runtime is required; JDK may be needed for certain plugins.

Architecture

Important concepts:

Broker – the server instance.

Exchange – routes messages based on rules.

Queue – stores messages.

Binding – links exchanges to queues.

Routing Key – determines routing.

Vhost – virtual host for multi‑tenant isolation.

Producer – sends messages.

Consumer – receives messages.

Channel – lightweight session within a connection.

High‑Availability

HA is achieved via clustering (disk nodes and memory nodes) and mirrored queues. At least one disk node must be present; mirrored queues replicate data across nodes to avoid loss.

Pros and Cons

Advantages

High concurrency thanks to Erlang.

Stable, easy to use, extensive language support.

Message acknowledgment and persistence.

Highly customizable routing.

Active community and frequent updates.

Disadvantages

Complexity for deep custom development.

Central broker can become a bottleneck, adding latency.

Steeper learning curve for protocols and APIs.

4. RocketMQ

RocketMQ, originally developed by Alibaba and donated to Apache in 2016, focuses on low latency, high reliability, and scalability.

Overview

Implemented in Java, latest version 4.3.1, widely adopted in large‑scale Alibaba services (e.g., handling trillions of messages during Double‑11).

Key Features

Supports Pub/Sub and point‑to‑point models.

Strict FIFO ordering and at‑least‑once delivery.

Pull and push delivery modes.

High throughput (millions of messages per broker).

Multiple protocols (JMS, MQTT, etc.).

Docker images and rich dashboard for monitoring.

Deployment Requirements

64‑bit OS (Linux, Unix, macOS).

JDK 1.8+ (64‑bit) and Maven 3.2+ for building.

Architecture

Components: NameServer (service discovery, similar to Zookeeper), Brokers, Producers, and Consumers.

High‑Availability

RocketMQ uses a replica‑based HA model without automatic election. Master and Slave roles are pre‑configured; the MasterId is 0, Slaves have higher IDs. Various deployment modes exist:

Single Master – simple but not production‑ready.

Multiple Masters – high performance, but may lose messages on failure.

Multiple Masters + Multiple Slaves (asynchronous replication) – minimal loss, high availability.

Multiple Masters + Multiple Slaves (synchronous double‑write) – highest data safety, slightly lower throughput.

Pros and Cons

Advantages

Supports >10,000 persistent queues per broker.

All messages are persisted with page‑cache then disk flush.

Simple model and easy‑to‑use APIs.

Excellent performance and large‑scale message accumulation.

Rich consumption modes (cluster, broadcast).

Active community with frequent releases.

Disadvantages

Limited client language support (Java, C++, Go).

No built‑in web management UI (only CLI tools).

Lacks native JMS implementation.

5. Comparison of the Four Message Queues

Each of the four open‑source MQs—Kafka, ActiveMQ, RabbitMQ, and RocketMQ—has distinct strengths and trade‑offs. No single solution dominates all scenarios; selection should consider throughput, latency, ordering guarantees, ecosystem, operational complexity, and community support.

6. References

RabbitMQ official site: https://www.rabbitmq.com/

ActiveMQ official site: http://activemq.apache.org/

RocketMQ GitHub: https://github.com/alibaba/RocketMQ

Kafka official site: http://kafka.apache.org/

Conclusion

The author encourages readers to like, share, and follow the "Code Monkey Technical Column" public account for additional PDF resources on Spring Cloud, Spring Boot, and MyBatis.