RabbitMQ vs Kafka: How to Choose the Right Messaging System

Asynchronous Messaging Patterns

Asynchronous messaging decouples producers from consumers. Two canonical patterns are used:

Queue Pattern

A queue buffers messages temporarily. Multiple producers can send to the same queue, but each message is consumed by only one consumer. When a consumer locks a message, the broker removes it from the queue.

Queue pattern typically means a message is processed by only one consumer.

If a consumer cannot handle a message, the broker returns the message to the queue for another consumer, providing fault tolerance and horizontal scalability.

Publish/Subscribe Pattern

In pub/sub a single message can be delivered to many subscribers simultaneously. Subscriptions can be:

Temporary – valid only while the consumer process is running; messages are lost when the consumer stops.

Durable – persist until explicitly deleted, allowing recovery after a restart.

RabbitMQ

RabbitMQ is a message broker that implements both queue and pub/sub patterns. It is comparable to other brokers such as ActiveMQ, ZeroMQ, Azure Service Bus, and Amazon SQS.

Queues

Developers declare named queues; producers publish messages to a queue, and consumers pull messages from that queue for processing.

Message Exchanges

RabbitMQ uses exchanges to realize pub/sub. A producer publishes to an exchange without knowing the subscribers. Each consumer creates a queue bound to the exchange; the exchange routes messages to those queues according to routing rules (direct, topic, fan‑out, headers).

RabbitMQ supports temporary and durable subscriptions via its API.

By mixing consumer groups with classic queues, RabbitMQ can emulate a hybrid model that combines strict queue semantics (single‑consumer processing) with pub/sub scalability (multiple consumers sharing a logical subscription).

Apache Kafka

Kafka is a distributed streaming platform whose storage layer consists of partitioned, immutable logs called topics.

Topics and Partitions

Each topic is split into one or more partitions. A partition is an ordered, append‑only sequence of records. By default Kafka distributes incoming messages across partitions with a round‑robin partitioner, but producers can supply a custom key to force all records with the same key (e.g., tenant ID) onto the same partition, preserving order for that logical stream.

Consumers and Consumer Groups

Consumers maintain an offset per partition and read records sequentially. A consumer group consists of multiple consumers that jointly own a topic’s partitions; the Kafka coordinator balances partition assignment and stores offsets on behalf of the group.

Using Kafka for Messaging

Kafka naturally implements pub/sub: producers write to a topic, and any number of consumer groups can read the same records. Offsets enable two subscription styles—persistent (offset stored, resume after restart) and transient (offset discarded, start from latest).

Kafka retains all records for a configurable retention period regardless of consumption. This enables replay, event sourcing, and audit logs. Simulating a classic queue by creating a single‑consumer‑group topic is possible but incurs drawbacks such as lack of per‑message acknowledgments and limited back‑pressure control (discussed in the follow‑up article).

Part 2 article address: https://betterprogramming.pub/rabbitmq-vs-kafka-1779b5b70c41

Comparison and Selection Guidance

RabbitMQ excels at flexible routing, mixed queue / pub‑sub workloads, and fine‑grained subscription control (temporary vs durable, per‑queue TTL, request‑reply patterns). Kafka excels at high‑throughput ingestion, durable log storage, built‑in replay, and scaling consumer groups independently of producers.

Choose RabbitMQ when the application requires complex routing topologies, short‑lived messages, or tight coupling between producers and a limited set of consumers. Choose Kafka when the workload demands massive data volume, long‑term retention, replayability, or when consumer groups need to scale horizontally without affecting producer logic.

Code example

相关阅读：