Introduction to Message Queues, JMS, RabbitMQ, and Kafka

Hello, I am a senior architect. Have you heard terms like message queue, JMS, MQ, Kafka and feel confused about their relationships? Let’s take a look at what they are.

1. Message Queue Introduction

Imagine a traditional courier delivering a package directly to you. The courier needs time to deliver, you need time to receive, and you both agree on a delivery time and place. However, the courier may have many packages and you may not be home when the package arrives, leading to missed deliveries or the need to reschedule.

To solve this, delivery lockers were introduced. The courier can drop the package in the locker at any time, and you can pick it up from the locker when convenient. The locker acts like a message queue: a container that stores messages (packages) until the consumer retrieves them.

1.1 What Is a Message Queue

A message queue is a container that stores messages; when a consumer needs a message, it can retrieve it from the queue. Wikipedia defines a message queue as a form of inter‑process communication that stores a series of inputs, usually from users.

We can think of a message queue as a container that holds messages, and when we need to use a message we take it out for our own use.

1.2 What Is a Message Queue Used For?

Message queues are important components in distributed systems. They are used to improve system performance and handle peak loads through asynchronous processing, and to reduce system coupling.

Asynchronous processing can improve performance (peak shaving, reducing response time). For example, during user registration, four steps are required: fill the form, submit the form, receive an email, and receive an SMS. In a synchronous, serial flow the total time is a + b + c + d. With synchronous parallelism the time becomes a + b + max(c,d). Using a message queue the time becomes a + b + queue‑processing‑time.

Message queues also reduce system coupling. When multiple companies integrate with a service, each integration requires changes to the original system. With a message queue, each consumer simply reads from the queue without direct integration, isolating the producer from the consumers.

1.3 Two Modes of Message Queues

Point‑to‑Point Mode

Consists of a queue, a producer, and a consumer. Each message is sent to a specific queue, and the consumer retrieves messages from that queue. The queue retains messages until they are consumed or expire.

Publish‑Subscribe Mode

Consists of topics, publishers, and subscribers. A publisher sends a message to a topic, which then distributes the message to all bound queues. Publishers and subscribers are anonymous to each other, and topics retain messages until they are delivered.

2. JMS Introduction

JMS (Java Message Service) is a Java API for message‑oriented middleware, similar to JDBC for databases. It enables asynchronous communication between two Java applications or components in a distributed system.

JMS is a standard that defines how Java programs can create, send, receive, and read messages, reducing coupling and increasing reliability.

2.1 JMS Message Model

The model is the same as described for message queues: point‑to‑point and publish‑subscribe.

2.2 JMS Consumption

Synchronous: the consumer calls receive() and blocks until a message arrives or a timeout occurs.

Asynchronous: the consumer registers a message listener; the JMS provider invokes onMessage() when a message arrives.

2.3 JMS Programming Model

The JMS programming model is very similar to JDBC. For example, the following code shows how to obtain a MyBatis SqlSession (illustrative only):

sqlSessionFactory = new SqlSessionFactoryBuilder().build(inputStream);
SqlSession openSession = sqlSessionFactory.openSession(true);
ProjectMapper mapper = openSession.getMapper(ProjectMapper.class);
mapper.queryAllTaskByInit("init");

The JMS model consists of:

ConnectionFactory creates a Connection.

Connection creates a Session.

Session creates producers and consumers.

Producers generate messages.

Consumers consume messages.

3. MQ Overview

JMS is an interface/standard, not a concrete technology. Implementations that follow the JMS specification include ActiveMQ, RabbitMQ, and RocketMQ (partially). Kafka does not implement JMS.

3.1 AMQP Protocol

AMQP (Advanced Message Queuing Protocol) is a network protocol for asynchronous message passing. RabbitMQ is a typical enterprise messaging system based on AMQP.

AMQP Workflow

A publisher sends a message to an exchange. The exchange routes the message to bound queues according to routing rules. Consumers then retrieve messages from those queues.

3.2 RabbitMQ Model

RabbitMQ consists of a server side (queues and exchanges) and a client side (producers and consumers). Multiple virtual brokers can be created on a RabbitMQ server, each managing its own exchanges and bindings.

Exchanges have four types:

Direct: routes messages to queues with a matching routing key.

Fanout: broadcasts messages to all bound queues regardless of routing key.

Topic: routes messages to queues whose binding pattern matches the routing key (using * and # wildcards).

4. Kafka

Kafka is not a JMS implementation. It stores messages in topics, which are divided into partitions. Each partition is an ordered log file; each message has an offset. Producers write to topics, consumers read from topics. Kafka clusters consist of multiple broker nodes, and replication ensures high availability.

4.1 Kafka Terminology

Producer: client that sends messages to a broker.

Consumer: client that reads messages from a broker.

Topic: a category of messages; each topic is split into partitions.

Partition: an ordered sequence of messages within a topic, identified by offset.

Offset: the position of a message within a partition.

Broker: a server in the Kafka cluster that stores partitions.

4.2 High Availability Mechanism

Multiple brokers form a cluster; each partition can have replicas on different brokers.

One replica is elected as the leader; producers and consumers interact with the leader.

Followers replicate data from the leader. If the leader fails, another replica is elected.

In summary:

Message Queue: a communication method between components.

JMS: Java’s standard API for message queues.

ActiveMQ / RabbitMQ: concrete implementations of the JMS API.

RocketMQ: a partial implementation of the JMS API.

Kafka: a non‑JMS message‑queue technology.