Why Message Queues Matter: From Basics to Kafka and JMS Explained

1 Message Queue Introduction

Using a courier‑locker analogy, a message queue is like a locker where the courier can drop a package at any time and the recipient picks it up when convenient, eliminating the need for both parties to be present simultaneously.

1.1 What is a Message Queue

A message queue is a container that stores messages until they are needed by a consumer. Wikipedia defines it as a form of inter‑process or intra‑process communication that queues inputs, often from users.

We can think of a message queue as a container that holds messages for later retrieval.

1.2 Uses of Message Queues

Message queues are key components in distributed systems. They improve performance through asynchronous processing, smooth traffic spikes, and reduce system coupling.

Example: during user registration, four steps (fill form, submit, email receipt, SMS receipt) take time a + b + c + d if done synchronously. With asynchronous processing the time becomes a + b + max(c,d). Using a message queue the time becomes a + b + queue‑overhead.

Reducing System Coupling

When many companies integrate directly with a single system, each integration requires code changes, increasing coupling. By inserting a message queue, each company simply consumes messages from the queue, eliminating the need for direct integration.

1.3 Two Message Queue Patterns

Point‑to‑Point

Consists of a queue, a producer, and a consumer. Each message is sent to a specific queue and remains there until consumed or timed out.

Publish/Subscribe

Involves a topic, publishers, and subscribers. Publishers send messages to a topic, which the broker distributes to all subscribed consumers. Publishers and subscribers are anonymous to each other.

2 JMS Introduction

JMS (Java Message Service) is a Java API for messaging middleware, similar to JDBC for databases. It enables asynchronous communication between Java applications by providing methods to create, send, receive, and read messages.

JMS is a standard that lowers coupling, improves reliability, and supports asynchronous processing.

Message Queue: a general communication mechanism in computer science.

JMS: a Java‑specific standard for distributed messaging.

In other words, JMS is Java’s way of implementing a message queue.

2.2 JMS Consumption

Synchronous: the consumer calls receive() and blocks until a message arrives or a timeout occurs.

Asynchronous: the consumer registers a listener; the provider invokes onMessage() when a message arrives.

2.3 JMS Programming Model

The JMS programming model resembles JDBC. For example, MyBatis uses a SqlSessionFactoryBuilder to build a session factory.

sqlSessionFactory = new SqlSessionFactoryBuilder().build(inputStream);
SqlSession openSession = sqlSessionFactory.openSession(true);
ProjectMapper mapper = openSession.getMapper(ProjectMapper.class);
mapper.queryAllTaskByInit("init");

The JMS workflow consists of:

ConnectionFactory creates a Connection.

Connection creates a Session.

Session creates producers and consumers.

Producers send messages.

Consumers receive messages.

3 MQ Introduction

JMS is an interface; concrete implementations include ActiveMQ, RabbitMQ, and RocketMQ (which only partially follows JMS). Kafka is not a JMS implementation.

3.1 AMQP Protocol

AMQP (Advanced Message Queuing Protocol) is a network protocol for asynchronous message passing.

AMQP Model

AMQP Workflow

Publishers send messages to an exchange, which routes them to bound queues based on routing rules. Consumers then retrieve messages from those queues.

3.2 RabbitMQ Model

RabbitMQ consists of a server side (queues and exchanges) and a client side (producers and consumers). Multiple virtual brokers can be created on a single server.

Exchange types: direct, fanout, topic, headers.

4 Kafka

Kafka is a distributed log‑based messaging system that is not a JMS implementation. Key concepts include:

Producer: sends messages to a broker.

Consumer: reads messages from a broker.

Topic: a category for messages; each topic is split into partitions.

Broker: a Kafka server; a cluster consists of multiple brokers.

Partition: an ordered log segment within a topic; each message has an offset.

Offset: the unique position of a message in a partition.

Kafka stores messages on disk, partitions them for scalability, and retains logs for a configurable period, even after consumption.

Kafka High‑Availability Mechanism

Multiple brokers form a cluster; each partition may have replicas on different brokers.

One replica is elected leader; producers and consumers interact with the leader.

Followers replicate data from the leader; if the leader fails, a new leader is elected.

In summary:

Message Queue: a communication method between components.

JMS: Java’s standard interface for message queues.

ActiveMQ / RabbitMQ: concrete JMS implementations.

RocketMQ: partially follows JMS.

Kafka: a non‑JMS, log‑based messaging system.