Introduction to Message Queues, JMS, and Kafka

1. Introduction to Message Queues

Imagine a traditional courier delivering a package to you; the courier needs time to deliver, you need time to receive, and you must agree on a time and place. When schedules don’t match, you either wait at home or reschedule. A parcel locker solves this by letting the courier drop the package whenever, and you pick it up when convenient. This locker is analogous to a message queue, a container that stores messages until the consumer retrieves them.

1.1 What is a Message Queue?

A message queue is a container for messages; consumers retrieve messages when needed. Wikipedia defines it as an inter‑process or intra‑process communication mechanism that queues inputs, often from users.

We can think of a message queue as a container that stores messages for later retrieval.

1.2 Uses of Message Queues

Message queues are crucial in distributed systems for improving performance via asynchronous processing, smoothing traffic spikes, and reducing system coupling.

Example: user registration involves four steps (fill form, submit, email notification, SMS notification). Synchronous serial execution takes time a+b+c+d; parallel execution reduces it to a+b+max(c,d); using a message queue reduces it to a+b+queue processing.

Message queues also lower system coupling by allowing multiple independent systems to consume messages without direct integration.

1.3 Two Messaging Models

Point‑to‑Point Model

Consists of a queue, a sender, and a receiver. Each message is sent to a specific queue and stays there until consumed or timed out.

Publish‑Subscribe Model

Involves topics, publishers, and subscribers. Publishers send messages to a topic, which distributes them to all subscribed consumers. Publishers and subscribers are anonymous to each other.

2. JMS Introduction

Java Message Service (JMS) is a Java API for message‑oriented middleware, similar to JDBC for databases. It enables asynchronous communication between Java applications, reducing coupling and increasing reliability.

JMS defines a standard for creating, sending, receiving, and reading messages on the Java EE platform.

2.1 JMS Messaging Model

The point‑to‑point and publish‑subscribe concepts are the same as described above.

2.2 JMS Consumption

Synchronous: the consumer calls receive() and blocks until a message arrives or a timeout occurs.

Asynchronous: the consumer registers a message listener; the JMS provider invokes onMessage() when a message arrives.

2.3 JMS Programming Model

The JMS programming model resembles JDBC. For comparison, consider MyBatis code:

sqlSessionFactory = new SqlSessionFactoryBuilder().build(inputStream);
SqlSession openSession = sqlSessionFactory.openSession(true);
ProjectMapper mapper = openSession.getMapper(ProjectMapper.class);
mapper.queryAllTaskByInit("init");

JMS workflow:

ConnectionFactory creates a Connection.

Connection creates a Session.

Session creates producers and consumers.

Producers send messages.

Consumers receive messages.

3. MQ Overview

JMS is an interface, not a concrete technology. Implementations include ActiveMQ, RabbitMQ, and RocketMQ (the latter does not fully follow JMS). Kafka is another MQ technology but not JMS‑compatible.

3.1 AMQP Protocol

Advanced Message Queuing Protocol (AMQP) is a network protocol for asynchronous message passing. RabbitMQ is a typical AMQP‑based broker.

AMQP Model

AMQP Workflow

Publishers send messages to an Exchange, which routes them to bound Queues based on routing rules. Consumers then retrieve messages from those Queues.

3.2 RabbitMQ Model

RabbitMQ consists of a server side (queues and exchanges) and a client side (producers and consumers). Multiple virtual brokers can be created on a server, each managing its own exchanges and bindings.

Exchange types: direct, fanout, topic, headers.

4. Kafka

Kafka is not a JMS implementation. It uses topics, partitions, brokers, and offsets to store and deliver messages. Messages are retained for a configurable period even after consumption.

4.1 Kafka Architecture

Key terms:

Producer: sends messages to brokers.

Consumer: reads messages from brokers.

Topic: logical category of messages, split into partitions.

Partition: ordered log segment; each message has a unique offset.

Broker: a Kafka server; a cluster consists of multiple brokers.

Offset: position of a message within a partition.

4.2 Kafka High Availability

Multiple brokers form a cluster.

Topics are divided into partitions distributed across brokers.

Each partition has replicas; one replica is the leader.

Producers and consumers interact with the leader; followers replicate data.

If a broker fails, another replica becomes the leader, ensuring continuity.

Overall, the article provides a comprehensive overview of message queue fundamentals, JMS specifications, and popular implementations such as RabbitMQ and Kafka.