Message Queue Interview Guide: Benefits, Drawbacks, High Availability, Idempotency, Ordering and Design Strategies

This article compiles interview‑focused knowledge about message queues (MQ), starting with the fundamental question of why to use an MQ and the three core scenarios it solves: decoupling, asynchronous processing, and traffic shaping (peak‑shaving).

It explains how decoupling works by letting a producer publish messages to a queue while consumers independently pull them, reducing tight coupling between systems and simplifying failure handling.

The asynchronous benefit is illustrated with latency calculations, showing how moving slow downstream writes to an MQ can reduce request response time from nearly one second to under ten milliseconds.

Peak‑shaving is described with a real‑world traffic spike example, demonstrating how an MQ can buffer bursts of requests and allow the consumer to process them at a sustainable rate.

Advantages are summarized as these three scenarios; disadvantages include reduced system availability due to added external dependency, increased architectural complexity (duplicate consumption, message loss, ordering), and consistency challenges when some downstream services succeed while others fail.

The article then compares four popular MQ products:

ActiveMQ – once popular but now less active and not recommended for large‑scale throughput.

RabbitMQ – stable, open‑source, but built on Erlang which may limit deep Java‑side expertise.

RocketMQ – Alibaba‑origin, good for mid‑size companies with strong in‑house expertise.

Kafka – the de‑facto standard for real‑time big‑data pipelines, highly active community and strong durability.

High‑availability strategies are detailed for RabbitMQ (single‑node, normal cluster, mirrored cluster) and Kafka (replication factor, min.insync.replicas, acks=all, retries). The mirrored‑cluster mode for RabbitMQ replicates queues across nodes, while Kafka’s leader‑follower replication ensures no data loss on broker failure.

To guarantee exactly‑once processing, the article discusses idempotency techniques such as checking primary keys before inserts, using Redis SET for deduplication, attaching a globally unique ID to each message, and leveraging database unique constraints.

Reliability measures include producer‑side transaction or confirm modes for RabbitMQ, persistent messages, and consumer‑side manual acknowledgments to avoid premature message loss.

Message ordering is addressed for both RabbitMQ (single queue with single consumer or ordered processing via internal memory queues) and Kafka (key‑based partitioning ensures order within a partition; multi‑threaded consumers must preserve order via per‑key queues).

When faced with massive backlog or full queues, the article suggests temporary scaling: increase partitions, spin up additional consumers, or create a temporary redistribution consumer that quickly drains the backlog into a larger set of queues.

For expired or TTL‑d messages, it recommends re‑injecting lost data after the peak period using a supplemental program.

Finally, a high‑level design for building a custom message queue is outlined: adopt a distributed broker‑topic‑partition model for scalability, use sequential disk writes for performance, replicate partitions for HA, enforce zero‑loss policies with replication and acknowledgments, and provide APIs for transaction/confirm semantics.

Code example for RabbitMQ transaction handling:

// 开启事务
channel.txSelect
try {
    // 这里发送消息
} catch (Exception e) {
    channel.txRollback
    // 这里再次重发这条消息
}
// 提交事务
channel.txCommit