Message Queue Fundamentals: Use Cases, Product Comparison, High Availability, and Reliability Strategies

1. Why use a message queue? (Application scenarios)

The essence of a message queue is a FIFO data structure; common scenarios include decoupling, asynchronous processing, and traffic shaping (peak shaving).

Decoupling

Order systems tightly depend on payment, inventory, and logistics services; using a message queue breaks this tight coupling.

Asynchronous

Traffic shaping (Peak shaving)

2. Comparison of various message queue products

MQ selection summary

ActiveMQ: early adoption, not validated for large‑scale throughput, community inactive, not recommended.

RabbitMQ: Erlang‑based, higher learning curve for Java developers, open source, active community, recommended for performance and stability.

RocketMQ: Java‑based, proven high concurrency inside Alibaba, stable and performant, recommended if custom development is needed.

Kafka: real‑time computing and log collection in big‑data scenarios, active community, recommended for big‑data and logging workloads.

3. Advantages and disadvantages of message queues

Advantages: decoupling, asynchronous processing, traffic shaping.

Disadvantages: reduced system availability, increased complexity, consistency challenges.

4. How to ensure high availability of message queues?

RabbitMQ HA

Standard cluster mode

Messages are stored on a single broker instance; other nodes fetch data via queue metadata, which does not provide true HA and suffers from single‑node bottlenecks.

Mirrored cluster mode

RocketMQ HA – dual‑master dual‑slave

5. How to guarantee no message loss?

Causes of message loss

Producer fails to successfully send to the broker.

Broker crashes after receiving the message, causing in‑memory data loss.

Consumer crashes before completing processing.

Solutions to prevent loss

Producer uses reliable sending mechanisms.

Broker persists messages to durable storage.

Consumer acknowledges only after successful processing; broker deletes the message after receiving the ack.

6. How to avoid duplicate consumption (idempotency)

Root cause of duplicate messages: network unreachable.

Duplicate during sending

Duplicate during consumption

Solution – message idempotency

Producer attaches a globally unique message ID.

Consumer checks DB/Redis for the ID before processing.

If not processed, consume and record the ID; otherwise skip.

7. How to ensure message ordering?

Global ordering: one producer, one queue, one consumer (1:1:1).

Partial ordering

Producer sends messages of the same group to a specific queue.

Multiple consumers read from that queue concurrently.

MQ uses segment locks to guarantee ordered consumption within the queue.

8. How to handle massive message backlog?

Reasons for backlog

Network failures.

Consumer does not acknowledge messages properly.

Backlog mitigation strategies

Fix and improve consumer processing speed.

Offload piled messages to a larger MQ cluster.

Add more consumer nodes for parallel consumption.

After processing, restore the original architecture.

9. How to handle message expiration?

Causes of expiration

Messages with a TTL that exceed the time without consumption become expired and can be moved to a dead‑letter queue.

Expiration handling

Expired messages are routed to a dead‑letter queue.

A dedicated consumer processes dead‑letter messages and logs them.

Inspect logs and optionally resend messages to the main queue.