How to Guarantee Zero Message Loss in MQ Systems – Interview Mastery

Interviewers often ask candidates how to ensure that messages are never lost when using MQ technologies such as Kafka, RabbitMQ or RocketMQ.

Using the JD system as a case study, a user may use JD beans to offset part of the payment; the transaction service sends a message like "Deduct 100 beans from account X" to an MQ queue, and the bean service consumes the message to perform the actual deduction.

Case Background

The interaction between the transaction service and the bean service via an MQ queue illustrates typical decoupling and traffic‑control scenarios in distributed systems.

Case Analysis

Introducing an MQ middleware primarily achieves system decoupling and traffic control, but it also brings consistency challenges. The three key interview points are:

How to detect message loss?

Which stages may cause loss?

How to guarantee no loss?

Message loss can occur in three stages: production, storage, and consumption.

Message Production Stage

If the producer receives an ACK from the broker, the send is considered successful; handling return values and exceptions prevents loss at this stage.

Message Storage Stage

The broker typically replicates messages to at least two nodes before acknowledging, ensuring durability.

Message Consumption Stage

The consumer should acknowledge only after business logic succeeds, which avoids loss due to premature ACKs.

Even with these safeguards, failures are inevitable, so a detection mechanism is required.

Detecting Message Loss

Assign a globally unique ID or a monotonically increasing version number to each message at the producer side, and verify continuity or presence on the consumer side using interceptors.

If multiple producers exist, a globally unique ID is preferred over simple version numbers.

Handling Duplicate Consumption

Duplicate consumption arises from retry mechanisms. Solving it requires idempotent processing: the same command can be executed many times without changing the final state.

One practical approach is to maintain a message log table (or a Redis key) with fields for message ID and execution status; before processing, check if the ID already exists.

Dealing with Message Backlog

Backlog indicates performance bottlenecks, usually at the consumer side. Immediate actions include scaling out consumer instances and degrading non‑critical services.

For systems like Kafka, increasing the number of partitions proportionally to consumer instances is essential because each partition is consumed by a single consumer thread.

Distributed ID Generation

Reliable ID generation methods include database auto‑increment keys, UUID, Redis counters, and the Twitter‑Snowflake algorithm. Choosing a solution involves trade‑offs among simplicity, availability, and performance.

Summary

Understand each stage of message flow and where loss can occur; monitor acknowledgments and broker replication.

Implement idempotent consumption via unique message IDs or logs to prevent duplicate processing.

Address backlog by scaling consumers, adding partitions, and optimizing business logic.

Demonstrating this systematic thinking in an interview showcases deeper expertise than merely reciting a solution.