How to Prevent Duplicate Consumption Across Multiple Instances in RabbitMQ Pub/Sub?

In microservice architectures, services communicate via HTTP, RPC, or message queues; RabbitMQ is typically used for event publishing. The challenge is that an event may have several subscribers, some of which run multiple instances, yet each subscriber must receive the event only once without duplicate processing.

Requirement

When an OrderCreatedEvent is emitted by the order service, the following services need to consume it:

Log service (single instance)

Message service for SMS/email (single instance)

Warehouse service (multiple instances, load‑balanced) – only one instance should process each message

Finance service (multiple instances, load‑balanced) – only one instance should process each message

BI service (single instance)

Problem

After the OrderCreatedEvent is published, all five services can receive it, but the warehouse and finance services have multiple instances, so each instance could receive the same message, leading to duplicate consumption.

Solution

Create a fanout exchange for the OrderCreatedEvent.

For each subscriber that needs to handle the event, create a dedicated queue. All instances of that subscriber consume from the same queue.

Bind each subscriber’s queue to the fanout exchange created in step 1.

The event publisher sends the OrderCreatedEvent to the exchange, which routes the message to each bound queue. RabbitMQ then delivers the message to exactly one consumer instance of each queue, preventing duplicate processing.

Below is a schematic diagram of the architecture:

Four Main Exchange Types

RabbitMQ provides four core exchange types:

Fanout

Feature : Broadcasts messages to all bound queues.

Routing rule : Ignores the routing key.

Use case : Log systems, notification broadcasting, etc.

Direct

Feature : Routes messages based on an exact routing key match.

Routing rule : The message’s routing key must exactly match the queue’s binding key.

Use case : A consumer handling a specific task type, such as user‑registration messages.

Topic

Feature : Uses wildcards ( * for a single word, # for zero or more words) for pattern matching.

Routing rule : The message’s routing key must match the pattern bound to the queue.

Use case : Log level filtering, e.g., logs.info.* for all info‑level logs or logs.# for all logs.

Headers

Feature : Routes based on message header key‑value pairs instead of the routing key.

Routing rule : Matches the header values with those specified in the binding.

Use case : Complex attribute matching; less common due to lower performance.

Demo

The author will provide concrete implementations of the above solution in both Java and .NET.