Mastering Message Redelivery in Apache Pulsar: Strategies and Code Samples

Introduction

Apache Pulsar is a multi‑tenant, high‑performance messaging solution that supports low latency, read/write separation, geo‑replication, rapid scaling, and flexible fault tolerance. In many scenarios users need to re‑deliver messages, such as after timeouts or processing errors. This guide outlines Pulsar's built‑in redelivery options.

Consumer‑Broker Message Flow

Pulsar combines push and pull mechanisms. When a consumer is created it receives a MaxReceiveQueue size that determines its permit count—the maximum number of messages it can buffer.

The consumer sends a FLOW request with its current permit (initially the full MaxReceiveQueue).

The broker records this as AvailablePermit and uses it to decide how many messages to send.

If AvailablePermit > 0, the broker reads up to N messages and pushes them, decrementing AvailablePermit by N.

Received messages are placed in the consumer’s ReceiveQueue. When the application calls receive(), the consumer increments its permit by one.

When the permit exceeds half of MaxReceiveQueueSize, the consumer issues another FLOW request.

By default, once a message is delivered to the consumer it is not re‑sent unless the consumer explicitly triggers redelivery.

SDK‑Unified Redelivery (AckTimeout)

The simplest approach is to configure an acknowledgment timeout. If a message is not acked within the configured period, Pulsar automatically redelivers it.

Consumer<String> consumer = pulsarClient.newConsumer(Schema.STRING)
    .ackTimeout(10, TimeUnit.SECONDS)
    .subscribe();

Internally the client creates an UnAckedMessageTracker that monitors all received messages. When the timeout expires, the tracker calls redeliverUnacknowledgedMessages(), causing the broker to resend the message identified by its MessageId.

User‑Driven Redelivery – NegativeAck

When applications need fine‑grained control, Pulsar offers NegativeAck. Only messages for which the client explicitly invokes negativeAcknowledge(messageId) are tracked by a NegativeAcksTracker and later redelivered.

Consumer<String> consumer = pulsarClient.newConsumer(Schema.STRING)
    .negativeAckRedeliveryDelay(1001, TimeUnit.MILLISECONDS)
    .subscribe();

// Trigger redelivery for a specific message
consumer.negativeAcknowledge(message);

The redelivery delay can be customized via negativeAckRedeliveryDelay.

User‑Driven Redelivery – Retry‑Later Queue (RLQ)

For scenarios where a message cannot be processed now but should be retried later, Pulsar provides a retry‑later queue (RLQ) using the reconsumeLater() API. Each call creates a new message in a dedicated RLQ topic named {originalTopic}-{subscription}_RLQ and acknowledges the original message.

Consumer<byte[]> consumer = pulsarClient.newConsumer(Schema.BYTES)
    .topic("my-topic")
    .subscriptionName("my-subscription")
    .subscriptionType(SubscriptionType.Shared)
    .enableRetry(true)
    .deadLetterPolicy(DeadLetterPolicy.builder()
        .maxRedeliverCount(maxRedeliveryCount)
        .build())
    .subscribe();

// Schedule a retry after 3 seconds
consumer.reconsumeLater(msg, 3, TimeUnit.SECONDS);

Messages placed in the RLQ carry special properties:

REAL_TOPIC : original topic name

ORIGIN_MESSAGE_ID : original MessageId RECONSUMETIMES : number of times the message has been re‑consumed

DELAY_TIME : configured delivery delay

Dead‑Letter Queue (DLQ) – Limiting Redelivery Attempts

Continuous failures can be handled by a dead‑letter queue. When a DLQ is configured, messages that exceed a maximum redelivery count are moved to a separate dead‑letter topic named {topic}-{subscription}_DLQ. The DLQ is consumed independently.

Consumer<byte[]> consumer = pulsarClient.newConsumer(Schema.BYTES)
    .topic("my-topic")
    .subscriptionName("my-subscription")
    .subscriptionType(SubscriptionType.Shared)
    .deadLetterPolicy(DeadLetterPolicy.builder()
        .maxRedeliverCount(maxRedeliveryCount)
        .build())
    .subscribe();

Relationship Among Redelivery Mechanisms and DLQ

If a DLQ is configured, redelivery triggered by AckTimeout, NegativeAck, or reconsumeLater() will all respect the DLQ limit—once the maximum count is reached the message is routed to the DLQ topic.

Redelivery counting differs: AckTimeout and NegativeAck use the broker’s RedeliveryTracker, which increments a RedeliveryCount field in the message.

RLQ counts are stored in the RECONSUMETIMES property generated by the client.

In summary, Apache Pulsar offers multiple flexible redelivery strategies—automatic timeout‑based, explicit negative acknowledgments, retry‑later queues, and dead‑letter queues—allowing developers to choose the most suitable approach for their use case.