Skip to content
MQ Ordered Consumption: Partial Order Patterns, Code Examples & Interview Guide

MQ Ordered Consumption: Partial Order Patterns, Code Examples & Interview Guide

This article explains how to guarantee message ordering in message queues by comparing RocketMQ, Kafka, and RabbitMQ implementations, detailing partial ordering via business key hashing, providing production-ready code for producers and consumers, and covering business-level safeguards like state machines and idempotency to handle queue rebalancing and head-of-line blocking.

Rain's Path to Java Mastery
Rain's Path to Java Mastery
Rain's Path to Java Mastery

Problem Essence: Why Messages Become Out of Order

The root cause of message disorder is ordered production → unordered transport/storage → unordered consumption . Three key points break ordering:

✅ Producer: single producer, single thread sends in order.

❌ Storage: messages distributed to different queues/partitions.

❌ Consumer: multiple consumers concurrently consume different queues.

Core Principles for Ordered Consumption

Global ordering : all messages strictly follow send order (poor performance, rarely used).

Partial ordering : messages with same business key keep order (industry standard).

Core formula: Same business key → same queue → same consumer thread .

Mainstream MQ Implementation Approaches

1. RocketMQ (Most Complete Ordered Consumption Support)

Steps:

Producer: use MessageQueueSelector to hash business key (e.g., order ID) to a specific queue.

Consumer: use MessageListenerOrderly (auto-locks, single-threaded per queue).

RocketMQ automatically handles broker failure queue reallocation without breaking order.

2. Kafka (Partition-Based Ordered Consumption)

Steps:

Producer: specify key; Kafka hashes key to same partition by default.

Consumer: one partition consumed by only one consumer thread within a group.

Limitation: Kafka guarantees order only within a partition, not globally.

3. RabbitMQ (Weaker Ordering Support)

Steps:

Producer: send messages with same business key to same queue.

Consumer: set prefetchCount=1, disable auto-ack, single-threaded consumption.

Drawback: extremely low concurrency, unsuitable for high-throughput scenarios.

Comparison of Three MQ Ordered Consumption Solutions

Ordering granularity : RocketMQ (queue-level), Kafka (partition-level), RabbitMQ (queue-level).

Concurrency : RocketMQ (high, queues = concurrency), Kafka (high, partitions = concurrency), RabbitMQ (very low).

Exception handling : RocketMQ (auto-retry, doesn't affect other queues), Kafka (manual, failure blocks), RabbitMQ (manual, failure blocks).

Recommendation : RocketMQ ⭐⭐⭐⭐⭐, Kafka ⭐⭐⭐⭐, RabbitMQ ⭐⭐.

Business-Level Safeguards

Even with MQ guarantees, production needs business-layer fallbacks:

1. Message State Machine Validation

Each message carries version or state.

On consumption, check if state transition is legal (e.g., order already paid cannot process creation message).

Example:

if (order.getStatus() >= PAID && message.getStatus() == CREATED) { discard }

.

2. Idempotency Design

Ensure duplicate or out-of-order consumption yields same result.

Use unique business ID as idempotency key (e.g., order ID + operation type).

3. Delayed Retry Mechanism

For out-of-order messages, don't fail immediately; delay and retry.

Example: receive "payment completed" but not "order created", delay 5 seconds and retry.

Common Pitfalls and Considerations

Don't use global ordering unless mandatory (throughput drops 100x+).

❌ Don't use multi-threaded consumption on same queue.

✅ Plan queue count upfront to avoid rebalancing breaking order.

✅ Handle retries and dead-letter queues to prevent single failed message blocking entire queue.

Summary: Best Practice

RocketMQ + MessageQueueSelector + MessageListenerOrderly + Business State Machine Validation ensures partial ordering, high throughput, and business-layer safety for edge cases.

Core Code Implementations (RocketMQ + Kafka)

1. RocketMQ Ordered Consumption (Industry Preferred)

Producer (Hash Routing by Business Key)

@Service
public class OrderProducer {
    @Autowired
    private DefaultMQProducer producer;

    public void sendOrderlyMessage(Long orderId, String messageBody) throws Exception {
        Message message = new Message(
            "ORDER_TOPIC",
            "ORDER_TAG",
            orderId.toString(), // business key for hash routing
            messageBody.getBytes(StandardCharsets.UTF_8)
        );

        SendResult result = producer.send(message, new MessageQueueSelector() {
            @Override
            public MessageQueue select(List<MessageQueue> mqs, Message msg, Object arg) {
                Long orderId = (Long) arg;
                int index = (int) (orderId % mqs.size());
                return mqs.get(index);
            }
        }, orderId);
        log.info("Ordered message sent, orderId: {}, queueId: {}", orderId, result.getMessageQueue().getQueueId());
    }
}

Consumer (Single-Threaded Per Queue, Auto-Lock)

@Component
public class OrderConsumer {
    @PostConstruct
    public void start() throws Exception {
        DefaultMQPushConsumer consumer = new DefaultMQPushConsumer("ORDER_CONSUMER_GROUP");
        consumer.setNamesrvAddr("127.0.0.1:9876");
        consumer.subscribe("ORDER_TOPIC", "ORDER_TAG");

        consumer.registerMessageListener(new MessageListenerOrderly() {
            @Override
            public ConsumeOrderlyStatus consumeMessage(List<MessageExt> msgs, ConsumeOrderlyContext context) {
                context.setAutoCommit(true);
                for (MessageExt msg : msgs) {
                    try {
                        String messageBody = new String(msg.getBody(), StandardCharsets.UTF_8);
                        Long orderId = Long.parseLong(msg.getKeys());
                        log.info("Ordered consumption, orderId: {}, body: {}", orderId, messageBody);
                        orderService.processOrderMessage(orderId, messageBody);
                    } catch (Exception e) {
                        log.error("Consumption failed, retry times: {}", msg.getReconsumeTimes(), e);
                        return ConsumeOrderlyStatus.SUSPEND_CURRENT_QUEUE_A_MOMENT;
                    }
                }
                return ConsumeOrderlyStatus.SUCCESS;
            }
        });
        consumer.start();
        log.info("RocketMQ ordered consumer started");
    }
}

2. Kafka Ordered Consumption

Producer (Key Routing to Same Partition)

@Service
public class KafkaOrderProducer {
    @Autowired
    private KafkaTemplate<String, String> kafkaTemplate;

    public void sendOrderlyMessage(Long orderId, String messageBody) {
        kafkaTemplate.send("ORDER_TOPIC", orderId.toString(), messageBody);
    }
}

Consumer (Thread Count = Partition Count)

@Component
public class KafkaOrderConsumer {
    @KafkaListener(topics = "ORDER_TOPIC", groupId = "ORDER_CONSUMER_GROUP", concurrency = "8")
    public void consumeMessage(ConsumerRecord<String, String> record) {
        Long orderId = Long.parseLong(record.key());
        String messageBody = record.value();
        log.info("Kafka ordered consumption, orderId: {}, partition: {}", orderId, record.partition());
        orderService.processOrderMessage(orderId, messageBody);
    }
}

3. Business-Layer Safeguard Code (State Machine + Idempotency)

@Service
public class OrderService {
    @Autowired
    private OrderMapper orderMapper;

    @Transactional
    public void processOrderMessage(Long orderId, String messageBody) {
        OrderMessage message = JSON.parseObject(messageBody, OrderMessage.class);
        String operationType = message.getOperationType();

        // 1. State machine validation (optimistic lock)
        Order order = orderMapper.selectById(orderId);
        if (order == null && !"CREATE".equals(operationType)) {
            throw new RuntimeException("Order not exist, delay retry");
        }
        if (order != null && order.getStatus().getCode() >= message.getTargetStatus().getCode()) {
            log.warn("Message out-of-order or duplicate, orderId: {}, current: {}, target: {}", orderId, order.getStatus(), message.getTargetStatus());
            return;
        }

        // 2. Idempotency guarantee
        String idempotentKey = orderId + "_" + operationType;
        if (idempotentService.checkIdempotent(idempotentKey)) {
            log.warn("Duplicate consumption, idempotentKey: {}", idempotentKey);
            return;
        }

        // 3. Execute business logic
        switch (operationType) {
            case "CREATE": createOrder(message); break;
            case "PAY": payOrder(orderId, message.getPayAmount()); break;
            case "DELIVER": deliverOrder(orderId, message.getLogisticsNo()); break;
            case "COMPLETE": completeOrder(orderId); break;
        }

        // 4. Record idempotency
        idempotentService.recordIdempotent(idempotentKey);
    }
}

Technical Challenges and Solutions (Interview Focus)

Broker failure causing queue reallocation breaking order : RocketMQ uses MessageListenerOrderly with distributed locks on broker and consumer; Kafka uses idempotent producer + transactions + business state machine; avoid single-broker deployment.

Message retry causing disorder : RocketMQ sends retry messages back to original queue tail; Kafka custom retry logic to original partition; limit retries, then dead-letter.

Queue/partition scaling breaking order : Pre-plan queue count; use "double expansion" to keep hash mapping; pause writes or dual-write during expansion.

Single failed message blocking entire queue : Distinguish retryable vs non-retryable exceptions; max retries (e.g., 16) then dead-letter; use SUSPEND_CURRENT_QUEUE_A_MOMENT to pause only current queue temporarily.

Throughput bottleneck under high concurrency : Set queue count ≈ CPU cores; split business into different topics; use partial ordering + eventual consistency for non-strict scenarios.

Cross-broker message order issues : Custom queue selector to pin same business key to same broker+queue; disable auto-create topic, manually define queue distribution.

Consumer restart causing duplicate consumption and disorder : Manual offset commit after success; business idempotency; graceful shutdown.

Technical Highlights Summary

RocketMQ native support : MessageQueueSelector + MessageListenerOrderly for MQ-level partial ordering.

Hash routing algorithm : Business key modulo ensures same identifier lands in same queue.

Distributed lock mechanism : Auto-locks on broker and consumer to prevent reallocation disorder.

Business-layer safeguard : State machine + idempotency solve extreme disorder and duplicates.

Graceful exception handling : Separate retryable vs non-retryable, avoid single failure blocking queue.

Interview Simulation Highlights

Candidate describes order state flow (create → pay → deliver → complete) requiring per-order ordering, not global. Chooses partition ordering with order ID hash. Explains RocketMQ MessageQueueSelector and MessageListenerOrderly with queue-level locking. For Kafka, notes rebalance risks, mitigated by larger session.timeout.ms and static membership plus idempotency. Handles head-of-line blocking with state machine optimistic lock + dead-letter after max retries. Emphasizes ordered consumption doesn't prevent duplicates; uses business serial + DB unique key or Redis lock for idempotency.

Real Project Core Code (Three Versions)

1. Producer: Ordered Order Message Sending

@Service
public class OrderMessageProducer {
    @Resource
    private RocketMQTemplate rocketMQTemplate;

    public void sendOrderMessage(OrderEvent orderEvent) {
        String topic = "order-status-topic";
        String orderId = orderEvent.getOrderId();
        Message<String> message = MessageBuilder
            .withPayload(JSON.toJSONString(orderEvent))
            .setHeader(MessageConst.PROPERTY_KEYS, orderId)
            .build();
        rocketMQTemplate.syncSendOrderly(topic, message, orderId);
    }
}

🧠 Highlight : syncSendOrderly encapsulates MessageQueueSelector; same orderId hashes to fixed queue.

2. Consumer Basic: Ordered Listener

@Component
@RocketMQMessageListener(
    topic = "order-status-topic",
    consumerGroup = "order-consumer-group",
    consumeMode = ConsumeMode.ORDERLY
)
public class OrderMessageConsumer implements RocketMQListener<String> {
    @Override
    public void onMessage(String message) {
        OrderEvent event = JSON.parseObject(message, OrderEvent.class);
        processOrderEvent(event);
    }
    private void processOrderEvent(OrderEvent event) { /* state transition */ }
}

🧠 Highlight : ConsumeMode.ORDERLY auto-configures MessageListenerOrderly for single-threaded per queue.

3. Consumer Advanced: State Machine + Dead-Letter (Core)

@Component
@RocketMQMessageListener(
    topic = "order-status-topic",
    consumerGroup = "order-consumer-group",
    consumeMode = ConsumeMode.ORDERLY
)
public class OrderMessageConsumerAdv implements RocketMQListener<MessageExt> {
    @Resource
    private OrderService orderService;
    @Resource
    private DeadLetterHandler deadLetterHandler;

    @Override
    public void onMessage(MessageExt msg) {
        String body = new String(msg.getBody(), StandardCharsets.UTF_8);
        OrderEvent event = JSON.parseObject(body, OrderEvent.class);
        try {
            boolean success = orderService.updateWithStatusCheck(event);
            if (success) return;
            if (event.getRetryTimes() >= MAX_RETRY) {
                deadLetterHandler.sendToDeadLetter(event);
                log.warn("Message state mismatch and retries exhausted, dead-letter: {}", event);
                return; // ACK, don't block queue
            }
            throw new IllegalStateException("Order state unexpected: " + event);
        } catch (Exception e) {
            throw new RuntimeException("Consumption exception, trigger retry", e);
        }
    }
}

State Machine SQL (Core of Core)

@Update("UPDATE t_order SET status = #{newStatus}, update_time = NOW() " +
    "WHERE order_id = #{orderId} AND status = #{expectedPreStatus}")
int updateWithStatusCheck(OrderEvent event);

🎯 Highlights : Optimistic lock + state machine filters invalid messages; max retries then dead-letter never blocks queue ; exception classification for finer strategy.

Four Core Technical Difficulties and Solutions

Partition selection & load balancing : Order ID hash modulo; consumer cluster mode balances queues. Code: syncSendOrderly internal hash.

Consumption failure head-of-line blocking : State machine optimistic lock + max retries + dead-letter, unexpected messages skipped. Code: advanced consumer.

Rebalance causing transient disorder : Increase session.timeout.ms=30000; use static group.instance.id; tolerate brief duplicates via idempotency.

Duplicate messages breaking idempotency : Business serial in message, deduplicate via DB unique key / Redis lock. Unique index: uk_order_event_id.

Production-Grade Detail

In RocketMQ ordered mode, failed message retry holds queue lock; other messages wait. Set retryTimes upper limit; upon exhaustion, force exit to avoid deadlocking entire queue. Many candidates miss this point.

Conclusion

Core principle: same business identifier → same partition/queue → single-threaded consumption. Production uses RocketMQ MessageListenerOrderly with order ID hash, preserving horizontal scaling. Consumption failures handled by state machine optimistic lock + dead-letter fallback , plus idempotency via business serial. Global ordering only for extreme cases; generally not recommended.

Original Source

Signed-in readers can open the original source through BestHub's protected redirect.

Sign in to view source
Republication Notice

This article has been distilled and summarized from source material, then republished for learning and reference. If you believe it infringes your rights, please contactadmin@besthub.devand we will review it promptly.

State MachineKafkaMessage QueueRabbitMQRocketMQIdempotencyinterview preparationOrdered ConsumptionHead-of-line BlockingPartial Ordering
Reader feedback

How this landed with the community

Sign in to like

Rate this article

Was this worth your time?

Sign in to rate
Discussion

0 Comments

Thoughtful readers leave field notes, pushback, and hard-won operational detail here.