10 Real-World Message Queue (MQ) Scenarios Every Backend Engineer Should Know

Introduction

Recently a colleague asked me: what are the typical scenarios for using a message queue (MQ) and is it always required in a project? When I first started, I couldn’t understand why we needed a "middleman" when a direct API call could finish the job.

After experiencing system crashes, data loss, and performance bottlenecks, I finally grasped the value of MQ.

Below are ten typical MQ scenarios that can help you decide when to adopt a message queue.

Why Use a Message Queue (MQ)?

Before diving into concrete scenarios, we should ask a basic question: why use a message queue?

Direct service calls:

After introducing a message queue:

We will now examine ten concrete scenarios to understand MQ’s value.

Scenario 1: System Decoupling

Background

In an early e‑commerce project, after an order is created we needed to notify multiple downstream systems.

// Early tightly‑coupled design
public class OrderService {
    private InventoryService inventoryService;
    private PointsService pointsService;
    private EmailService emailService;
    private AnalyticsService analyticsService;

    public void createOrder(Order order) {
        // 1. Save order
        orderDao.save(order);
        // 2. Call inventory service
        inventoryService.updateInventory(order);
        // 3. Call points service
        pointsService.addPoints(order.getUserId(), order.getAmount());
        // 4. Send email notification
        emailService.sendOrderConfirmation(order);
        // 5. Record analytics data
        analyticsService.trackOrderCreated(order);
        // ...more services
    }
}

Tight Coupling : Order service must know all downstream services

Single Point of Failure : Any downstream failure makes order creation fail

Performance Bottleneck : Synchronous calls slow response

MQ Solution

After introducing MQ, the architecture becomes:

Code Implementation :

// Order service – producer
@Service
public class OrderService {
    @Autowired
    private RabbitTemplate rabbitTemplate;

    public void createOrder(Order order) {
        // 1. Save order
        orderDao.save(order);
        // 2. Send message to MQ
        rabbitTemplate.convertAndSend(
            "order.exchange",
            "order.created",
            new OrderCreatedEvent(order.getId(), order.getUserId(), order.getAmount())
        );
    }
}

// Inventory service – consumer
@Component
@RabbitListener(queues = "inventory.queue")
public class InventoryConsumer {
    @Autowired
    private InventoryService inventoryService;

    @RabbitHandler
    public void handleOrderCreated(OrderCreatedEvent event) {
        inventoryService.updateInventory(event.getOrderId());
    }
}

Technical Points

Message Protocol Selection : Choose RabbitMQ, Kafka or RocketMQ based on business needs

Message Format : Use JSON or Protobuf for cross‑language compatibility

Error Handling : Implement retry mechanisms and dead‑letter queues

Scenario 2: Asynchronous Processing

Background

When a user uploads a video, transcoding, thumbnail generation, and content moderation are time‑consuming; synchronous handling forces the user to wait.

MQ Solution

// Video service – producer
@Service
public class VideoService {
    @Autowired
    private KafkaTemplate<String, Object> kafkaTemplate;

    public UploadResponse uploadVideo(MultipartFile file, String userId) {
        // 1. Save raw video
        String videoId = saveOriginalVideo(file);
        // 2. Send processing message
        kafkaTemplate.send("video-processing", new VideoProcessingEvent(videoId, userId));
        // 3. Return immediate response
        return new UploadResponse(videoId, "upload_success");
    }
}

// Video processing service – consumer
@Service
@KafkaListener(topics = "video-processing")
public class VideoProcessingConsumer {
    public void processVideo(VideoProcessingEvent event) {
        videoProcessor.transcode(event.getVideoId());
        videoProcessor.generateThumbnails(event.getVideoId());
        contentModerationService.checkContent(event.getVideoId());
        // Notify user after processing
        notificationService.notifyUser(event.getUserId(), event.getVideoId());
    }
}

Architecture Advantages

Fast Response : User receives immediate acknowledgment after upload

Elastic Scaling : Consumer count can be adjusted based on processing load

Fault Isolation : Failure of processing services does not affect upload functionality

Scenario 3: Traffic Shaping (Peak‑Smoothing)

Background

During a flash‑sale, traffic can spike to hundreds of times the normal level, overwhelming databases and services.

MQ Solution

Code Implementation :

// SecKill service
@Service
public class SecKillService {
    @Autowired
    private RedisTemplate<String, Object> redisTemplate;
    @Autowired
    private RabbitTemplate rabbitTemplate;

    public SecKillResponse secKill(SecKillRequest request) {
        // 1. Validate user qualification
        if (!checkUserQualification(request.getUserId())) {
            return SecKillResponse.failed("User not qualified");
        }
        // 2. Pre‑decrement stock atomically
        Long remaining = redisTemplate.opsForValue().decrement("sec_kill_stock:" + request.getItemId());
        if (remaining == null || remaining < 0) {
            // Restore stock if insufficient
            redisTemplate.opsForValue().increment("sec_kill_stock:" + request.getItemId());
            return SecKillResponse.failed("Out of stock");
        }
        // 3. Send success message to MQ
        rabbitTemplate.convertAndSend(
            "sec_kill.exchange",
            "sec_kill.success",
            new SecKillSuccessEvent(request.getUserId(), request.getItemId())
        );
        return SecKillResponse.success("SecKill succeeded");
    }
}

// Order creation consumer
@Component
@RabbitListener(queues = "sec_kill.order.queue")
public class SecKillOrderConsumer {
    @RabbitHandler
    public void handleSecKillSuccess(SecKillSuccessEvent event) {
        orderService.createSecKillOrder(event.getUserId(), event.getItemId());
    }
}

Technical Points

Stock Pre‑Deduction : Use Redis atomic operations to avoid overselling

Queue Buffering : MQ buffers requests, protecting the database from spikes

Rate Limiting : Apply gateway‑level throttling to reject excess traffic

Scenario 4: Data Synchronization

Background

In a micro‑service architecture each service owns its database; data consistency across services must be ensured.

MQ Solution

// User service – send change event
@Service
@Transactional
public class UserService {
    public User updateUser(User user) {
        userDao.update(user);
        rocketMQTemplate.sendMessageInTransaction(
            "user-update-topic",
            MessageBuilder.withPayload(new UserUpdateEvent(user.getId(), user.getStatus())).build(),
            null
        );
        return user;
    }
}

// Other services – consume user update
@Service
@RocketMQMessageListener(topic = "user-update-topic", consumerGroup = "order-group")
public class UserUpdateConsumer implements RocketMQListener<UserUpdateEvent> {
    @Override
    public void onMessage(UserUpdateEvent event) {
        orderService.updateUserCache(event.getUserId(), event.getStatus());
    }
}

Consistency Guarantees

Local Transaction Table : Store message and business data in the same DB transaction

Transactional Message : Use RocketMQ’s transaction message mechanism

Idempotent Consumption : Consumers implement idempotency to avoid duplicate processing

Scenario 5: Log Collection

Background

In distributed systems logs are scattered across nodes and need centralized collection and analysis.

MQ Solution

Code Implementation :

// Log collector component
@Component
public class LogCollector {
    @Autowired
    private KafkaTemplate<String, String> kafkaTemplate;

    public void collectLog(String appId, String level, String message, Map<String, Object> context) {
        LogEvent logEvent = new LogEvent(appId, level, message, context, System.currentTimeMillis());
        kafkaTemplate.send("app-logs", appId, JsonUtils.toJson(logEvent));
    }
}

// Log consumer
@Service
@KafkaListener(topics = "app-logs", groupId = "log-es")
public class LogConsumer {
    public void consumeLog(String message) {
        LogEvent logEvent = JsonUtils.fromJson(message, LogEvent.class);
        elasticsearchService.indexLog(logEvent);
        if ("ERROR".equals(logEvent.getLevel())) {
            alertService.checkAndAlert(logEvent);
        }
    }
}

Technical Advantages

Decoupling : Application nodes do not need to know how logs are processed

Buffering : Handles log rate fluctuations

Multiple Consumers : Same log can be processed by several consumers

Scenario 6: Message Broadcasting

Background

When system configuration changes, all service nodes must update their local cache.

MQ Solution

// Config service – broadcast update
@Service
public class ConfigService {
    @Autowired
    private RedisTemplate<String, Object> redisTemplate;

    public void updateConfig(String key, String value) {
        configDao.updateConfig(key, value);
        redisTemplate.convertAndSend("config-update-channel", new ConfigUpdateEvent(key, value));
    }
}

// Service node – subscribe to updates
@Component
public class ConfigUpdateListener {
    @Autowired
    private LocalConfigCache localConfigCache;

    @RedisListener(channel = "config-update-channel")
    public void handleConfigUpdate(ConfigUpdateEvent event) {
        localConfigCache.updateConfig(event.getKey(), event.getValue());
    }
}

Application Scenarios

Feature toggles – dynamically enable/disable functions

Parameter adjustments – modify timeouts, rate‑limit thresholds, etc.

Blacklist/whitelist updates – refresh access control lists

Scenario 7: Ordered Messages

Background

In some business cases, the processing order of messages is critical, such as order‑status changes.

MQ Solution

// Order state service – send ordered message
@Service
public class OrderStateService {
    @Autowired
    private RocketMQTemplate rocketMQTemplate;

    public void changeOrderState(String orderId, String oldState, String newState) {
        OrderStateEvent event = new OrderStateEvent(orderId, oldState, newState);
        // Use orderId as sharding key to guarantee order
        rocketMQTemplate.syncSendOrderly("order-state-topic", event, orderId);
    }
}

// Order state consumer – ordered consumption
@RocketMQMessageListener(topic = "order-state-topic", consumerGroup = "order-state-group", consumeMode = ConsumeMode.ORDERLY)
public class OrderStateConsumer implements RocketMQListener<OrderStateEvent> {
    @Override
    public void onMessage(OrderStateEvent event) {
        orderService.processStateChange(event);
    }
}

Order Guarantee Mechanism

Partition Order : Messages in the same partition are ordered

Ordered Delivery : MQ ensures messages are delivered in send order

Ordered Processing : Consumer processes messages sequentially

Scenario 8: Delayed Messages

Background

Some tasks, such as automatically cancelling unpaid orders after a timeout, require delayed execution.

MQ Solution

// Order service – send delayed message
@Service
public class OrderService {
    @Autowired
    private RabbitTemplate rabbitTemplate;

    public void createOrder(Order order) {
        orderDao.save(order);
        // Send delayed message, 30 minutes later check payment status
        rabbitTemplate.convertAndSend(
            "order.delay.exchange",
            "order.create",
            new OrderCreateEvent(order.getId()),
            message -> {
                message.getMessageProperties().setDelay(30 * 60 * 1000); // 30 minutes
                return message;
            }
        );
    }
}

// Order timeout consumer
@Component
@RabbitListener(queues = "order.delay.queue")
public class OrderTimeoutConsumer {
    @RabbitHandler
    public void checkOrderPayment(OrderCreateEvent event) {
        Order order = orderDao.findById(event.getOrderId());
        if ("UNPAID".equals(order.getStatus())) {
            orderService.cancelOrder(order.getId(), "Timeout unpaid");
        }
    }
}

Alternative Solutions Comparison

Solution

Advantages

Disadvantages

Database Polling

Simple to implement

Low real‑time performance, high DB load

Delay Queue

Good real‑time performance

Complex implementation, possible message pile‑up

Scheduled Task

Strong controllability

Distributed coordination is complex

Scenario 9: Message Retry

Background

Temporary failures during message processing require a retry mechanism to eventually succeed.

MQ Solution

// Consumer with retry logic
@Service
@Slf4j
public class RetryableConsumer {
    @Autowired
    private RabbitTemplate rabbitTemplate;

    @RabbitListener(queues = "business.queue")
    public void processMessage(Message message, Channel channel) {
        try {
            businessService.process(message);
            channel.basicAck(message.getMessageProperties().getDeliveryTag(), false);
        } catch (TemporaryException e) {
            log.warn("Processing failed, will retry", e);
            // Requeue the message
            channel.basicNack(message.getMessageProperties().getDeliveryTag(), false, true);
        } catch (PermanentException e) {
            log.error("Processing failed, sending to dead‑letter queue", e);
            // Do not requeue, let dead‑letter queue handle it
            channel.basicNack(message.getMessageProperties().getDeliveryTag(), false, false);
        }
    }
}

Retry Strategies

Immediate Retry : Retry instantly for temporary faults

Delayed Retry : Gradually increase retry intervals

Dead‑Letter Queue : Unprocessable messages are routed to a dead‑letter queue

Scenario 10: Transactional Messages

Background

In distributed systems we need to guarantee data consistency across multiple services.

MQ Solution

// Transactional message producer
@Service
public class TransactionalMessageService {
    @Autowired
    private RocketMQTemplate rocketMQTemplate;

    @Transactional
    public void createOrderWithTransaction(Order order) {
        // 1. Save order in DB transaction
        orderDao.save(order);
        // 2. Send transactional message
        TransactionSendResult result = rocketMQTemplate.sendMessageInTransaction(
            "order-tx-topic",
            MessageBuilder.withPayload(new OrderCreatedEvent(order.getId())).build(),
            order // transaction argument
        );
        if (!result.getLocalTransactionState().equals(LocalTransactionState.COMMIT_MESSAGE)) {
            throw new RuntimeException("Transactional message send failed");
        }
    }
}

// Transactional message listener
@Component
@RocketMQTransactionListener
public class OrderTransactionListener implements RocketMQLocalTransactionListener {
    @Autowired
    private OrderDao orderDao;

    @Override
    public RocketMQLocalTransactionState executeLocalTransaction(Message msg, Object arg) {
        try {
            Order order = (Order) arg;
            Order exist = orderDao.findById(order.getId());
            if (exist != null && "CREATED".equals(exist.getStatus())) {
                return RocketMQLocalTransactionState.COMMIT_MESSAGE;
            } else {
                return RocketMQLocalTransactionState.ROLLBACK_MESSAGE;
            }
        } catch (Exception e) {
            return RocketMQLocalTransactionState.UNKNOWN;
        }
    }

    @Override
    public RocketMQLocalTransactionState checkLocalTransaction(Message msg) {
        String orderId = (String) msg.getHeaders().get("order_id");
        Order order = orderDao.findById(orderId);
        if (order != null && "CREATED".equals(order.getStatus())) {
            return RocketMQLocalTransactionState.COMMIT_MESSAGE;
        } else {
            return RocketMQLocalTransactionState.ROLLBACK_MESSAGE;
        }
    }
}

Transactional Message Flow

Conclusion

Through the ten scenarios we can summarise the core principles of using MQ:

Asynchronous Processing : Improves system response speed

System Decoupling : Reduces inter‑service dependencies

Traffic Shaping : Handles burst traffic

Data Synchronisation : Guarantees eventual consistency

Distributed Transactions : Solves data consistency problems across services

Technical Selection Advice

Scenario

Recommended MQ

Reason

High Throughput

Kafka

High throughput with persistent storage

Transactional Messages

RocketMQ

Full transactional message mechanism

Complex Routing

RabbitMQ

Flexible routing configuration

Delayed Messages

RabbitMQ

Native support for delayed queues

Best Practices

Message Idempotency : Consumers must implement idempotent handling

Dead‑Letter Queues : Provide fallback for permanently failed messages

Monitoring & Alerts : Set up metrics for message backlog and failure alerts

Performance Tuning : Adjust MQ parameters according to business characteristics