Unlocking the Power of Message Queues: Decoupling, Load Smoothing, and Consistency

Decoupling

Message queues (MQ) allow two independent systems to communicate without a direct, synchronous connection. In an order‑payment scenario, the order service publishes a message to MQ and immediately returns success to the user, while the settlement service consumes the message later to deduct the account balance, keeping each service focused on its own responsibility.

Peak Shaving (削峰填谷)

During promotional spikes, an order service can generate a massive burst of requests that would overload the settlement service if called directly via RPC. By routing orders through MQ, the consumption rate is controlled by the settlement side, flattening the traffic curve and preventing resource saturation.

In pull mode, the consumer actively calls pull(topic, listener) in a loop, deciding how many messages to fetch and processing them with multithreading if needed. Example:

messageConsumer.start();
for (;;) {
    // manual pull
    messageConsumer.pull(topic, messageListener);
}

The method signature is pull(String topic, MessageListener listener), where topic identifies the queue and listener receives each message via onMessage(). Unlike listener mode, pull mode requires the client to trigger consumption, giving full control over throughput.

Final Consistency

Internet services often settle for eventual consistency rather than strong consistency. Using the same order‑settlement example, the order creation and the MQ notification can be wrapped in a local transaction so that either both succeed or both fail. If sending to MQ fails, a scheduled compensation task retries the send, ensuring the message eventually lands in the queue. The settlement side keeps retrying until it successfully processes the deduction, achieving idempotent, eventually consistent state.

Broadcast Consumption

MQ supports two messaging models: point‑to‑point and publish/subscribe. Within publish/subscribe, there are cluster consumption (one random consumer in the group processes each message) and broadcast consumption (every consumer in the group receives the message). Broadcast is useful for scenarios like push notification services where each server must see the same event.

Broadcast guarantees each server receives the message at least once, but it does not provide acknowledgments, so duplicate consumption is possible and offset tracking differs from cluster mode.

Simulating Broadcast with Cluster Consumption

Because broadcast cannot guarantee ordering and may cause duplicate processing, a common workaround is to use cluster consumption with distinct consumer identifiers (APPIDs) on each server. By assigning a unique APPID per instance, the same message is effectively delivered to every server, mimicking broadcast while retaining cluster semantics.

Retry Pitfalls

MQ’s retry mechanism ensures that messages are eventually processed, but it also forces developers to handle idempotency. In a payment‑deduction flow, if the order service retries a request after a network glitch, the settlement service may receive the same transaction twice. Using a stable transaction identifier (e.g., a serial number) and making the settlement service check for prior processing prevents duplicate deductions.

Conclusion

The article outlines common MQ usage scenarios—decoupling services, smoothing traffic peaks, achieving eventual consistency, and employing broadcast consumption—while highlighting practical considerations such as pull vs. listener modes, idempotent retries, and ways to simulate broadcast with cluster consumption. Applying these patterns helps developers harness MQ’s strengths for robust backend systems.