How WeChat’s MQ 2.0 Redefined Asynchronous Queues for Massive Scale

Background Introduction

MQ is a self‑developed asynchronous queue component used extensively in WeChat’s backend. MQ 1.0 satisfied basic async needs with high‑performance single‑machine persistence and scheduling, consisting of an MQ layer for persistence and a Worker layer for processing.

Key characteristics of the original design were single‑machine focus and full lifecycle management (enqueue, dispatch, process, commit, destroy).

As business grew, MQ 1.0 became insufficient, leading to MQ 2.0 with three major optimizations: better task scheduling, more efficient task processing, and stronger overload protection.

Better Task Scheduling

Current Situation

IOS push notifications illustrate a scenario where network quality varies across IDC sites, causing worker consumption slowdown and message backlog.

To address this, a cross‑machine consumption model was introduced, aiming for a decentralized, adaptive consumption network.

Core Scheduling Problem

The central question is which worker should handle which task when consumption expands from single‑machine to multi‑machine.

Rule: Prefer local consumption; only when backlog occurs, trigger cross‑machine consumption.

Pull vs. Push

Push: low latency but backlog accumulates on workers and cannot be re‑scheduled.

Pull: backlog stays in MQ, allowing idle workers to pull; latency slightly higher.

MQ 2.0 adopts the pull approach to avoid worker‑side backlog.

Worker Awareness of Backlog

A broadcast mode pushes backlog information to all workers via long‑living connections, with flexible subscription filters for precise targeting.

Eliminating Backlog

Workers first clear local backlog; only when they have spare capacity do they help others, using prioritized pulling.

Load‑Balancing Analysis

The cross‑machine model is fully decentralized; while perfect balance is impossible, the system self‑adjusts during overload to achieve relative equilibrium.

Summary

MQ and Workers can be deployed independently.

Local backlog can be cleared by scaling Workers.

Worker failures are automatically compensated by other Workers.

More Efficient Task Processing

MapReduce‑Like Framework

Group chat batch delivery is modeled as a series of MapReduce processes. MQ 2.0 provides a generic MapReduce task framework with concurrent scheduling and pool isolation, simplifying complex nested parallelism.

Stream Task Framework

After a task finishes, it can emit new tasks that are directly enqueued by the MQ internal framework, offering a lightweight, transactional asynchronous model.

Summary

The MapReduce‑style and stream task frameworks give developers powerful tools to handle complex, high‑throughput workloads.

Stronger Overload Protection

Current Limitations

MQ 1.0 relied on manual queue speed configuration, which could not adapt to real‑time overload conditions.

Forward Throttling

CPU‑based flow control: When CPU usage spikes, task processing speed is reduced to preserve buffering capacity.

Success‑rate‑based flow control: High failure rates indicate backend issues; MQ limits retry speed based on observed success metrics.

Backward Throttling

Business services can feed back control signals to MQ, allowing it to adjust dispatch speed based on actual backend RPC traffic.

Summary

MQ 2.0 implements both forward and backward flow‑control mechanisms, providing a solid foundation for overload protection and future fine‑grained strategies.

Final Thoughts

Compared with other queue solutions, WeChat’s MQ is tightly aligned with real business scenarios, greatly boosting developer productivity. MQ 2.0 has been deployed across core services and survived the 2017 New Year traffic peak. Ongoing work will focus on persistence disaster recovery and scheduling performance.