Design and Implementation of Full-Scale Message Push Using a Public Mailbox in Baidu's IM System
The article explains Baidu's need for low‑cost, high‑latency full‑user notifications, analyzes existing IM architecture limitations, introduces a public mailbox read‑expansion approach, compares three push strategies, and details the design, management, caching, tiered release, advantages, drawbacks, and real‑world applications of this solution.
Baidu's App requires a mechanism to deliver notifications to all users with minimal disturbance and high timeliness; the existing private‑mailbox based approach cannot efficiently handle the write load for billions of messages.
The article outlines the background and requirements for full‑user messaging, emphasizing coverage of both active and recently registered users, rapid delivery (hour‑ or minute‑level), and a single‑touch notification without repeated disturbances.
Current IM architecture is described, including user entities, accounts, relationships, chat sessions, and the notify‑pull model where the client receives a notify signal over a long‑connected service and then pulls messages from the server.
Three conventional push mechanisms are compared: unicast, batch unicast, and broadcast, all of which write to each user's private mailbox and suffer from high storage and write costs at Baidu's scale (over 600 million monthly active users).
The proposed solution replaces the write‑expansion model with a read‑expansion model by introducing a public mailbox that stores a single copy of a notification; during message pull, the client merges private and public mailbox contents.
Implementation details include:
Full‑message management via an O‑side operation platform for editing, publishing, and withdrawing messages.
Connection layer, logic processing layer, mailbox proxy, and storage layer reusing existing IM services.
Memory‑cached public mailbox to handle peak pull traffic (≈47 k QPS) and periodic synchronization with MySQL storage.
Tiered release with whitelist support for pre‑release validation, configurable via CURL.
The design solves the write‑cost problem by shifting the burden to read operations, achieving faster distribution and lower resource consumption, while acknowledging limitations such as lack of personalization and unsuitability for real‑time alert scenarios.
Real‑world usage includes large‑scale notifications, feature updates, and message revocation within Baidu App, with documented deployments during the 2022 pandemic reopening that reached over 200 million users per campaign.
Future outlook suggests extending the public mailbox approach to broadcast and batch‑unicast revocation, further optimizing Baidu's instant‑messaging capabilities for cost‑effective, stable, and scalable user communication.
Top Architect
Top Architect focuses on sharing practical architecture knowledge, covering enterprise, system, website, large‑scale distributed, and high‑availability architectures, plus architecture adjustments using internet technologies. We welcome idea‑driven, sharing‑oriented architects to exchange and learn together.
How this landed with the community
Was this worth your time?
0 Comments
Thoughtful readers leave field notes, pushback, and hard-won operational detail here.