How WeChat Generates Trillions of Sequence Numbers with Sub‑Millisecond Latency

Background

WeChat assigns a unique, monotonically increasing sequence number (called a sequence) to every piece of data that needs to be synchronized with the client, such as messages. The client sends its latest known sequence; the server returns the incremental data, ensuring reliable synchronization and reducing redundant traffic.

seqsvr Overview

The sequence numbers are generated by a dedicated service called seqsvr , which handles billions of calls per day. Each request for a sequence normally takes 1 ms, and 99.9 % of calls finish within 3 ms. The service runs on hundreds of 4‑core servers.

Core Properties of a Sequence

64‑bit incrementing integer. Each user has an independent 64‑bit sequence space.

This independence avoids global contention and simplifies consistency checks across different data types.

Architecture Prototype

Conceptually, seqsvr can be seen as a massive 64‑bit array where each user occupies an 8‑byte slot storing the latest allocated sequence (cur_seq). Allocating a new sequence simply increments cur_seq and returns it.

Pre‑allocation Layer

To avoid disk I/O at the trillion‑level QPS, seqsvr keeps the latest sequence (cur_seq) and an allocation ceiling (max_seq) in memory. When cur_seq exceeds max_seq, max_seq is increased by a step (e.g., 10 000) and persisted. On restart, max_seq is loaded into cur_seq.

Section‑Shared Storage

Storing max_seq for every user would require ~32 GB. To reduce this, users are grouped into sections; a section shares a single max_seq. With 100 000 users per section, the total persisted data is only a few hundred kilobytes.

Engineering Implementation

The system is split into two modules:

StoreSvr – the persistent storage layer using multi‑node NRW replication for durability. AllocSvr – an in‑memory allocation layer deployed on many machines, each handling a set of sections.

Sets of uid ranges are isolated; each set contains its own StoreSvr + AllocSvr pair, providing disaster isolation.

Disaster Recovery Design

Two guiding principles: keep the architecture simple and avoid strong dependencies on external components. The core requirement is that each uid’s sequence must be strictly monotonic.

AllocSvr must be the sole provider for a given uid at any moment. An arbitration service monitors AllocSvr health and writes the routing configuration to StoreSvr. AllocSvr periodically reads this configuration, establishing a lease: if it cannot read the config for N seconds, it stops serving.

Disaster 1.0 – Primary‑Backup

Each set has a primary and a cold standby AllocSvr. The standby takes over when the primary fails, but this model suffers from idle resources and difficult scaling.

Disaster 2.0 – Embedded Routing Table

AllocSvr now embeds the full uid‑to‑AllocSvr routing table in the response to a sequence request. Clients cache this table and its version. When a client’s version is stale, AllocSvr returns the updated table, allowing immediate correction without external configuration changes.

Routing Synchronization Optimization

Clients first use their cached routing table; if missing, they pick a random AllocSvr. Requests include the table’s version. AllocSvr validates the version and, if outdated, attaches the latest table to the response. Clients update their cache and optionally retry.

Conclusion

The seqsvr design—simple, high‑performance pre‑allocation, section sharing, and evolving disaster‑recovery mechanisms—has reliably supported WeChat’s massive scale for years and continues to be a cornerstone of its backend infrastructure.