How to Simulate 100 Million WeChat Red‑Packet Requests on a Single Server

Introduction

Inspired by a 2015 article on building a reliable Spring Festival red‑packet system, the author reproduces the scenario to evaluate whether a single server can handle the load of 100 billion shake‑red‑packet requests.

Target Goals

Support at least 1 million concurrent connections, achieve a peak QPS of 60 000, and reliably process shake‑red‑packet and send‑red‑packet operations.

Load Estimation

Based on 638 backend servers handling 1.43 billion users, a single server would serve roughly 228 000 users. Assuming 600 active servers, the per‑server user count is about 900 000. The target QPS for the whole system is 14 million, which translates to roughly 23 000 QPS per server; the experiment pushes this to 30 000 QPS and 60 000 QPS.

Hardware & Software

Software: Go 1.8, shell, Python. OS: Ubuntu 12.04 (server) and Debian 5.0 (client). Hardware: Dell R2950 8‑core, 16 GB RAM for the server; 17 ESXi virtual machines (4 CPU, 5 GB RAM each) as clients, each maintaining 60 000 connections.

Architecture

The design splits one million connections into multiple independent SET groups, each managing a few thousand connections. Each SET has a dedicated goroutine for message handling, reducing the total goroutine count to roughly the number of connections rather than millions. Red‑packet generation is performed by a lightweight service that distributes packets evenly across SETs.

Implementation Highlights

Client coordination uses NTP‑synchronised timestamps to decide when each user should send a request, ensuring an even distribution of QPS. The server records per‑second request counters and monitors network traffic using a Python script combined with ethtool. The following alias was used to count active connections:

Alias ss2='ss -ant | grep 1025 | grep EST | awk -F: "{print $8}" | sort | uniq -c'

Testing Phases

Phase 1: Start server and monitoring, launch 17 client VMs, establish one million connections, verify connection counts.

Phase 2: Adjust client QPS to 30 000, observe stable request rates, generate red‑packets at 200 /s, and confirm client receipt.

Phase 3: Increase client QPS to 60 000, repeat the red‑packet generation and collection steps, and record performance degradation.

Results

At 30 000 QPS the server handled traffic stably; at 60 000 QPS occasional spikes and network packet loss caused QPS fluctuations. Profiling showed GC pauses over 10 ms on the old hardware, which was acceptable for the prototype.

Conclusion

The prototype achieved the design goals: a single server can sustain 1 million connections and process at least 30 000 QPS (up to 60 000 QPS with some instability). Scaling to 600 servers would complete 100 billion shake‑red‑packet requests in roughly 7 minutes, demonstrating that the architecture is horizontally scalable and viable for large‑scale red‑packet services.