Rebuilding Huajiao Live Chat with Go: Architecture, Scaling & Flow Control

Background

Huajiao's early live streaming platform used a long‑connection push service and chat/danmaku service built on an internal Erlang system. As user volume grew and business lines evolved, the Erlang‑based service became a performance bottleneck, prompting a complete backend reconstruction.

Refactor

Development Language

Go was chosen because the company already had extensive high‑concurrency Go services and strong internal expertise.

Erlang, while historically strong in concurrency and hot upgrades, suffered from weaker libraries for HTTP, Unicode, and JSON, and required more effort to maintain.

Architecture

The legacy Erlang system (named "TalkTalk") consisted of modular components such as msgrouter, srm, and distribute. The new QChat architecture was designed to be fully compatible with client protocols while separating connection handling from business logic.

Key design points:

Full compatibility with existing client protocols and business interfaces.

Lightweight long‑connection layer that avoids embedding core business logic.

Unified component interfaces to reduce maintenance complexity.

Centralized access to Redis, MongoDB, MySQL, etc.

Dynamic configuration.

Upgrade Switch

The migration was performed over roughly four weeks. Initially QChat ran as a sub‑system, receiving forwarded requests from TalkTalk. Read‑only queries were served by TalkTalk, while QChat only handled notifications. Gradual traffic shifting via DNS, LVS, and gray‑release ensured a seamless, transparent upgrade for clients.

Flow Control

Initial simple degradation rules (e.g., dropping low‑priority messages) were insufficient for massive events like the Olympics broadcast, where hundreds of thousands of users generated over 10,000 messages per second. The system was enhanced to categorize messages by display position and apply per‑category rate limits, ensuring critical paid messages (gifts, flying screens) were never dropped.

Four‑Layer Proxy

To improve connectivity for overseas users, Huajiao evaluated two CDN providers offering TCP four‑layer proxy services. Tests across US, Asia, and Hong Kong showed lower connection latency and more stable push latency compared to direct connections. The chosen provider (Wangsu) became the primary proxy, with another (BaiXun) as a cold backup.

Performance before and after proxy deployment:

Before:

After:

The proxy eliminated connection failures for overseas users and was later extended to all Huajiao services, improving HTTP/HTTPS API accessibility globally.

Server‑Side Architecture

The final backend includes new components: gateway (handles connections), router (protocol parsing and chat routing), saver (unified storage access), center (business interface), plus coordinator and dispatcher for load balancing across IDC data centers.

The coordinator caches messages, performs per‑IDC write‑spreading, and dynamically throttles traffic when an IDC's outbound bandwidth is insufficient, ensuring stable client experience without overload.