Building a Million‑Scale WebSocket Push Service with Go
This article compares pull and push models, explains WebSocket fundamentals, evaluates Node.js, C/C++ and Go for server implementation, provides complete Go and HTML client code, analyzes kernel, lock and CPU bottlenecks of a ten‑million‑user push system, and presents concrete optimization and clustering strategies.
WebSocket enables server‑initiated data delivery. Pull (periodic polling) suffers from low update frequency, high query load with many online users, and cannot meet timeliness requirements. Push via WebSocket sends data only when updates occur, maintains long‑lived connections, and delivers messages instantly, making it suitable for high‑frequency real‑time scenarios.
Server language selection
Node.js : single‑threaded model (even with multiple processes) limits push performance.
C/C++ : implementing TCP communication and the WebSocket protocol incurs high development cost.
Go : lightweight goroutine concurrency, compiled speed, and the mature github.com/gorilla/websocket library avoid reinventing the protocol.
Go WebSocket server implementation
package main
import (
"net/http"
"github.com/gorilla/websocket"
"github.com/myproject/gowebsocket/impl"
"time"
)
var upgrader = websocket.Upgrader{CheckOrigin: func(r *http.Request) bool {return true}}
func wsHandler(w http.ResponseWriter, r *http.Request) {
var (
wsConn *websocket.Conn
err error
conn *impl.Connection
data []byte
)
if wsConn, err = upgrader.Upgrade(w, r, nil); err != nil {return}
if conn, err = impl.InitConnection(wsConn); err != nil {goto ERR}
go func() {
var err error
for {
if err = conn.WriteMessage([]byte("heartbeat")); err != nil {return}
time.Sleep(1 * time.Second)
}
}()
for {
if data, err = conn.ReadMessage(); err != nil {goto ERR}
if err = conn.WriteMessage(data); err != nil {goto ERR}
}
ERR:
conn.Close()
}
func main() {
http.HandleFunc("/ws", wsHandler)
http.ListenAndServe("0.0.0.0:7777", nil)
}HTML client for testing
<!DOCTYPE html>
<html>
<head>
<title>go websocket</title>
<meta charset="utf-8"/>
</head>
<body>
<script type="text/javascript">
var wsUri = "ws://127.0.0.1:7777/ws";
var output;
function init() {output = document.getElementById("output"); testWebSocket();}
function testWebSocket() {
websocket = new WebSocket(wsUri);
websocket.onopen = function(evt) {onOpen(evt);};
websocket.onclose = function(evt) {onClose(evt);};
websocket.onmessage = function(evt) {onMessage(evt);};
websocket.onerror = function(evt) {onError(evt);};
}
function onOpen(evt) {writeToScreen("CONNECTED");}
function onClose(evt) {writeToScreen("DISCONNECTED");}
function onMessage(evt) {writeToScreen("RESPONSE: " + evt.data);}
function onError(evt) {writeToScreen("ERROR: " + evt.data);}
function writeToScreen(message) {
var pre = document.createElement("p");
pre.style.wordWrap = "break-word";
pre.innerHTML = message;
output.appendChild(pre);
}
window.addEventListener("load", init, false);
</script>
<h2>WebSocket Test</h2>
<input type="text" id="input"/>
<button onclick="sendBtnClick()">send</button>
<button onclick="closeBtnClick()">close</button>
<div id="output"></div>
<script>
function sendBtnClick(){
var msg = document.getElementById("input").value;
websocket.send(msg);
document.getElementById("input").value = '';
}
function closeBtnClick(){websocket.close();}
</script>
</body>
</html>The handshake flow: the client sends an HTTP request with an Upgrade header, the server replies with 101 Switching Protocols, and the underlying TCP connection remains open for bidirectional WebSocket frames.
Scalability bottlenecks for ten‑million concurrent users
Kernel packet‑sending limit : approximately 100 W packets per second.
Lock contention : a single dictionary holding all connections forces a global lock during push.
CPU overhead : JSON encoding for each message; pushing to 100 W online users at 10 messages per second requires 100 W JSON encodings per second.
Optimizations applied
Network packet reduction : aggregate N messages within one second into a single packet, turning per‑message sends into per‑second sends.
Lock sharding : split the connection map into multiple shards, each with its own lock, allowing parallel pushes without lock competition.
Read/Write lock : replace a mutex with a sync.RWMutex so multiple push goroutines can traverse the same shard concurrently.
CPU saving : pre‑encode JSON once per aggregated batch, then broadcast the encoded payload.
Clustering : deploy multiple nodes behind a load balancer; use a logical cluster where messages are broadcast to all gateway nodes.
Internal distribution : employ HTTP/2 between gateways for connection multiplexing and high‑throughput RPC; expose an external HTTP/1 API for client simplicity.
Business services call the HTTP API of the logical cluster; the cluster forwards the message to all gateways, each of which pushes it to its online connections. The described challenges and solutions enable a functional, high‑throughput bullet‑style message push service.
Signed-in readers can open the original source through BestHub's protected redirect.
This article has been distilled and summarized from source material, then republished for learning and reference. If you believe it infringes your rights, please contactand we will review it promptly.
Golang Shines
We share daily the latest Golang technical articles, practical resources, language news, tutorials, and real-world projects to help everyone learn and improve.
How this landed with the community
Was this worth your time?
0 Comments
Thoughtful readers leave field notes, pushback, and hard-won operational detail here.