How to Build a Million‑User Ticket Spike System with Load Balancing, Redis, and Go

Introduction

During holidays, millions of users in major Chinese cities compete for train tickets, creating a massive spike in traffic that challenges any system’s QPS limits. The author studies the 12306 ticketing service architecture and shares insights on building a system that can serve 1 million users buying 10 000 tickets concurrently.

Large‑Scale High‑Concurrency Architecture

High‑concurrency systems typically use distributed clusters with multiple layers of load balancing to ensure high availability and even traffic distribution across servers.

Load‑Balancing Overview

The three layers of load balancing are illustrated in a diagram and include:

OSPF (Open Shortest Path First) – an interior gateway protocol that builds a link‑state database and calculates shortest paths, allowing up to six equal‑cost paths for load distribution.

LVS (Linux Virtual Server) – an IP‑level load‑balancer that hides server failures and presents a virtual high‑performance server.

Nginx – a high‑performance HTTP reverse proxy that supports round‑robin, weighted round‑robin, and IP‑hash scheduling.

Nginx Weighted Round‑Robin Demo

The following Nginx upstream configuration assigns different weights to four local services listening on ports 3001‑3004:

# Configure load balancing
upstream load_rule {
    server 127.0.0.1:3001 weight=1;
    server 127.0.0.1:3002 weight=2;
    server 127.0.0.1:3003 weight=3;
    server 127.0.0.1:3004 weight=4;
}
...
server {
    listen 80;
    server_name load_balance.com www.load_balance.com;
    location / {
        proxy_pass http://load_rule;
    }
}

Requests are distributed according to the weights, which is verified by logging request counts for each port.

Ticket Spike System Design

The core problem is to guarantee that tickets are neither oversold nor undersold while handling extreme concurrency. Three processing models are examined:

Order‑then‑deduct – create an order, then reduce stock; simple but incurs heavy DB I/O.

Pay‑then‑deduct – wait for payment before reducing stock; avoids oversell but blocks under high load.

Pre‑deduct (reserve stock) – reserve inventory first, generate orders asynchronously; reduces DB pressure.

The author adopts the pre‑deduct approach, storing a portion of the total inventory locally in memory and using Redis as a centralized stock counter. When a request arrives, the service first decrements the local counter; if successful, it then atomically decrements the Redis counter via a Lua script. Only when both succeed does the user receive a “ticket purchased” response.

Go Service Implementation

Key Go structures:

type LocalSpike struct {
    LocalInStock   int64
    LocalSalesVolume int64
}

type RemoteSpikeKeys struct {
    SpikeOrderHashKey string // Redis hash key for orders
    TotalInventoryKey string // Total tickets
    QuantityOfOrderKey string // Sold tickets
}

Redis connection pool initialization:

func NewPool() *redis.Pool {
    return &redis.Pool{
        MaxIdle:   10000,
        MaxActive: 12000,
        Dial: func() (redis.Conn, error) {
            return redis.Dial("tcp", ":6379")
        },
    }
}

Local stock deduction (returns bool):

func (spike *LocalSpike) LocalDeductionStock() bool {
    spike.LocalSalesVolume++
    return spike.LocalSalesVolume < spike.LocalInStock
}

Remote atomic deduction using Lua:

const LuaScript = `
local ticket_key = KEYS[1]
local total_key = ARGV[1]
local sold_key = ARGV[2]
local total = tonumber(redis.call('HGET', ticket_key, total_key))
local sold = tonumber(redis.call('HGET', ticket_key, sold_key))
if total >= sold then
    return redis.call('HINCRBY', ticket_key, sold_key, 1)
end
return 0
`
func (r *RemoteSpikeKeys) RemoteDeductionStock(conn redis.Conn) bool {
    lua := redis.NewScript(1, LuaScript)
    result, err := redis.Int(lua.Do(conn, r.SpikeOrderHashKey, r.TotalInventoryKey, r.QuantityOfOrderKey))
    if err != nil { return false }
    return result != 0
}

The HTTP handler combines both checks and writes the outcome to a log file:

func handleReq(w http.ResponseWriter, r *http.Request) {
    redisConn := redisPool.Get()
    <-done // channel used as a binary semaphore
    if localSpike.LocalDeductionStock() && remoteSpike.RemoteDeductionStock(redisConn) {
        util.RespJson(w, 1, "抢票成功", nil)
        LogMsg = "result:1,localSales:" + strconv.FormatInt(localSpike.LocalSalesVolume, 10)
    } else {
        util.RespJson(w, -1, "已售罄", nil)
        LogMsg = "result:0,localSales:" + strconv.FormatInt(localSpike.LocalSalesVolume, 10)
    }
    done <- 1
    writeLog(LogMsg, "./stat.log")
}

Performance Test

The service is stress‑tested with ApacheBench (ab) using 10 000 requests and 100 concurrent connections:

ab -n 10000 -c 100 http://127.0.0.1:3005/buy/ticket

Results on a low‑spec Mac show ~4 300 requests per second, average latency ~23 ms, and a clear split between successful and sold‑out responses in the log file.

Conclusion

The article demonstrates that a ticket‑spike system can achieve million‑level QPS by:

Distributing traffic with multi‑layer load balancing (OSPF, LVS, Nginx weighted round‑robin).

Avoiding database bottlenecks through local in‑memory stock and Redis atomic counters.

Leveraging Go’s lightweight goroutines and channel‑based locking for high‑throughput request handling.

These techniques illustrate how careful architecture, load distribution, and asynchronous processing can turn a seemingly impossible high‑concurrency problem into a stable, scalable solution.