How to Build a Million‑User Ticket‑Sniping System with Load Balancing, Redis, and Go

Problem Overview

During holidays, millions of users in major Chinese cities compete for train tickets, causing the 12306 ticketing platform to experience extreme spikes in QPS that exceed typical flash‑sale systems.

System Architecture

The backend adopts a three‑layer load‑balancing hierarchy: OSPF for internal routing, LVS for IP‑level load distribution, and Nginx for HTTP reverse‑proxy with weighted round‑robin. This distributes traffic evenly across a cluster of servers.

Nginx Weighted Round‑Robin

Each upstream server is assigned a weight reflecting its capacity. The following configuration balances four instances on ports 3001‑3004 with weights 1, 2, 3, and 4 respectively.

# Configure load balancing
upstream load_rule {
    server 127.0.0.1:3001 weight=1;
    server 127.0.0.1:3002 weight=2;
    server 127.0.0.1:3003 weight=3;
    server 127.0.0.1:3004 weight=4;
}
server {
    listen 80;
    server_name load_balance.com www.load_balance.com;
    location / {
        proxy_pass http://load_rule;
    }
}

Stock Deduction Strategies

Three classic approaches are examined:

Order‑then‑deduct : create order first, then reduce inventory – simple but creates heavy DB I/O.

Pay‑then‑deduct : wait for payment before reducing stock – avoids overselling but can cause “oversell” under extreme concurrency.

Pre‑deduct (reserve stock) : reserve inventory in memory, generate orders asynchronously, and use a timeout to release unused stock.

Pre‑deduction is identified as the most suitable for high‑concurrency ticketing.

Local Stock Deduction

Each server holds a portion of the total tickets in memory. The local deduction function increments a sales counter and checks against the local stock limit.

package localSpike
// Local stock deduction, returns bool
func (spike *LocalSpike) LocalDeductionStock() bool {
    spike.LocalSalesVolume = spike.LocalSalesVolume + 1
    return spike.LocalSalesVolume < spike.LocalInStock
}

Unified Stock with Redis

Redis stores the global ticket inventory using a hash. A Lua script guarantees atomic check‑and‑increment operations.

const LuaScript = `
    local ticket_key = KEYS[1]
    local ticket_total_key = ARGV[1]
    local ticket_sold_key = ARGV[2]
    local ticket_total_nums = tonumber(redis.call('HGET', ticket_key, ticket_total_key))
    local ticket_sold_nums = tonumber(redis.call('HGET', ticket_key, ticket_sold_key))
    if (ticket_total_nums >= ticket_sold_nums) then
        return redis.call('HINCRBY', ticket_key, ticket_sold_key, 1)
    end
    return 0
`
func (r *RemoteSpikeKeys) RemoteDeductionStock(conn redis.Conn) bool {
    lua := redis.NewScript(1, LuaScript)
    result, err := redis.Int(lua.Do(conn, r.SpikeOrderHashKey, r.TotalInventoryKey, r.QuantityOfOrderKey))
    if err != nil { return false }
    return result != 0
}

Go Service Implementation

The service initializes local stock, Redis connection pool, and a channel used as a binary semaphore to serialize critical sections.

package main
import (
    "net/http"
    "os"
    "strings"
)
func main() {
    http.HandleFunc("/buy/ticket", handleReq)
    http.ListenAndServe(":3005", nil)
}
func handleReq(w http.ResponseWriter, r *http.Request) {
    redisConn := redisPool.Get()
    <-done // acquire semaphore
    var logMsg string
    if localSpike.LocalDeductionStock() && remoteSpike.RemoteDeductionStock(redisConn) {
        util.RespJson(w, 1, "抢票成功", nil)
        logMsg = "result:1,localSales:" + strconv.FormatInt(localSpike.LocalSalesVolume, 10)
    } else {
        util.RespJson(w, -1, "已售罄", nil)
        logMsg = "result:0,localSales:" + strconv.FormatInt(localSpike.LocalSalesVolume, 10)
    }
    done <- 1 // release semaphore
    writeLog(logMsg, "./stat.log")
}

Performance Testing

ApacheBench (ab) was used to simulate 10,000 requests with 100 concurrent connections. The single‑machine test on a low‑spec Mac achieved ~4,300 requests per second with an average latency of 23 ms.

ab -n 10000 -c 100 http://127.0.0.1:3005/buy/ticket

Log analysis showed a smooth distribution of successful and failed requests, confirming that the weighted load balancing and Redis stock control work as intended.

Conclusion

The article demonstrates a practical high‑concurrency ticket‑sniping architecture that avoids heavy database I/O by combining local in‑memory stock, Redis‑based unified inventory, multi‑layer load balancing, and Go’s native concurrency model. It also highlights two key takeaways: (1) distribute load across many nodes to scale, and (2) leverage asynchronous processing wherever possible to maximize CPU utilization.