Inside 12306’s High‑Concurrency Ticket System: Architecture, Load Balancing & Go Demo

12306 Ticket Spike: Extreme Concurrency Insights

During holidays, millions of users compete for train tickets, creating a massive QPS load that rivals any flash‑sale system. The author studied 12306’s backend architecture and reproduced a simplified example that can serve 1 million users buying 10 000 tickets.

1. Large‑Scale High‑Concurrency Architecture

High‑concurrency systems are deployed as distributed clusters with multiple layers of load balancers and disaster‑recovery mechanisms (dual data centers, node fault tolerance, backup servers) to ensure high availability. Traffic is evenly distributed across servers.

1.1 Load Balancing Overview

The request flow passes through three layers of load balancing:

OSPF (Open Shortest Path First) – an interior gateway protocol that builds a link‑state database and can perform load balancing across up to six equal‑cost paths.

LVS (Linux Virtual Server) – IP‑level load balancing that distributes requests among a server pool while masking failures.

Nginx – a high‑performance HTTP reverse proxy offering round‑robin, weighted round‑robin, and IP‑hash strategies. The article focuses on weighted round‑robin.

1.2 Nginx Weighted Round‑Robin Demo

The upstream module is used to assign weights to four backend services listening on ports 3001‑3004.

# Configure load balancing
upstream load_rule {
    server 127.0.0.1:3001 weight=1;
    server 127.0.0.1:3002 weight=2;
    server 127.0.0.1:3003 weight=3;
    server 127.0.0.1:3004 weight=4;
}
...
server {
    listen       80;
    server_name  load_balance.com www.load_balance.com;
    location / {
        proxy_pass http://load_rule;
    }
}

A simple Go program starts four HTTP servers on those ports and logs request handling.

package main

import (
    "net/http"
    "os"
    "strings"
)

func main() {
    http.HandleFunc("/buy/ticket", handleReq)
    http.ListenAndServe(":3001", nil)
}

// handle request and write log
func handleReq(w http.ResponseWriter, r *http.Request) {
    failedMsg := "handle in port:"
    writeLog(failedMsg, "./stat.log")
}

func writeLog(msg string, logPath string) {
    fd, _ := os.OpenFile(logPath, os.O_RDWR|os.O_CREATE|os.O_APPEND, 0644)
    defer fd.Close()
    content := strings.Join([]string{msg, "
"}, "")
    fd.Write([]byte(content))
}

Requests are stress‑tested with ApacheBench (ab) to simulate 1 000 concurrent users.

ab -n 1000 -c 100 http://www.load_balance.com/buy/ticket

The logs show request distribution matching the configured weights (100, 200, 300, 400), confirming correct load balancing.

2. Flash‑Sale System Design Choices

To guarantee normal, stable service under extreme concurrency, the article examines three ordering strategies:

2.1 Order‑Then‑Deduct

Creating an order first and then reducing inventory ensures no overselling but incurs heavy DB I/O and risks “under‑selling” when users create orders without paying.

2.2 Pay‑Then‑Deduct

Waiting for payment before deducting inventory avoids under‑selling but can cause “overselling” under high load and still suffers from DB I/O bottlenecks.

2.3 Pre‑Deduction (Reservation)

Pre‑deduct inventory first, then create orders asynchronously (e.g., via MQ/Kafka). If a user does not pay within a timeout, the reserved stock is released back. This reduces DB I/O and improves response speed.

3. The Art of Stock Deduction

Local in‑memory stock deduction avoids DB bottlenecks. Each server holds a portion of total tickets (e.g., 100 tickets per server across 100 servers). After local deduction succeeds, a remote Redis deduction is performed to ensure global consistency.

Redis stores a hash with total inventory and sold count. A Lua script atomically checks availability and increments the sold count.

const LuaScript = `
    local ticket_key = KEYS[1]
    local ticket_total_key = ARGV[1]
    local ticket_sold_key = ARGV[2]
    local ticket_total_nums = tonumber(redis.call('HGET', ticket_key, ticket_total_key))
    local ticket_sold_nums = tonumber(redis.call('HGET', ticket_key, ticket_sold_key))
    if (ticket_total_nums >= ticket_sold_nums) then
        return redis.call('HINCRBY', ticket_key, ticket_sold_key, 1)
    end
    return 0
`

Initialization of Redis inventory:

hmset ticket_hash_key "ticket_total_nums" 10000 "ticket_sold_nums" 0

4. Go Implementation

Initialization sets local stock, Redis keys, and a channel used as a lightweight lock.

... // localSpike struct
type LocalSpike struct {
    LocalInStock     int64
    LocalSalesVolume int64
}

... // remoteSpike struct and Redis pool
type RemoteSpikeKeys struct {
    SpikeOrderHashKey string
    TotalInventoryKey string
    QuantityOfOrderKey string
}

Local deduction simply increments a counter and checks against the local limit.

func (spike *LocalSpike) LocalDeductionStock() bool {
    spike.LocalSalesVolume = spike.LocalSalesVolume + 1
    return spike.LocalSalesVolume < spike.LocalInStock
}

Remote deduction runs the Lua script via redigo.

func (RemoteSpikeKeys *RemoteSpikeKeys) RemoteDeductionStock(conn redis.Conn) bool {
    lua := redis.NewScript(1, LuaScript)
    result, err := redis.Int(lua.Do(conn, RemoteSpikeKeys.SpikeOrderHashKey, RemoteSpikeKeys.TotalInventoryKey, RemoteSpikeKeys.QuantityOfOrderKey))
    if err != nil {
        return false
    }
    return result != 0
}

The HTTP handler performs both local and remote deductions; on success it returns a success JSON, otherwise “sold out”. Logs are written to stat.log.

func handleReq(w http.ResponseWriter, r *http.Request) {
    redisConn := redisPool.Get()
    var LogMsg string
    <-done // acquire lock
    if localSpike.LocalDeductionStock() && remoteSpike.RemoteDeductionStock(redisConn) {
        util.RespJson(w, 1, "抢票成功", nil)
        LogMsg = "result:1,localSales:" + strconv.FormatInt(localSpike.LocalSalesVolume, 10)
    } else {
        util.RespJson(w, -1, "已售罄", nil)
        LogMsg = "result:0,localSales:" + strconv.FormatInt(localSpike.LocalSalesVolume, 10)
    }
    done <- 1 // release lock
    writeLog(LogMsg, "./stat.log")
}

Performance testing with ab -n 10000 -c 100 http://127.0.0.1:3005/buy/ticket shows ~4 300 requests per second on a modest Mac, with uniform request distribution and stable Redis performance.

5. Summary

The prototype demonstrates how to build a high‑concurrency ticket‑spike system that avoids DB I/O by using in‑memory stock, Redis for global consistency, weighted Nginx load balancing, and Go’s native concurrency. It ensures no overselling or underselling, tolerates partial server failures via buffer stock, and leverages asynchronous processing for scalability.