How to Build a Million-User Ticket Spike System with Go, Nginx, and Redis

12306 Ticket Spike: Lessons from Extreme Concurrency

During holidays, millions of users in major Chinese cities compete for train tickets, creating a massive spike in traffic that challenges any system’s QPS capacity.

The author studied 12306’s backend architecture and shares key design highlights, then demonstrates a simulation where 1 million users simultaneously attempt to purchase 10 000 tickets while maintaining stable service.

Large‑Scale High‑Concurrency Architecture

High‑concurrency systems typically use distributed clusters with multiple layers of load balancers and various fault‑tolerance mechanisms (dual data centers, node failover, disaster recovery) to ensure high availability. Traffic is balanced across servers based on capacity and configuration.

Load‑Balancing Overview

The diagram shows three layers of load balancing:

OSPF – an interior gateway protocol that builds a link‑state database and computes shortest‑path trees; costs are inversely proportional to bandwidth, enabling up to six equal‑cost paths.

LVS (Linux Virtual Server) – a cluster technology using IP load balancing and content‑based request distribution, providing high throughput and automatic server failure masking.

Nginx – a high‑performance HTTP reverse proxy that supports round‑robin, weighted round‑robin, and IP‑hash load‑balancing methods.

The following Nginx weighted round‑robin configuration distributes traffic across four backend services with weights 1, 2, 3, 4.

# Configure load balancing
upstream load_rule {
    server 127.0.0.1:3001 weight=1;
    server 127.0.0.1:3002 weight=2;
    server 127.0.0.1:3003 weight=3;
    server 127.0.0.1:3004 weight=4;
}
server {
    listen 80;
    server_name load_balance.com www.load_balance.com;
    location / {
        proxy_pass http://load_rule;
    }
}

Local hosts are configured to resolve www.load_balance.com to the test machine.

Spike System Design Choices

Three typical order‑processing flows are examined:

Order‑then‑deduct‑stock: Create an order first, then reduce inventory. Guarantees no oversell but incurs heavy DB I/O under extreme load.

Pay‑then‑deduct‑stock: Reduce stock after payment. Prevents undersell but still suffers from high DB pressure and can cause “sold‑out” responses while stock remains.

Pre‑deduct‑stock (reserve inventory): Reserve stock in memory, generate orders asynchronously, and enforce a payment timeout to release unused stock. This minimizes DB I/O and improves response speed.

The author adopts the pre‑deduct approach, using local in‑memory stock and a remote Redis hash for unified stock management.

Local Stock Deduction

Each server holds a portion of the total tickets (e.g., 100 tickets per machine). When a request arrives, the server increments its local sales counter and checks against its local stock limit.

Unified Stock Deduction with Redis

Redis stores the global inventory as a hash. A Lua script ensures atomic check‑and‑decrement operations:

local ticket_key = KEYS[1]
local ticket_total_key = ARGV[1]
local ticket_sold_key = ARGV[2]
local ticket_total_nums = tonumber(redis.call('HGET', ticket_key, ticket_total_key))
local ticket_sold_nums = tonumber(redis.call('HGET', ticket_key, ticket_sold_key))
if (ticket_total_nums >= ticket_sold_nums) then
    return redis.call('HINCRBY', ticket_key, ticket_sold_key, 1)
end
return 0

The Go implementation initializes Redis connection pools, defines the hash keys, and invokes the script for each successful local deduction.

Go Service Implementation

The service starts four HTTP listeners (ports 3001‑3004) with identical handlers. The handler performs:

Acquire a channel‑based lock to serialize local stock updates.

Execute LocalDeductionStock() and RemoteDeductionStock().

Return JSON indicating success ( 抢票成功) or sold‑out ( 已售罄).

func handleReq(w http.ResponseWriter, r *http.Request) {
    redisConn := redisPool.Get()
    <-done
    if localSpike.LocalDeductionStock() && remoteSpike.RemoteDeductionStock(redisConn) {
        util.RespJson(w, 1, "抢票成功", nil)
    } else {
        util.RespJson(w, -1, "已售罄", nil)
    }
    done <- 1
    writeLog(...)
}

Performance Testing

Using ApacheBench ( ab -n 10000 -c 100 http://127.0.0.1:3005/buy/ticket) the single‑machine setup handled over 4 000 requests per second with average latency ~23 ms and zero failures. Log analysis confirmed correct request distribution and stock depletion.

Key Takeaways

Layered load balancing distributes massive traffic across many servers, reducing per‑node load.

Combining local in‑memory stock with remote Redis stock ensures atomicity while avoiding frequent DB writes.

Channel‑based locking in Go provides lightweight concurrency control without heavy mutex contention.

Asynchronous order creation and payment timeout mechanisms prevent oversell and undersell.

Overall, the article demonstrates a practical, high‑performance spike system architecture that leverages Go’s concurrency model, Nginx load balancing, and Redis’s fast in‑memory operations to handle extreme traffic while maintaining data consistency.