Building a 1M‑Request Ticket‑Grab System with Go, Nginx, and Redis

Background

During holidays, millions of users in China compete for train tickets on the 12306 platform, creating a flash‑sale scenario with QPS levels that surpass most commercial systems. The author examines 12306’s server architecture and reproduces a similar environment where 1 million users attempt to purchase 10 000 tickets concurrently.

1. Large‑Scale High‑Concurrency Architecture

To sustain extreme traffic, the system is deployed as a distributed cluster with multiple layers of load balancers and disaster‑recovery mechanisms (dual data centers, node fault tolerance, backup servers). Traffic is evenly distributed across many servers.

1.1 Load‑Balancing Overview

OSPF – an interior gateway protocol that builds a link‑state database and can perform load‑balancing across up to six equal‑cost paths.

LVS – Linux Virtual Server provides IP‑level load‑balancing and masks server failures, presenting a single virtual server to clients.

Nginx – widely used HTTP reverse proxy; supports round‑robin, weighted round‑robin, and IP‑hash. The article focuses on weighted round‑robin.

1.2 Nginx Weighted Round‑Robin Demo

The following Nginx upstream configuration assigns different weights to four backend services listening on ports 3001‑3004.

# Configure load balancing
upstream load_rule {
    server 127.0.0.1:3001 weight=1;
    server 127.0.0.1:3002 weight=2;
    server 127.0.0.1:3003 weight=3;
    server 127.0.0.1:3004 weight=4;
}
server {
    listen 80;
    server_name load_balance.com www.load_balance.com;
    location / {
        proxy_pass http://load_rule;
    }
}

Hosts are mapped locally so that www.load_balance.com resolves to the four services.

2. Flash‑Sale System Design Choices

The core challenge is to guarantee that tickets are neither oversold nor undersold while handling massive concurrency. Three typical workflows are compared:

2.1 Order‑First, Inventory‑Then‑Payment

Create an order, deduct inventory, then wait for payment. This avoids overselling but incurs heavy DB I/O and suffers when users abandon orders.

2.2 Payment‑First, Inventory‑After

Only deduct inventory after payment succeeds. This prevents underselling but can cause overselling under extreme load because many orders may be created before stock runs out.

2.3 Pre‑Deduction (Reservation) Strategy

Reserve inventory first, then create the order asynchronously. If a user does not pay within a timeout (e.g., five minutes), the reservation is released back to the pool. This reduces DB pressure and improves response time.

3. The Art of Inventory Deduction

To avoid costly DB transactions, the system stores a portion of inventory locally in memory and uses Redis for a global atomic counter. The workflow is:

Each server holds a local stock pool (e.g., 100 tickets).

When a request arrives, the server decrements its local counter.

If local stock is exhausted, the request falls back to Redis, which atomically increments the sold count via a Lua script.

Redis also holds a “buffer” stock to compensate for failed nodes.

4. Code Demonstration (Go)

4.1 Initialization

The init function creates the local stock struct, the Redis key definitions, a connection pool, and a channel used as a lightweight distributed lock.

package localSpike

type LocalSpike struct {
    LocalInStock   int64
    LocalSalesVolume int64
}

package remoteSpike

type RemoteSpikeKeys struct {
    SpikeOrderHashKey string // Redis hash for orders
    TotalInventoryKey  string // Field for total tickets
    QuantityOfOrderKey string // Field for sold tickets
}

func NewPool() *redis.Pool {
    return &redis.Pool{MaxIdle: 10000, MaxActive: 12000, Dial: func() (redis.Conn, error) { return redis.Dial("tcp", ":6379") }}
}

func init() {
    localSpike = localSpike2.LocalSpike{LocalInStock: 150, LocalSalesVolume: 0}
    remoteSpike = remoteSpike2.RemoteSpikeKeys{SpikeOrderHashKey: "ticket_hash_key", TotalInventoryKey: "ticket_total_nums", QuantityOfOrderKey: "ticket_sold_nums"}
    redisPool = remoteSpike2.NewPool()
    done = make(chan int, 1)
    done <- 1
}

4.2 Local and Remote Deduction

Local deduction simply increments the sales counter and checks against the local pool.

func (spike *LocalSpike) LocalDeductionStock() bool {
    spike.LocalSalesVolume++
    return spike.LocalSalesVolume < spike.LocalInStock
}

Remote deduction runs a Lua script that atomically checks total inventory and increments the sold count.

const LuaScript = `
local ticket_key = KEYS[1]
local ticket_total_key = ARGV[1]
local ticket_sold_key = ARGV[2]
local ticket_total_nums = tonumber(redis.call('HGET', ticket_key, ticket_total_key))
local ticket_sold_nums = tonumber(redis.call('HGET', ticket_key, ticket_sold_key))
if ticket_total_nums >= ticket_sold_nums then
    return redis.call('HINCRBY', ticket_key, ticket_sold_key, 1)
end
return 0
`

func (r *RemoteSpikeKeys) RemoteDeductionStock(conn redis.Conn) bool {
    lua := redis.NewScript(1, LuaScript)
    result, err := redis.Int(lua.Do(conn, r.SpikeOrderHashKey, r.TotalInventoryKey, r.QuantityOfOrderKey))
    if err != nil { return false }
    return result != 0
}

4.3 Request Handling

The HTTP handler obtains a Redis connection, attempts both local and remote deductions, and returns JSON indicating success or sold‑out. The result is logged to ./stat.log.

func handleReq(w http.ResponseWriter, r *http.Request) {
    redisConn := redisPool.Get()
    var LogMsg string
    <-done
    if localSpike.LocalDeductionStock() && remoteSpike.RemoteDeductionStock(redisConn) {
        util.RespJson(w, 1, "抢票成功", nil)
        LogMsg = "result:1,localSales:" + strconv.FormatInt(localSpike.LocalSalesVolume, 10)
    } else {
        util.RespJson(w, -1, "已售罄", nil)
        LogMsg = "result:0,localSales:" + strconv.FormatInt(localSpike.LocalSalesVolume, 10)
    }
    done <- 1
    writeLog(LogMsg, "./stat.log")
}

4.4 Single‑Node Stress Test

Using ApacheBench: ab -n 10000 -c 100 http://127.0.0.1:3005/buy/ticket The test on a low‑spec Mac shows ~4 300 requests per second with zero failures, confirming that the in‑memory and Redis‑backed design can sustain high QPS.

// Sample log excerpt
result:1,localSales:145
result:1,localSales:146
result:0,localSales:151
...

5. Conclusions

The prototype demonstrates that a flash‑sale ticket system can achieve:

No overselling or underselling through combined local and remote inventory checks.

High throughput by avoiding frequent DB writes; most operations stay in memory.

Fault tolerance via buffer stock in Redis, allowing a few nodes to fail without affecting overall availability.

Scalability by distributing traffic with weighted load‑balancing and leveraging Go’s lightweight goroutine model.

Key takeaways are the importance of proper load‑balancing, judicious use of concurrency and asynchronous processing, and the effectiveness of Redis Lua scripts for atomic operations in high‑concurrency environments.