Designing a High-Concurrency Ticket Booking System with Load Balancing, Nginx Weighted Round Robin, and Redis

Although most users can normally purchase tickets, the moment tickets are released sees a massive surge of requests, especially during Chinese New Year, where billions of users compete for limited seats, pushing the 12306 service to handle unprecedented QPS.

The author studies the 12306 backend architecture and shares a simulated example of how to serve 1 million concurrent users buying 10 000 tickets while maintaining stability.

Large‑Scale High‑Concurrency System Architecture

High‑concurrency systems typically deploy distributed clusters with multiple layers of load balancers and disaster‑recovery mechanisms (dual data centers, node fault tolerance, backup servers) to ensure high availability, distributing traffic across servers based on capacity.

Load‑Balancing Overview

The three layers of load balancing illustrated include OSPF (an interior gateway protocol that computes shortest paths and supports up to six equal‑cost paths), LVS (Linux Virtual Server, a cluster technology using IP load balancing and content‑based request distribution), and Nginx (a high‑performance HTTP reverse proxy).

Nginx implements load balancing mainly via three methods: round‑robin, weighted round‑robin, and IP‑hash round‑robin.

Nginx Weighted Round‑Robin Demonstration

The following Nginx upstream configuration assigns different weights to four local services listening on ports 3001‑3004:

#配置负载均衡
    upstream load_rule {
       server 127.0.0.1:3001 weight=1;
       server 127.0.0.1:3002 weight=2;
       server 127.0.0.1:3003 weight=3;
       server 127.0.0.1:3004 weight=4;
    }
    ...
    server {
    listen       80;
    server_name  load_balance.com www.load_balance.com;
    location / {
       proxy_pass http://load_rule;
    }
}

Hosts file entries map www.load_balance.com to the local machine.

Go Service Implementation

Four HTTP services are started in Go, each listening on a different port. Below is the code for the service on port 3001 (other ports only need the port number changed):

package main

import (
    "net/http"
    "os"
    "strings"
)

func main() {
    http.HandleFunc("/buy/ticket", handleReq)
    http.ListenAndServe(":3001", nil)
}

//处理请求函数,根据请求将响应结果信息写入日志
func handleReq(w http.ResponseWriter, r *http.Request) {
    failedMsg :=  "handle in port:"
    writeLog(failedMsg, "./stat.log")
}

//写入日志
func writeLog(msg string, logPath string) {
    fd, _ := os.OpenFile(logPath, os.O_RDWR|os.O_CREATE|os.O_APPEND, 0644)
    defer fd.Close()
    content := strings.Join([]string{msg, "\r\n"}, "3001")
    buf := []byte(content)
    fd.Write(buf)
}

Requests are logged to ./stat.log and then stress‑tested using ApacheBench:

ab -n 1000 -c 100 http://www.load_balance.com/buy/ticket

The log shows request distribution of 100, 200, 300, and 400 for ports 3001‑3004, matching the configured weights.

Flash‑Sale System Design Choices

The core problem is ensuring that ticket orders are neither oversold nor undersold under extreme concurrency. Three typical approaches are examined:

Order‑then‑deduct inventory (creates DB pressure and risks “undersell” if orders are not paid).

Pay‑then‑deduct inventory (avoids undersell but can cause oversell and still incurs DB I/O).

Pre‑deduct inventory (reserve stock first, generate orders asynchronously, and use order expiration to prevent undersell).

Pre‑deduction is deemed most reasonable. To avoid frequent DB I/O, the author proposes a local‑stock strategy where each machine holds a portion of the total tickets in memory.

Local Stock Deduction

In low‑concurrency scenarios, stock deduction involves a transactional DB operation, which is too heavy for flash‑sale workloads.

Optimized local deduction stores a fixed amount of tickets on each server and decrements the count in memory, dramatically improving single‑machine concurrency.

Remote Unified Stock Deduction with Redis

Redis is used as a high‑performance centralized stock store (single‑machine QPS can reach 100 k). After a successful local deduction, the service atomically decrements the global stock in Redis using a Lua script:

const LuaScript = `
        local ticket_key = KEYS[1]
        local ticket_total_key = ARGV[1]
        local ticket_sold_key = ARGV[2]
        local ticket_total_nums = tonumber(redis.call('HGET', ticket_key, ticket_total_key))
        local ticket_sold_nums = tonumber(redis.call('HGET', ticket_key, ticket_sold_key))
        if(ticket_total_nums >= ticket_sold_nums) then
            return redis.call('HINCRBY', ticket_key, ticket_sold_key, 1)
        end
        return 0
`
func (RemoteSpikeKeys *RemoteSpikeKeys) RemoteDeductionStock(conn redis.Conn) bool {
    lua := redis.NewScript(1, LuaScript)
    result, err := redis.Int(lua.Do(conn, RemoteSpikeKeys.SpikeOrderHashKey, RemoteSpikeKeys.TotalInventoryKey, RemoteSpikeKeys.QuantityOfOrderKey))
    if err != nil {
        return false
    }
    return result != 0
}

Initial Redis state is set with:

hmset ticket_hash_key "ticket_total_nums" 10000 "ticket_sold_nums" 0

Request Handling and Logging

The HTTP handler performs local deduction, then remote deduction, and returns a JSON response indicating success or sold‑out. It also writes the result to ./stat.log for later analysis.

package main
... 
func handleReq(w http.ResponseWriter, r *http.Request) {
    redisConn := redisPool.Get()
    LogMsg := ""
    <-done
    if localSpike.LocalDeductionStock() && remoteSpike.RemoteDeductionStock(redisConn) {
        util.RespJson(w, 1, "抢票成功", nil)
        LogMsg = LogMsg + "result:1,localSales:" + strconv.FormatInt(localSpike.LocalSalesVolume, 10)
    } else {
        util.RespJson(w, -1, "已售罄", nil)
        LogMsg = LogMsg + "result:0,localSales:" + strconv.FormatInt(localSpike.LocalSalesVolume, 10)
    }
    done <- 1
    writeLog(LogMsg, "./stat.log")
}

Single‑Machine Performance Test

Using ApacheBench with 10 000 requests and 100 concurrent connections, the service achieved about 4 300 requests per second, with an average latency of 23 ms. Log excerpts confirm that the weighted distribution matches the Nginx configuration.

//stat.log
... 
result:1,localSales:145
result:1,localSales:146
... 
result:0,localSales:156
...

Conclusion

The flash‑sale system is complex; this article demonstrates a simplified yet high‑performance implementation, covering single‑machine optimization, cluster fault tolerance, and strategies to prevent overselling and underselling. Key lessons include effective load balancing, leveraging Go’s concurrency model, and using Redis for fast, atomic stock management.