How to Build a Million‑User Ticket‑Spike System: High‑Concurrency Architecture & Go Implementation

1. Large High‑Concurrency System Architecture

High‑concurrency systems typically use distributed clusters with multiple layers of load balancers and various disaster‑recovery mechanisms (dual data centers, node fault tolerance, backup servers) to ensure high availability. Traffic is balanced across servers based on capacity and configuration.

1.1 Load Balancing Overview

The diagram shows three layers of load balancing that user requests pass through. The three common methods are:

OSPF (Open Shortest Path First) – an interior gateway protocol that builds a link‑state database and can perform load balancing on equal‑cost paths (up to six links).

LVS (Linux Virtual Server) – a cluster technology that uses IP load balancing and content‑based request distribution, automatically masking server failures.

Nginx – a high‑performance HTTP reverse proxy. It supports round‑robin, weighted round‑robin, and IP‑hash load balancing. The example focuses on weighted round‑robin.

1.2 Nginx Weighted Round‑Robin Demo

Weighted round‑robin is configured via the

upstream

module. The following configuration assigns weights 1‑4 to four backend servers listening on ports 3001‑3004.

# Configuration for weighted load balancing
upstream load_rule {
    server 127.0.0.1:3001 weight=1;
    server 127.0.0.1:3002 weight=2;
    server 127.0.0.1:3003 weight=3;
    server 127.0.0.1:3004 weight=4;
}
server {
    listen 80;
    server_name load_balance.com www.load_balance.com;
    location / {
        proxy_pass http://load_rule;
    }
}

A Go program starts four HTTP services on ports 3001‑3004. The example below shows the service on port 3001; the others are identical except for the port number.

package main

import (
    "net/http"
    "os"
    "strings"
)

func main() {
    http.HandleFunc("/buy/ticket", handleReq)
    http.ListenAndServe(":3001", nil)
}

// handleReq writes request results to a log file
func handleReq(w http.ResponseWriter, r *http.Request) {
    failedMsg := "handle in port:"
    writeLog(failedMsg, "./stat.log")
}

func writeLog(msg string, logPath string) {
    fd, _ := os.OpenFile(logPath, os.O_RDWR|os.O_CREATE|os.O_APPEND, 0644)
    defer fd.Close()
    content := strings.Join([]string{msg, "
"}, "3001")
    fd.Write([]byte(content))
}

Using

ab

for load testing shows that ports 3001‑3004 receive 100, 200, 300, and 400 requests respectively, matching the configured weights.

2. Spike System Selection

When millions of users simultaneously attempt to purchase a limited number of tickets, the system must prevent both overselling and underselling. Three typical order‑processing flows are discussed:

Order‑then‑deduct: Create an order first, then reduce inventory. This ensures no oversell but can cause high DB I/O and potential undersell due to unpaid orders.

Pay‑then‑deduct: Reduce inventory after payment. This avoids undersell but risks oversell under extreme concurrency.

Pre‑deduct (reserve) inventory: Reserve stock first, then create orders asynchronously. This reduces DB I/O and improves response time, while unpaid orders can be reclaimed after a timeout.

Orders typically have a validity period (e.g., five minutes). Expired orders return their tickets to the pool, and asynchronous order creation can be handled via message queues such as Kafka.

3. The Art of Inventory Deduction

In low‑concurrency scenarios, inventory deduction uses a database transaction: check stock, decrement, and commit. This approach is too slow for spike systems.

Optimized single‑machine deduction moves a portion of inventory to local memory, performing deductions in‑process and only syncing with a central store when necessary. The diagram below illustrates the local‑deduction flow.

To achieve high availability, the total inventory is distributed across many machines (e.g., 100 servers each holding 100 tickets for a total of 10,000). If a few servers fail, a buffered “extra” inventory on each machine compensates, preventing undersell.

Redis is used as the central inventory store because of its high QPS capability (≈100k). After a local deduction succeeds, the system also performs a remote deduction in Redis using a Lua script to guarantee atomicity.

4. Code Demonstration

4.1 Initialization

The

init

function prepares local stock, Redis keys, a connection pool, and a channel used as a lightweight lock.

package localSpike

type LocalSpike struct {
    LocalInStock     int64
    LocalSalesVolume int64
}

package remoteSpike

type RemoteSpikeKeys struct {
    SpikeOrderHashKey string // Redis hash key for orders
    TotalInventoryKey string // Field for total tickets
    QuantityOfOrderKey string // Field for sold tickets
}

func NewPool() *redis.Pool {
    return &redis.Pool{
        MaxIdle:   10000,
        MaxActive: 12000,
        Dial: func() (redis.Conn, error) {
            return redis.Dial("tcp", ":6379")
        },
    }
}

func init() {
    localSpike = LocalSpike{LocalInStock: 150, LocalSalesVolume: 0}
    remoteSpike = RemoteSpikeKeys{SpikeOrderHashKey: "ticket_hash_key", TotalInventoryKey: "ticket_total_nums", QuantityOfOrderKey: "ticket_sold_nums"}
    redisPool = NewPool()
    done = make(chan int, 1)
    done <- 1
}

4.2 Local and Remote Deduction

func (spike *LocalSpike) LocalDeductionStock() bool {
    spike.LocalSalesVolume++
    return spike.LocalSalesVolume < spike.LocalInStock
}

const LuaScript = `
    local ticket_key = KEYS[1]
    local total_key = ARGV[1]
    local sold_key = ARGV[2]
    local total = tonumber(redis.call('HGET', ticket_key, total_key))
    local sold = tonumber(redis.call('HGET', ticket_key, sold_key))
    if total >= sold then
        return redis.call('HINCRBY', ticket_key, sold_key, 1)
    end
    return 0
`

func (r *RemoteSpikeKeys) RemoteDeductionStock(conn redis.Conn) bool {
    lua := redis.NewScript(1, LuaScript)
    result, err := redis.Int(lua.Do(conn, r.SpikeOrderHashKey, r.TotalInventoryKey, r.QuantityOfOrderKey))
    if err != nil {
        return false
    }
    return result != 0
}

4.3 Handling Requests

package main

func main() {
    http.HandleFunc("/buy/ticket", handleReq)
    http.ListenAndServe(":3005", nil)
}

func handleReq(w http.ResponseWriter, r *http.Request) {
    redisConn := redisPool.Get()
    <-done
    if localSpike.LocalDeductionStock() && remoteSpike.RemoteDeductionStock(redisConn) {
        util.RespJson(w, 1, "抢票成功", nil)
        LogMsg = "result:1,localSales:" + strconv.FormatInt(localSpike.LocalSalesVolume, 10)
    } else {
        util.RespJson(w, -1, "已售罄", nil)
        LogMsg = "result:0,localSales:" + strconv.FormatInt(localSpike.LocalSalesVolume, 10)
    }
    done <- 1
    writeLog(LogMsg, "./stat.log")
}

4.4 Single‑Machine Load Test

Using

ab -n 10000 -c 100 http://127.0.0.1:3005/buy/ticket

yields ~4,300 requests per second with no failures, demonstrating that a single machine can handle thousands of concurrent spike requests.

5. Summary Review

The spike system design combines distributed load balancing, local in‑memory inventory reservation, and a centralized Redis store with atomic Lua scripts to achieve high throughput while preventing both oversell and undersell. It also incorporates buffer inventory to tolerate server failures. Key takeaways include the effectiveness of dividing traffic across many nodes and leveraging Go’s native concurrency model to fully utilize multi‑core servers.