How to Build a High‑Concurrency Queue System with OpenResty, Lua, and Redis

Background

Based on historical performance test data, the current system shows two main bottlenecks: the authentication service can handle a maximum of 3000 concurrent users, while downstream business systems support only 1500 concurrent users, yet real‑world traffic often spikes beyond 4000 QPS, causing overload and service avalanche.

Authentication service: max concurrent capacity 3000 users

Downstream services: max concurrent capacity 1500 users, actual spikes >4000 QPS

High Concurrency Traffic Control Solution

Two common traffic‑control strategies are considered:

(1) Hard rate‑limiting using frameworks like Sentinel or Redisson to reject excess requests when QPS exceeds a threshold.

(2) Flexible queuing that asynchronously queues excess traffic, shows queue position, length, and estimated wait time to users.

After technical evaluation, the chosen solution combines OpenResty+Lua to implement a non‑intrusive traffic‑control layer with the following advantages:

Zero modification to existing system architecture

High performance leveraging Nginx

Scalable with dynamic queue‑strategy adjustments

Visualization of real‑time queue status

Solution Architecture Design

The architecture adopts a four‑layer design:

Gateway Layer – OpenResty cluster as entry point for request preprocessing and load balancing.

Control Layer – Lua scripts implement core logic: queue_control.lua: queue admission control. queue_status.lua: maintains queue status, calculates wait time, and provides real‑time feedback. queue_release.lua: resource release (implemented by downstream services). queue_init.lua: management interface for initialization and configuration.

Storage Layer – Redis cluster stores four key data structures: queue:zset (ordered set) – user session info for FIFO queue. queue:active:count (string) – atomic counter of active users. queue:config:max_active (string) – configurable max concurrent users. queue:authorized:set (set) – authorized user identifiers.

Presentation Layer – UI displays system load and queue progress, automatically redirects users when resources become available.

Core Implementation Principles

The system uses layered traffic control with dynamic capacity checks to ensure stability under high load.

System Capacity Settings (queue_init.lua)

The default maximum concurrent users is 1000, stored in Redis key queue:config:max_active. Administrators can adjust this value at runtime.

curl "http://localhost:8089/api/queue/init?value=1000"

User Access Control (queue_control.lua)

Requests pass through access_by_lua_file queue_control.lua. The script checks Redis for current capacity and decides:

If active_count < max_active, the user is authorized, active count is incremented, the user is added to the authorized set, removed from the queue, and a cookie is set.

If the system is full, the user is added to queue:zset with a timestamp score and redirected to the queue page.

if active_count < max_active then
    -- authorize user
    red:incr(active_count_key)
    red:sadd(authorized_set, user_id)
    red:zrem(queue_zset, user_id)
    ngx.header["Set-Cookie"] = "health_authorized=true; path=/health; max-age=1800"
    return true
end
-- system full, add to queue
local score = ngx.now()
red:zadd(queue_zset, score, user_id)

In high‑concurrency scenarios, using if active_count < max_active can cause race conditions; a distributed lock would guarantee strict consistency but reduces throughput, so the design accepts occasional over‑allocation for higher performance.

Queue Page Handling (queue_status.lua)

The queue page calls /api/queue/status every 10 seconds to update rank and heartbeat. The system removes users with stale heartbeats via a timed task.

Rate Limiting for Queue Page

A fixed‑window counter algorithm limits page refreshes, balancing simplicity, performance, and Redis compatibility.

local DEFAULT_LIMITS = {window_size = 60, max_requests = 30}
function _M.check_limit(path)
    local visitor_id = user_identity.get_user_id()
    if not visitor_id then return false, "Unable to identify user" end
    local current_window = math.floor(ngx.time() / DEFAULT_LIMITS.window_size)
    local redis_key = string.format("rate_limit:%s:%d", visitor_id, current_window)
    local result, err = redis_pool.execute(function(red)
        local current = red:get(redis_key)
        if current == ngx.null then current = 0 end
        if tonumber(current) >= DEFAULT_LIMITS.max_requests then
            return {allowed = false, current = current, limit = DEFAULT_LIMITS.max_requests, reset_time = (current_window+1)*DEFAULT_LIMITS.window_size}
        end
        local new = red:incr(redis_key)
        if new == 1 then red:expire(redis_key, DEFAULT_LIMITS.window_size + 10) end
        return {allowed = true, current = new, limit = DEFAULT_LIMITS.max_requests, reset_time = (current_window+1)*DEFAULT_LIMITS.window_size}
    end)
    if err then return true, "Redis error, allow access" end
    return result.allowed, result
end

Resource Release (queue_release.lua)

When a user finishes business logic, the active user count is decremented and resources are freed for queued users (implementation resides in downstream services).

Performance Testing

Single‑machine tests reached 833.3 QPS with 0.348 s average latency. A 10‑node cluster achieved 4062 QPS at 2000 concurrent load, maintaining ~0.355 s latency. The system scales well up to ~3000‑4000 QPS, satisfying medium‑scale queue‑system requirements.