How to Build a Scalable Non‑Intrusive Traffic‑Control System with OpenResty, Lua, and Redis

Background

Based on historical performance test data, the current system shows two main bottlenecks: the authentication service can handle a maximum of 3000 concurrent users, while downstream services support only 1500, yet real‑world traffic often exceeds 4000 QPS, causing overload and service avalanche.

High‑Concurrency Traffic‑Control Strategies

Two common approaches are used:

Hard rate limiting with frameworks such as Sentinel or Redisson, which reject excess requests when QPS exceeds a preset threshold.

Soft queueing, which asynchronously queues excess users, displays queue position, length, and estimated wait time, and only releases users when capacity is available.

After technical evaluation, the solution adopts the OpenResty+Lua combination because it requires zero changes to existing architecture, leverages Nginx’s high performance, is extensible, and provides real‑time visual feedback.

Solution Architecture Design

The design follows a four‑layer architecture:

Gateway Layer – an OpenResty cluster that performs request preprocessing and load balancing.

Control Layer – Lua scripts implementing core logic, including queue_control.lua, queue_status.lua, queue_release.lua, and queue_init.lua.

Storage Layer – Redis cluster storing four key data structures: queue:zset (ordered set for user sessions), queue:active:count (atomic counter of active users), queue:config:max_active (dynamic max concurrent users), and queue:authorized:set (set of authorized users).

Presentation Layer – a web page that shows real‑time load status and queue progress, redirecting users when resources become available.

Core Implementation Principles

The system uses dynamic capacity control. The maximum concurrent users default to 1000 and are stored in the Redis key queue:config:max_active, which can be adjusted at runtime.

curl "http://localhost:8089/api/queue/init?value=1000"

When a request arrives, access_by_lua_file loads queue_control.lua. The script checks if the user is already authorized via cookies or Redis. If authorized, it refreshes the cookie and allows access. If not, it evaluates current load:

If active users < max capacity, the script atomically increments the active counter, adds the user to the authorized set, removes any queue entry, sets an authorization cookie, and permits access.

If the system is full, the user is added to queue:zset with a timestamp score and redirected to a queue page.

if active_count < max_active then
    -- grant access
    red:incr(active_count_key)
    red:sadd(authorized_set, user_id)
    red:zrem(queue_zset, user_id)
    ngx.header["Set-Cookie"] = "health_authorized=true; path=/health; max-age=1800"
    return true
end
-- system full, enqueue user
local score = ngx.now()
red:zadd(queue_zset, score, user_id)

Using if active_count < max_active can cause race conditions; a distributed lock would guarantee strict consistency but would reduce throughput. The design intentionally sacrifices strict consistency for higher performance, which is acceptable for the queue system.

Queue page logic ( queue_status.lua) calculates available slots, determines the user’s rank in queue:zset, and decides if the user can enter based on two conditions: slots available and the user’s position within those slots.

local available_slots = max_active - active_count
local rank = red:zrank(queue_zset, user_id)
local user_position = rank and rank + 1 or queue_length
local is_system_available = (available_slots > 0)
local is_user_next_in_line = (user_position <= available_slots)
if is_system_available and is_user_next_in_line then
    can_enter = true
end

When a user is allowed to enter, the system atomically increments the active counter, marks the user as authorized, removes the user from the queue, sets a secure cookie, and redirects to the downstream service.

Rate Limiting for Queue Page

The queue page employs a fixed‑window rate limiter implemented in Lua, with a 60‑second window and a maximum of 30 requests per window per user. The limiter stores counters in Redis and uses pipelines for batch operations.

local DEFAULT_LIMITS = {window_size = 60, max_requests = 30}
function _M.check_limit(path)
    local visitor_id = user_identity.get_user_id()
    if not visitor_id then return false, "Unable to identify user" end
    local current_window = math.floor(ngx.time() / DEFAULT_LIMITS.window_size)
    local redis_key = string.format("rate_limit:%s:%d", visitor_id, current_window)
    -- pipeline to get and increment counter
    -- return allowed flag and metadata
end

Heartbeat Maintenance and Cleanup

Clients poll /api/queue/status every 10 seconds, updating a heartbeat ZSet entry. A server‑side task runs every 2 minutes, scanning for entries older than 120 seconds and removing stale users from the queue. The cleanup uses Redis pipelines and batch processing (1000 users per batch) to minimize network latency.

local batch_size = 1000
for batch = 1, batch_count do
    -- collect user IDs for this batch
    red:init_pipeline()
    for _, user_id in ipairs(batch_users) do
        red:zrem(queue_zset, user_id)
    end
    local results, err = red:commit_pipeline()
    -- handle results
end

Performance Testing

Single‑node tests achieved a peak throughput of 833.3 QPS with an average latency of 0.348 s. A 10‑node cluster scaled throughput to 4062 QPS at 2000 concurrent load, maintaining latency around 0.35 s. The system remains stable up to 3000 concurrent requests, with peak throughput exceeding 4000 QPS and response times between 0.3 s and 0.6 s, satisfying medium‑scale business requirements.