Implementing High‑Concurrency Circuit Breaking in Nginx with Lua

Nginx is a core component in large‑scale architectures, often serving as a high‑performance web server and reverse proxy. In high‑concurrency environments it is used for traffic control to ensure system stability.

Why Circuit Breaking?

When a backend service experiences a sustained increase in error rate or response latency, circuit breaking prevents cascading failures by temporarily rejecting requests or returning degraded responses. This protects downstream services and smooths traffic spikes.

Implementation Overview

Nginx does not provide a built‑in circuit‑breaker, but the functionality can be achieved with Lua scripts (via OpenResty) and shared memory dictionaries. The approach tracks request counts, error counts, and calculates error rates within a sliding window. If the error rate exceeds a threshold, the circuit is opened for a configurable period (e.g., 30 seconds).

Lua Configuration

lua_shared_dict cb_metrics 10m;

server {
    listen 80;
    server_name api.example.com;

    location /api/report/ {
        access_by_lua_block {
            local dict = ngx.shared.cb_metrics
            local state = dict:get("report_state") or "closed"
            local open_until = dict:get("report_open_until") or 0
            local now = ngx.now()
            -- If circuit is open and not yet recovered, return 503
            if state == "open" and now < open_until then
                ngx.status = 503
                ngx.say('{"code":503,"msg":"report service temporarily unavailable"}')
                return ngx.exit(ngx.HTTP_OK)
            end
            proxy_pass http://upstream_report;
        }

        log_by_lua_block {
            local dict = ngx.shared.cb_metrics
            local key_err = "report_err"
            local key_total = "report_total"
            -- Increment total request count
            dict:incr(key_total, 1, 0)
            local status = ngx.status
            local cost = ngx.now() - ngx.req.start_time()
            -- Count errors or slow responses (>2 s)
            if status >= 500 or cost > 2 then
                dict:incr(key_err, 1, 0)
            end
            local total = dict:get(key_total)
            local err = dict:get(key_err)
            if total and total >= 50 then
                local err_rate = err / total
                -- Open circuit for 30 s if error rate > 50 %
                if err_rate > 0.5 then
                    dict:set("report_state", "open")
                    dict:set("report_open_until", ngx.now() + 30)
                else
                    dict:set("report_state", "closed")
                end
                -- Reset counters for the next window
                dict:set(key_total, 0)
                dict:set(key_err, 0)
            end
        }
    }
}

How It Works

The lua_shared_dict cb_metrics 10m creates a shared memory zone for storing metrics.

In the access_by_lua_block, the script checks the current circuit state. If the state is open and the recovery time has not passed, it immediately returns a 503 response with a custom JSON payload.

The log_by_lua_block runs after the request is processed. It increments total request counters and, when a request fails (status ≥ 500) or exceeds a 2‑second latency, increments the error counter.

When at least 50 requests have been recorded, the script calculates the error rate. If the error rate exceeds 50 %, the circuit is opened for 30 seconds; otherwise it remains closed.

After each evaluation, the counters are reset to start a new monitoring window.

This solution provides a lightweight, in‑process circuit‑breaker without requiring external services, making it suitable for high‑traffic Nginx deployments.