OpenResty Dynamic Gateway: Phases, Cosocket, Shared Dict & Production Patterns
This article explains how to build a dynamic API gateway with OpenResty by moving service discovery, canary routing, and TLS certificates from static config files into runtime memory, covering Nginx phases, non-blocking cosocket, atomic shared dict operations, and production hardening lessons from real incidents.
Why Static Nginx Config Fails at Scale
The first eight articles covered single-node capabilities: routing, proxy, cache, rate limiting, encryption, debugging, and HA. They share a flaw: adding a backend, cutting a canary, or rotating a certificate all require editing nginx.conf and reloading. When backends are hundreds of ephemeral pods, certificates cover thousands of domains, and canary ratios change per user in real time, reloads turn from routine ops into incident sources.
This article moves service discovery, canary splitting, and dynamic certificates from config files into runtime memory. Three core concepts govern everything:
Phase determines what you can do
Cosocket determines whether you block
Shared dict determines whether you share across workers
Three Incidents, Three Lessons
Incident 1: Scale-out OOM from Reload Storm
Service up/down used "render config + reload". 300 machines, 2,000 daily changes. A promo added 200 pods in five minutes — 200 reloads. Reload mechanism: new workers start, old workers gracefully exit waiting for keep-alive connections (75s). Result: 60+ old workers piled up, each holding a connection pool and shared memory mapping. Memory tripled, OOM killer struck, site-wide 502 for five minutes. Reload cost scales with connection count.
Incident 2: One curl Dropped 8,000 QPS to 200
An access_by_lua block used os.execute to call curl for auth. Normal RT 5ms; auth service slowed to 30s. The worker's event loop blocked entirely — not just that request, but all thousands of connections on that worker queued. QPS collapsed, health checks failed, node evicted. Nginx concurrency relies on zero blocking; break it and performance falls off a cliff.
Incident 3: Rate Limit Lost 70% of Counts
Rate limiting written as get → increment → set. Single-node test passed; production failed. Multiple workers read 5, each wrote back 6. Five concurrent requests only incremented once. Should block 10k, actually blocked 3k. get and set are independent; only incr , add , replace are truly atomic.
OpenResty's Four-Layer Architecture
1.1 What It Is
OpenResty = Nginx core (event loop + protocol parsing) + LuaJIT (business logic) + C modules (expose Nginx internals to Lua). LuaJIT compiles hot code to machine code — performance near C, an order of magnitude faster than interpreted Lua. The game-changer is cosocket : Lua calls look synchronous, but when I/O isn't ready the coroutine yields control back to the event loop and resumes when ready. Without cosocket, Lua only does pure computation (header rewrite, signatures); with it, you can query Redis, call auth services, and still not block any connection.
1.2 Eleven Phases — Six Common Hooks
Nginx processes a request through eleven phases. Doing the right thing in the wrong phase still fails. Key hooks: init_by_lua — runs once in master during config load (pre-fork). Use for require modules, warm dictionaries. No event loop, no cosocket, no sleep. init_worker_by_lua — runs in each worker after start. Only place to start ngx.timer . Periodic sync belongs here. access_by_lua — auth, rate limiting, canary tagging. Main battlefield. content_by_lua — generate response directly. balancer_by_lua — pick upstream. Can see retry count. Only entry for dynamic upstreams. log_by_lua — post-response metrics. Request ended, cosocket forbidden.
1.3 Cost of Blocking
Worker = single-threaded event loop handling thousands of connections. One blocking call stalls all connections on that worker . Blacklist: os.execute / io.popen (fork + wait) io.open / io.read (sync disk I/O)
LuaSocket (non-cosocket)
Sync DNS resolution
Rule: if the call makes the OS park the thread, it cannot sit on the request path. Two hidden costs: DNS — standard Lua uses sync getaddrinfo; must configure resolver for Nginx async resolution. GC — per-request string concatenation and large tables spike P99. Mitigation: module-level cache, reuse tables, table.concat in loops.
1.4 Shared Dict: The Only Shared Ground
Lua globals are per-worker — four workers = four isolated copies. True cross-worker sharing only via lua_shared_dict: master pre-allocates a memory slab. Three must-know traits:
Fixed capacity — allocated at startup, cannot grow. Full = silent LRU eviction (no error, most dangerous).
Only some ops atomic — get/set/add/replace/delete/incr/ttl; only incr and add are atomic. get + set combo is never atomic.
get_keys is O(n) blocking — banned in production. To enumerate, maintain your own index key.
Capacity estimate:
memory ≈ entries × (key + value + ~60 bytes metadata) × 1.5. 10k node entries fit in 16MB. But "one key per user" rate limiting at millions of users = hundreds of MB — that's Redis territory.
1.5 Three "No-Reload" Entry Points
balancer_by_lua— inside upstream block, set_current_peer(ip, port) picks backend. Node list from shared dict. "Add backend = reload" becomes "write one memory row". ssl_certificate_by_lua — TLS handshake phase. Load cert by SNI. Thousands of domains without thousands of server blocks.
Access phase variable mutation — set custom variable referenced by proxy_pass, or ngx.req.set_header for canary routing.
Dynamic covers data, not structure. Adding location, server, changing timeouts, log format — still need reload. Split config into structure config (rare, stable, CI/CD) vs runtime data (high-frequency, volatile, in memory) . Guard this boundary; reloads drop from 2,000/day to 1/week.
Core Flow: Service Discovery to Dynamic Certs
2.1 Production Config: Dynamic Upstream + Service Discovery
# /usr/local/openresty/nginx/conf/nginx.conf
worker_processes auto;
env CONSUL_ADDR; // passed to Lua
events {
worker_connections 20480;
use epoll;
multi_accept on;
}
http {
lua_shared_dict svc_pool 16m; // node table
lua_shared_dict svc_stat 8m; // health counters
lua_package_path "/data/lua/?.lua;;";
init_worker_by_lua_block {
require("svc_sync").start()
}
upstream order_svc {
server 0.0.0.1; // placeholder, required for syntax
balancer_by_lua_block {
require("svc_pick").pick()
}
keepalive 128;
}
server {
listen 80 reuseport;
location /api/order/ {
proxy_pass http://order_svc;
proxy_http_version 1.1;
proxy_set_header Connection "";
proxy_next_upstream error timeout;
}
}
}Key details: server 0.0.0.1 placeholder — upstream syntax requires at least one server; never used because balancer calls set_current_peer first. env directive — Lua's os.getenv only sees variables explicitly exported via env. Top reason for "env var exists but Lua can't read it". keepalive 128 requires HTTP/1.1 + empty Connection header; all three needed or pool is useless. proxy_next_upstream error timeout only — exclude 502 because upstream 502 may mean business logic already executed (e.g., inventory deducted); retry would amplify side effects.
2.2 Control Plane Sync: Periodic Pull to Memory
-- /data/lua/svc_sync.lua
local _M = {}
local http = require "resty.http"
local cjson = require "cjson.safe"
local dict = ngx.shared.svc_pool
local stat = ngx.shared.svc_stat
local ADDR = os.getenv("CONSUL_ADDR")
local PATH = "/v1/health/service/order"
local QS = "?passing=1"
local function pull()
local hc = http:new()
hc:set_timeout(800)
local res = hc:request_uri(ADDR .. PATH .. QS)
if not res then return nil end
if res.status ~= 200 then return nil end
local body = cjson.decode(res.body)
if not body then return nil end
local list = {}
for _, it in ipairs(body) do
local s = it.Service
list[#list + 1] = s.Address .. ":" .. s.Port
end
hc:close()
return list
end
function _M.start()
local function tick(premature)
if premature then return end
local list = pull()
if list and #list > 0 then
local v = table.concat(list, ",")
dict:set("order_svc", v, 30)
stat:incr("sync_ok", 1, 0)
else
stat:incr("sync_err", 1, 0)
-- keep old value, do not overwrite
end
end
ngx.timer.every(5, tick)
end
return _MThe critical line: on pull failure, do not overwrite old value . First reliability rule of dynamic gateway: control plane may fail, data plane must not die . Registry timeout, DNS jitter, network partition all return nil. If you clear the dict, gateway instantly becomes "zero available backends" — one control-plane blip becomes full-site outage. Correct strategy: only replace when new data is valid; on failure, keep stale value and increment error counter. TTL of 30s (not infinite) ensures prolonged pull failure eventually surfaces (expired → balancer reports "no upstream") rather than running on stale list indefinitely.
2.3 Node Selection, Health, Retry
-- /data/lua/svc_pick.lua
local _M = {}
local b = require "ngx.balancer"
local dict = ngx.shared.svc_pool
local stat = ngx.shared.svc_stat
local function alive(raw)
local list = {}
for host in string.gmatch(raw, "[^,]+") do
if not stat:get("down:" .. host) then
list[#list + 1] = host
end
end
return list
end
function _M.pick()
local raw = dict:get("order_svc")
if not raw then
return ngx.log(ngx.ERR, "no upstream")
end
local list = alive(raw)
if #list == 0 then
return ngx.log(ngx.ERR, "all down")
end
local idx
if not b.get_last_failure() then
local uid = ngx.var.arg_uid or ngx.var.remote_addr
idx = ngx.crc32_short(uid) % #list + 1
else
local bad = ngx.ctx.bad_peer
if bad then
stat:incr("down:" .. bad, 1, 0, 30)
end
idx = math.random(#list)
end
local host, port = string.match(list[idx], "(.+):(%d+)")
ngx.ctx.bad_peer = list[idx]
local p = tonumber(port)
return b.set_current_peer(host, p)
end
return _MThis logic solves three production problems:
Health filtering — down:IP keys act as passive circuit breaker. Node failing during retry increments counter with 30s TTL; excluded until expiry. Cheapest passive health check: the error itself is the signal.
Consistent hashing — first attempt uses uid CRC32 modulo, pinning same user to same node (cache-friendly). Retry switches to random to avoid hammering the same bad node.
set_current_peer return value — can fail (invalid params). Must return error; otherwise Nginx falls back to placeholder 0.0.0.1 → user sees connection timeout.
Remember get_last_failure semantics: returns previous attempt's failure state (e.g., "next", "failed"), nil on first entry — that's the discriminator for "first try hash, retry random". ngx.ctx.bad_peer is request-scoped storage recording last chosen node so retry knows whom to mark down. These two APIs close the "pick → fail → mark → repick" loop. On empty node list, don't ngx.exit(502) yourself; log and return, let Nginx default error handling produce cleaner logs.
2.4 Production Config: Dynamic Certs + Canary
http {
lua_shared_dict cert_store 32m; // certs
lua_shared_dict route_cfg 4m; // canary ratios
resolver 10.0.0.2 valid=10s ipv6=off;
server {
listen 443 ssl;
server_name _; // wildcard, SNI distinguishes
ssl_protocols TLSv1.2 TLSv1.3;
ssl_certificate_by_lua_block {
require("cert_loader").load()
}
set $target "order_v1"; // fallback
location /api/order/ {
access_by_lua_block {
require("gray").route()
}
proxy_pass http://$target;
}
}
upstream order_v1 {
server 10.0.1.11:8080;
server 10.0.1.12:8080;
keepalive 64;
}
upstream order_v2 {
server 10.0.2.11:8080;
keepalive 16;
}
}Solves two "thousands":
Thousands of domains via server_name _ + SNI dynamic certs — no per-domain server blocks. Certs in shared dict, fetched by SNI at handshake.
Thousands of canary adjustments via access-phase variable mutation — ratio in dict, ops dashboard changes one number, entire cluster picks it up next request. No reload, no push, no intermediate state.
Critical design: upstream name must exactly match variable value because variable proxy_pass first tries to match an upstream name; if matched, no resolver needed. If unmatched, falls back to DNS — without resolver directive, startup fails. This is the #1 pitfall of variable proxy_pass.
Three non-negotiables: set $target fallback — if Lua errors before assignment, empty variable makes proxy_pass parse fail → 500. Default ensures worst case hits v1. order_v2 keepalive only 16 — canary pool small; larger wastes connection slots. ssl_certificate_by_lua runs before HTTP request — no request body, no redirect. Only SNI and client info available. "Read cookie to pick cert" impossible here.
2.5 Dynamic Cert & Canary Implementation
-- /data/lua/cert_loader.lua
local ssl = require "ngx.ssl"
local dict = ngx.shared.cert_store
local lrucache = require "resty.lrucache"
local lru = lrucache.new(2000)
local _M = {}
function _M.load()
local sni = ssl.server_name()
if not sni then
return ngx.log(ngx.ERR, "no sni")
end
local pair = lru:get(sni)
if not pair then
local c = dict:get("cert:" .. sni)
local k = dict:get("key:" .. sni)
if not (c and k) then
return ngx.log(ngx.ERR, "no cert")
end
local dc, e1 = ssl.parse_pem_cert(c)
local dk, e2 = ssl.parse_pem_priv_key(k)
if not (dc and dk) then
return ngx.log(ngx.ERR, e1 or e2)
end
pair = { dc, dk }
lru:set(sni, pair, 300)
end
ssl.clear_certs()
local c2, k2 = pair[1], pair[2]
local ok, err = ssl.set_cert(c2, k2)
if not ok then ngx.log(ngx.ERR, err) end
end
return _MBest pattern: two-level cache . Shared dict stores PEM raw (cross-worker, pushable by control plane). Per-worker lrucache stores parsed cdata objects. Why two levels? parse_pem_cert builds real X509 struct — tens of microseconds. At thousands of TLS QPS, parsing every handshake is measurable overhead. Parsed cdata binds to SSL_CTX, cannot cross workers — must stay in-process. Result: "dict stores raw + each worker parses once + LRU caches 300s" — that's why certs are parsed per worker.
Three details you cannot skip:
ssl.clear_certs()</strong> must be called — clears static certs from config; otherwise dynamic cert gets "covered" by default cert. Symptom: "configured but not working" with no error.</li><li>Parse failures must distinguish cause — expiry, incomplete chain, key mismatch. Log with SNI for debugging.</li><li>LRU 300s is not invalidation mechanism — just ensures new cert perceived within 5 minutes. For second-level rollout, bump version number, compare version in request before re-parse.</li></ol><pre><code>-- /data/lua/gray.lua
local _M = {}
local dict = ngx.shared.route_cfg
function _M.route()
local v = dict:get("gray:order")
local pct = tonumber(v) or 0
if pct <= 0 then return end
local uid = ngx.var.cookie_uid or ngx.var.remote_addr
local h = ngx.crc32_short(uid) % 100
if h < pct then
ngx.var.target = "order_v2"
end
end
return _MFifteen lines hide three correctness points:
Hash, not random — uid CRC32 ensures same user always same side. Random would flip users between versions → features flicker.
Monotonic ratio — 10% → 30%: original 10% stays in canary, only new 20% added. No one gets kicked back to old version. Random cannot guarantee this.
IP fallback for anonymous — no cookie_uid → remote_addr. Still stable; NAT users share outcome, acceptable error.
Canary's worst enemy isn't imprecise split — it's users bouncing back and forth.
2.6 One Request End-to-End Timeline
T1 TCP connect + TLS handshake → triggers ssl_certificate_by_lua → fetch cert by SNI → clear_certs → set_cert.
T2 Handshake done → enter access → hash decides $target.
T3 content phase parses $target → matches upstream name.
T4 balancer_by_lua reads node table, filters circuit-broken nodes, hashes by uid, calls set_current_peer.
T5 Uses keepalive pool → connects upstream → forwards.
T6 Upstream fails → proxy_next_upstream triggers retry → re-enters balancer → get_last_failure non-nil → "mark down + random repick".
T7 Success → log phase writes upstream_addr, status, latency into shared dict aggregation buckets.
Zero reloads. Every decision from memory.
2.7 Seven Pre-Launch Validations
Syntax & load — openresty -t passes; Lua modules require successfully in init phase (wrong path = top startup failure).
Empty dict — deliberately clear at start; verify 502 not hang.
Dynamic effect — add node to dict; see traffic shift within 5s.
Drain drill — kill a node process; confirm marked down within 30s, traffic stops.
Control plane disconnect — make registry unreachable; business must continue normally (most critical).
Cert rotation — push new cert to dict; new connections use new cert, old connections unaffected.
Canary to zero — ratio back to 0; all traffic returns to v1, no stray sessions.
Core Source: Four Critical Logic Pieces
3.1 Balancer's Two Hidden Constraints
Called multiple times — each retry re-enters. Logic must be idempotent and retry-aware. Discriminator: get_last_failure() non-nil = retry.
Zero network I/O allowed — this phase sits at "pick backend". Querying Redis here creates "need backend to pick backend" loop. OpenResty disables cosocket and errors "API disabled". Node data must be ready before balancer — "periodic sync + shared dict" isn't preference, it's framework constraint. Also limit retries explicitly: set_more_tries caps remaining attempts. Unset = infinite retry; one slow node drags request to timeout ceiling. Rule of thumb: two retries. Two failures = systemic issue; more retries only amplify outage.
3.2 Shared Dict Atomicity: Three Correct Patterns
-- Anti-pattern: read-modify-write, loses under concurrency
local n = dict:get(key) or 0
dict:set(key, n + 1) -- all read 5
-- Pattern 1: incr atomic increment
-- args: key, delta, init, ttl
local v = dict:incr(key, 1, 0, 60)
-- Pattern 2: add for "only if absent"
local ok = dict:add("lk:" .. uid, 1, 3)
if not ok then return ngx.exit(429) end
-- Pattern 3: resty.lock for critical sections
local lock = require "resty.lock"
local l = lock:new("my_lock")
local e = l:lock("k", 0.5)
-- ... critical section ...
l:unlock()Three patterns, distinct use cases:
incr for counters (rate limit, QPS, failure counts). Entire command in shared memory, naturally atomic. Third arg init = "if missing, set 0 then add". Fourth = TTL. Many miss these two args, write "add then incr" — extra round-trip, still racy.
add for acquisition (distributed lock, idempotency key, dedup). Returns false if exists — no "get then set" needed.
resty.lock for critical sections — suits "read multiple keys, compute, write back". Cost: blocks (internal ngx.sleep yields, max wait = timeout). Critical section must be tiny; any network I/O inside serializes the whole chain.
3.3 Cosocket & Connection Pool: Top Leak Source
local http = require "resty.http"
local hc = http:new()
hc:set_timeout(300) -- connect+read+write
local res, err = hc:request_uri("http://auth/check?id=" .. uid)
if not res then
hc:close() -- failure MUST close
return ngx.exit(503)
end
hc:set_keepalive(10000, 100)
-- forgetting this line = connection leakForgetting set_keepalive is the #1 leak. Each request creates connection, doesn't return it. Connections pile on worker until peer times out. Symptom: "hours later connection count creeps up, eventually worker_connections exhausted ", restart fixes, recurs days later. Rule: "fail → close, success → set_keepalive" must pair. Second leak: pool size unset. set_keepalive second arg defaults to 30; too small = frequent new connections; too large = exhaust idle slots. Heuristic: 1.2× single-worker concurrency. Timeouts: 300ms hard ceiling for hard dependencies — exceed → degrade (allow or reject per business tolerance), not hang user for 3s. Fine-grained control via set_timeouts for connect/send/read separately.
3.4 ngx.ctx Lifecycle
ngx.ctx = request-scoped table for inter-phase data (e.g., access computes uid , balancer reads chosen node). Three behaviors:
Lifetime = one request — destroyed at end. Cannot be cross-request cache.
Internal redirect resets — error_page, X-Accel-Redirect, rewrite to new location → ctx replaced with fresh table; prior data gone.
Subrequest has own ctx — location.capture cannot see parent's ctx.
Clear layering: cross-phase persistent data → ngx.var (string); request-scoped temp → ctx ; cross-request → shared dict. Mix them → "value mysteriously nil" bugs.
Advanced Topics: Seven Accounts to Pre-Calculate
4.1 Performance Budget: Lua Overhead
Empty access_by_lua adds tens of microseconds — negligible vs Nginx's hundreds. One shared dict read + hash = still sub-100µs. Real costs:
Per-request require — hoist modules to file top-level upvalue cache.
Uncached regex — ngx.re.match must use "jo" option for compile cache.
Loop string concat — switch to table.concat.
Per-request config read — read once + version compare.
Rule of thumb: pure Lua logic under 200µs. Over? Suspect design, not code.
4.2 Memory Budget: Shared Dict Sizing
Shared memory allocated once at master start, not counted in per-worker RSS — common blind spot: "dict full but monitoring sees nothing". Sizing: count entries → formula → ×1.5 fragmentation headroom. Two runaway risks:
Keys without TTL — unbounded growth until full → silent LRU evicts useful data.
Oversized values — certs few KB × hundreds = fine; × tens of thousands = hundreds MB. Slab allocator fragments large objects; actual usage often 2× theoretical.
Only metric: periodic capacity() + free_space() trend.
4.3 Observability: Three Custom Metrics
stub_status only gives connection counts. Dynamic gateway needs:
Per-node request count & error rate — balancer writes chosen node to ngx.ctx; log phase aggregates by node. Only way to answer "which node drags us down".
Control-plane sync success rate & latency — sync_ok /
sync_err</sub> growth signals registry issues.</li><li><strong>Dict utilization</strong> — <code>free_space< 30% → alert. Full = silent LRU eviction randomly kills canary ratios, node tables.
Hard constraint: log phase has no cosocket — cannot POST directly. Write to shared dict aggregation buckets; timer batch-pushes to Prometheus Pushgateway or log service.
4.4 Degradation: Control Plane Down, Data Plane Runs
Key difference from "config center pushes config": config center down → old config still in process memory → business runs. If every request hits registry, registry down = site down. Iron law: "Request path must not call control plane." Four-layer degradation:
Dict has data → normal.
Dict expired → serve stale + alert.
Dict empty → fall back to hardcoded allowlist nodes in config (even just two IPs).
Upstream error rate exceeds threshold → full eject, keep only allowlist.
Better degraded to single node than full 502.
4.5 Flash Sales: Push Rejection to Edge
Flash sale traits: instant 100× traffic, vast majority fail, backend must stay untouched. OpenResty value: reject at ingress.
Local token bucket — timer refills N tokens/sec into dict; request incr takes; fail → 429. Blocks 90% traffic.
Local inventory — stock in dict; incr negative delta atomically decrements; hit 0 → sold out. No Redis call needed.
Consistent hash to fixed nodes — reduces cache stampede.
Note: incr is single-machine atomic. Across gateways: either split quota "total / node count" (simple, imprecise) or use Redis global decrement (accurate, adds dependency).
4.6 Circuit Breaker & Retry Budget: Don't Amplify Failure
Retry double-edged: saves transient failures; under downstream overload, multiplies request volume 3×. Budget three dimensions:
Count (max 2)
Time ( proxy_next_upstream_timeout 3s)
Ratio (error rate > 50% → stop retry, fast-fail)
Circuit breaker cheapest in balancer: per-node failure accumulation → exceed threshold → write down:IP with 30s TTL. Simplest breaker. Half-open via "30s later auto-retry once" naturally. Must watch idempotency : GET safe; POST may have already executed (re-order). Strategy by "path + method": write APIs only retry connection errors ( error ), never timeout ( timeout may have processed).
4.7 DIY Cost: Three Underestimated Areas
Working dynamic gateway in a week; production-ready in six months. Gaps:
Release & rollback — Lua change requires reload (OpenResty no hot module reload). "Dynamic" only covers data, not code. Rollback path must be smooth.
Config consistency — canary ratios, node tables scattered across each machine's dict. Who pushes? Failure handling? Convergence on inconsistency? Needs a control plane.
Observability — Lua errors only in error.log stack trace. No APM, no distributed tracing. Debug = tail + reproduce.
Pragmatic advice: APISIX for standard capabilities (auth, rate limit, canary, observability, multi-tenant); only write custom Lua modules for truly unique logic (custom signature, protocol translation, special rewrite).
Learning Path: From Writing Lua to Daring Traffic Changes
Master phases — write a location that prints current phase; watch rewrite/access/content/log order. In access do ngx.sleep(1); observe yield (concurrency unaffected) vs block.
Rate limit with dict — deliberately write get+set version; ab load test, watch count loss. Rewrite with incr; compare. This contrast burns atomicity into memory.
Make upstream dynamic — copy 2.1–2.3. Then do three things: add node, stop node, kill registry. Observe three behaviors.
Read three source pieces — set_current_peer, shared dict incr, resty.http connection pool recycle. Understand "looks like Lua, actually C".
Run one chaos drill — make a module infinite loop or throw; watch error.log, 502 shape, whether reload recovers.
High-Frequency Interview Questions (with Answers)
Q1: OpenResty vs Nginx native Lua module? Nginx native = C, compile, restart, high barrier, no hot logic change. OpenResty uses LuaJIT + cosocket for non-blocking I/O, enabling safe Redis/service calls on request path. Key: LuaJIT near-C perf + cosocket non-blocking event loop. Q2: Why no os.execute or io.open in Lua? Worker = single-threaded event loop handling thousands of connections. os.execute forks+waits; io.open sync reads disk. Both park the whole worker → all connections on that worker freeze. Same for LuaSocket, sync DNS. External calls must use cosocket. Q3: How does cosocket achieve "sync code, async execution"? Lua coroutines. I/O not ready → yield current request's coroutine, return control to Nginx event loop. Ready → resume. Writer sees sequential logic; runtime sees suspendable coroutines. Prerequisite: running in cosocket-enabled phase (rewrite/access/content/balancer); init and log phases cannot. Q4: What's wrong with shared dict get + set ? Each atomic alone; combo not atomic. Multiple workers read 5, each writes 6 → five concurrent increments only count once. Counters use incr (args: key, delta, init, TTL). Acquisition uses add (succeeds only if absent). Compound logic uses resty.lock . This is why Incident 3 lost 70% counts. Q5: What happens when shared dict fills up? set evicts LRU silently, no error. Canary ratios, node tables randomly disappear → "config reverted itself". Fix: TTL every key, split dicts by purpose, monitor capacity + free_space , alert < 30%. get_keys is O(n) blocking — production banned. Q6: What can/can't balancer_by_lua do? Can: read shared dict, set_current_peer , get_last_failure for retry detection, set_more_tries limit retries, set_timeouts per-attempt. Cannot: any cosocket network I/O (creates "pick backend needs I/O" loop; framework disables), cannot modify request headers (belongs to rewrite/access). Q7: Why server 0.0.0.1 placeholder in upstream? Upstream syntax requires ≥1 server; else nginx -t fails. Real pick by balancer via set_current_peer ; placeholder never used. But set_current_peer failure must return error; else Nginx falls back to placeholder → user sees connect timeout. Q8: init_by_lua vs init_worker_by_lua ? init_by_lua runs once in master at config load (pre-fork). For require modules, load config, warm dict. No event loop → no cosocket, no sleep. init_worker_by_lua runs per worker after start. Only place to start ngx.timer . Periodic sync goes here. Q9: Timer runs in every worker — duplicate writes? Yes, 4 workers = 4 pulls, 4 writes same key. Task must be idempotent full replacement ( set complete list); four writes same value harmless. If truly need single-run, use dict add on expiring lock key to elect leader. Q10: Why dynamic cert parsed per worker? PEM raw shareable via shared dict. But parse_pem_cert output cdata binds to process-internal SSL_CTX — cannot cross workers. Structure: "dict stores raw + each worker parses once + LRU caches 300s" — avoids per-handshake re-parse. Q11: Dynamic cert "configured but not working" — top cause? Missing ssl.clear_certs() . Static ssl_certificate in config stays as default, dynamic cert "covered". Handshake sends default cert, no error. Correct order: clear_certs → set_cert (chain, key). Also client without SNI → no domain. Q12: Variable proxy_pass pitfalls? With variable, no compile-time resolve; runtime decides: value matches upstream name → use directly (no resolver needed). Else → must have resolver for runtime DNS; else startup error or 502. Variable must have fallback; Lua error → empty var → parse fail → 500. Q13: ngx.ctx survive internal redirect? No. error_page , X-Accel-Redirect , rewrite to new location all reset ctx . location.capture subrequest has own ctx . Cross-phase persistent → ngx.var (string); request-scoped → ctx ; cross-request → shared dict. Q14: Why can't log_by_lua POST metrics directly? Log phase = request ended; cosocket disabled (no suspendable request context). Correct: log phase writes to shared dict aggregation buckets; timer (started in init_worker ) batch-pushes to Prometheus Pushgateway or log service. Non-blocking, amortized cost. Q15: Registry down — gateway still works? Yes, if request path doesn't call control plane. Node table, routes, certs, canary ratios pre-synced to shared dict; request reads only memory. Sync failure keeps stale value + increments error counter, no overwrite/clear. Plus hardcoded allowlist nodes in config. Control plane outage only affects "change propagation speed", not service. Q16: Why hash not random for canary? Random = different result per request → user flips between versions → features flicker. uid CRC32 mod 100 fixes hash; same user always same side. Ratio 10→30: original 10% stays in canary, only new 20% added. Monotonic expansion — no one kicked back to old. Random can't guarantee. Q17: Passive health check at Nginx layer? In balancer combine get_last_failure + ngx.ctx.bad_peer : retry means previous node failed → incr down:IP with 30s TTL; filter these keys on pick. Expiry = auto-recovery, natural half-open. No extra probe thread; error itself is signal. Q18: Retry config to avoid amplifying failure? Cap three dimensions: count (max 2), total time ( proxy_next_upstream_timeout 3s ), ratio (error rate > threshold → stop retry, fast-fail). Watch idempotency: GET retry freely; POST only retry connection errors ( error ), never timeout ( timeout may have executed). Else downstream overload → retry multiplies traffic. Q19: Forget set_keepalive — symptom? Connection leak. Each request new connection, not returned. Accumulates until peer timeout releases. Manifest: "hours later connection count creeps up, eventually worker_connections exhausted ", restart fixes, recurs days later. Rule: "fail close, success set_keepalive" paired. Pool size ≈ 1.2× single-worker concurrency. Q20: When APISIX vs custom Lua? Standard API management (auth, rate limit, canary, observability, multi-tenant) → APISIX, don't reinvent. Embed special business logic in traffic path (custom signature, protocol translation, special rewrite) → write Lua directly; plugin model constrains. Pragmatic: APISIX base + critical logic as independent Lua modules.
One-Page Cheat Sheet
OpenResty Cheat Sheet:
Composition : Nginx + LuaJIT + C modules; LuaJIT → near-C perf.
Lifeline : Zero blocking — os.execute / io.open /LuaSocket/sync DNS all banned.
Phases : init no I/O; init_worker only timer starter; balancer no cosocket; log no reporting.
Sharing : globals per-worker isolated; cross-worker only shared dict.
Atomic : incr and add atomic; get+set never.
Dynamic three doors : balancer picks node; ssl_certificate swaps cert; access mutates variable for canary.
Iron law : Request path never calls control plane; failure keeps stale, never clears.
Four pits : forget clear_certs, forget set_keepalive, upstream missing placeholder, variable proxy_pass missing resolver.
Capacity : Shared mem fixed at startup; full = silent LRU eviction.
Summary & Next Episode Preview
Dynamic gateway value isn't "using Lua" — it's reducing change granularity from "config file" to "one key in memory" . Recap:
Reload isn't free — cost scales with connections; high-frequency changes must stay off request path.
Non-blocking is the only performance foundation — one os.execute drops 8k QPS to 200, no exceptions.
Shared ≠ consistent — shared dict only few ops atomic; misuse = silent data loss.
One more easily forgotten rule: Control plane can fail, data plane must not die. Next: Production Practice — capacity planning, release process, incident severity classification, ten real incident postmortem templates — stitching the nine articles into a landable ops system.
Signed-in readers can open the original source through BestHub's protected redirect.
This article has been distilled and summarized from source material, then republished for learning and reference. If you believe it infringes your rights, please contactand we will review it promptly.
How this landed with the community
Was this worth your time?
0 Comments
Thoughtful readers leave field notes, pushback, and hard-won operational detail here.
