Implementing a Fastest‑First Load‑Balancing Strategy for NGINX with Lua and OpenResty

The author built a custom load‑balancing strategy that always prefers the backend with the shortest response time, using OpenResty (NGINX with many bundled modules) and Lua scripts.

The implementation relies on a Lua module ( dynamic-upstream-weight.lua) and two global key‑value caches to store latency data, as described in the author’s NGINX site configuration.

To enable dynamic weight adjustments, the author forked lua-upstream-nginx-module (commit 6b40d40a4) and added Lua APIs for modifying a server’s weight, effective_weight, and current_weight.

Design principles (four laws) :

Define “fast” by the NGINX variable upstream_response_time; a smaller value indicates a faster server, and assign it a larger weight.

If a server becomes unavailable, lower its weight to a minimal value (e.g., 1) instead of removing it entirely, allowing occasional health checks.

Update weights only when certain triggers occur (e.g., response‑time deviation exceeds a threshold) to avoid excessive re‑weighting.

Avoid getting stuck in a local optimum by ensuring the algorithm can still discover the truly fastest backend.

Implementation details:

Lua code runs in log_by_lua_file to read response times and adjust weights.

The custom directive labor is used in the upstream block to activate the strategy, e.g.:

upstream backend {

labor;

server back1.example.com;

server back2.example.com;

}

Docker build notes:

The author shares a Dockerfile that installs OpenResty dependencies, compiles the modified module, and configures log forwarding to Docker’s stdout/stderr using symbolic links:

RUN ln -sf /dev/stdout ar/loginx/access.log

RUN ln -sf /dev/stderr ar/loginx/error.log

Additional observations include the difficulty of compiling OpenResty locally, the use of a pre‑built compilation environment on a CI platform, and various ways to handle log collection inside containers.

Overall, the article provides a practical guide for developers who need latency‑aware load balancing in NGINX without purchasing NGINX Plus, demonstrating how Lua and OpenResty can extend NGINX’s capabilities.