Which Nginx Load‑Balancing Strategy Wins? Deep Code Dive & Real‑World Tests

Introduction

With the explosive growth of internet traffic, load balancing has become essential for ensuring service availability and fast response times. While hardware load balancers are powerful, their cost drives the popularity of software solutions like Nginx, which offers several built‑in and extensible balancing strategies.

Source‑Code Analysis

Weighted Round Robin

The weighted round robin (WRR) algorithm first distributes requests to the highest‑weight servers until their weight drops below the next server, then proceeds to the next weight tier. When all back‑ends are down, Nginx resets the state flags to avoid a total timeout.

The relevant source files are ngx_http_upstream_round_robin.c and ngx_http_upstream_round_robin.h in the Nginx 1.0.15 source tree.

Key structures include current_weight (dynamic during request handling) and weight (static configuration). The tried bitmap records whether a server has already been attempted; it uses a single int for up to 32 servers or allocates memory from the pool for larger sets.

IP Hash

IP hash uses the client IP combined with the number of back‑ends to compute a hash. The algorithm can generate up to 1,045 distinct hash values before it falls back to round robin after 20 unsuccessful attempts, making it effectively a variant of round robin with potential load‑distribution pitfalls when many clients share the same IP range.

Fair

The fair module, an optional third‑party strategy, selects the back‑end with the lowest recent response time. While it adapts well to varying loads, its performance can fluctuate dramatically in unstable network conditions, so thorough testing is recommended before production use.

Generic Hash & Consistent Hash

Both are third‑party extensions. Generic hash hashes a configurable key (often an Nginx variable), while consistent hash uses a ring structure that can integrate with services like memcached, providing smoother key redistribution when servers are added or removed.

Comparative Testing

Test Tools

easyABC : An internal performance tool capable of generating tens of thousands of requests per second, used to drive the Nginx reverse‑proxy under test.

polygraph : An open‑source load generator that supports configurable virtual robots, request rates, and random URL generation, useful for exercising hash‑based strategies.

Test Environment

Five physical machines were used: one 8‑core server running the Nginx instance under test, and four 4‑core machines hosting easyABC, polygraph, and stub web servers.

Test Plan

Four scenarios were executed, each building on the previous state without restarting Nginx:

All back‑ends operational.

Server 4 down.

Servers 3 and 4 down.

All servers restored.

easyABC generated ~17,000 QPS, while polygraph generated ~4,000 QPS. Metrics recorded included balance, consistency, and fault‑tolerance (QPS per server) for each strategy.

Results

Weighted Round Robin demonstrated excellent balance and fault‑tolerance across both tools, with identical results confirming tool‑independent behavior.

Fair showed large variance due to its adaptive nature; while it can exploit idle resources, its balance suffered under the test conditions.

Generic Hash & Consistent Hash behaved similarly, but both lost traffic when a back‑end failed because the hash key remained bound to the missing server. In contrast, IP hash falls back to round robin, avoiding traffic loss.

Real‑world IP hash traffic patterns, captured from production, illustrate its imbalance: the first third of requests follow round robin, the middle third follow IP hash (showing clustering), and the final third revert to round robin.

Conclusion & Outlook

Testing confirms that no single load‑balancing algorithm fits all scenarios. Weighted round robin offers reliable balance and resilience, IP hash provides deterministic routing but can suffer from uneven traffic distribution, fair adapts to response times but is unstable in volatile networks, and hash‑based extensions require careful handling of server failures.

Choosing the appropriate strategy depends on the specific workload, traffic patterns, and operational requirements. Future work may explore hybrid approaches or dynamic strategy selection based on real‑time metrics.