Master Nginx Load Balancing: Upstream, Proxy Pass & Advanced Algorithms

You can place Nginx in front of a web application as a load balancer.

For example, if your enterprise application runs on Apache (or Tomcat), you can set up a second instance of the enterprise application on Apache (or Tomcat) on different servers.

Then you can put Nginx in front, which will load‑balance between the two Apache (or Tomcat, or JBoss) servers.

If you are new to Nginx, it is important to understand the differences between Nginx and Apache and the Nginx architecture.

Nginx supports three types of load balancing:

round-robin – the default type that uses a simple cyclic algorithm to decide where to send incoming requests.

least-connected – as the name suggests, incoming requests are sent to the server with the fewest active connections.

ip-hash – useful when you want to persist or stick a client’s requests to the same server; the client’s IP address determines the target server.

1. Define upstream and proxy_pass in Nginx config file

For load balancing you need to add two things in the nginx configuration file: upstream and proxy_pass.

First, upstream: specify a unique name (often the application name) and list all servers that Nginx will balance.

In the example below, "crmdev" is the upstream name for an application running on two Apache servers (101.1 and 101.2). You can use any name you like.

Note: the upstream should be defined in the http context.

upstream crmdev {
    server 192.168.101.1;
    server 192.168.101.2;
}

Note: if the servers run on ports other than 80, specify the port number in the upstream.

upstream crmdev {
    server 192.168.101.1:8080;
    server 192.168.101.2:8080;
}

Second, proxy_pass: reference the upstream name in a location block inside a server block.

server {
  listen 80;
  
  location / {
    proxy_pass http://crmdev;
  }
}

Note: in this example Nginx itself listens on port 80.

You can also use proxy_pass to set Nginx as a reverse proxy for Apache/PHP.

2. Define upstream and proxy_pass in Nginx default configuration file

Usually the above content should be placed under an http or https block.

http {
  upstream crmdev {
   server 192.168.101.1:8080;
   server 192.168.101.2:8080;
  }

  server {
      listen 80;

      location / {
          proxy_pass http://crmdev;
      }
  }
}

If you use the default default.conf that comes with nginx.conf, you do not need the outer http block because it is already defined.

For HTTPS, replace the http context with https and use https://{your-upstream-name} in the proxy_pass line.

If you keep the http directive in an HTTPS configuration you may see an error like:

Starting nginx: nginx: [emerg] "http" directive is not allowed here in /etc/nginx/conf.d/default.conf:1

Below is a copy of my default.conf where I added the upstream (without the http block) and replaced the default location with a new one using proxy_pass:

# vi /etc/nginx/conf.d/default.conf
upstream crmdev {
    server 127.0.0.1:4080;
    server 127.0.0.1:7080;
}
server {
  listen       3080;
  server_name  localhost;
  
  #location / {
  #    root   /usr/share/nginx/html;
  #    index  index.html index.htm;
  #}
  
  location / {
    proxy_pass http://crmdev;
  }
  ..
  ..
}

3. Set the least_conn algorithm for Nginx load balancer

This algorithm sends incoming requests to the server with the fewest active connections.

upstream crmdev {
    least_conn;
    server 192.168.101.1:8080;
    server 192.168.101.2:8080;
}

If multiple servers have a similar number of active connections, the selection falls back to a weighted round‑robin.

4. Set a sticky (session‑persistence) algorithm for Nginx load balancer

Round‑robin and least‑connected do not guarantee that subsequent connections from the same client go to the same server, which is fine for stateless applications.

For session‑dependent applications you should use the ip_hash algorithm so that all requests from a client are directed to the same server.

upstream crmdev {
    ip_hash;
    server 192.168.101.1:8080;
    server 192.168.101.2:8080;
}

For IPv4 addresses the hash uses the first three octets; for IPv6 the entire address is used.

5. Weight option for individual servers

By default all servers have the same priority (weight = 1). You can assign a different weight to influence load distribution.

upstream crmdev {
    server 192.168.101.1:8080;
    server 192.168.101.2:8080;
    server 192.168.101.3:8080 weight 2;
    server 192.168.101.4:8080;
    server 192.168.101.5:8080;
}

In this example the third server has weight 2, meaning it will receive 2 out of every 6 new requests, while the others receive 1 each.

You can also use weights with the least_conn and ip_hash algorithms.

6. Timeout options for individual servers – max_fails and fail_timeout

You can specify max_fails and fail_timeout for each server:

upstream crmdev {
    server 192.168.101.1:8080 max_fails=3 fail_timeout=30s;
    server 192.168.101.2:8080;
    server 192.168.101.3:8080 weight 2;
    server 192.168.101.4:8080;
    server 192.168.101.5:8080;
}

The default fail_timeout is 10 seconds; in the example it is set to 30 seconds. If a server fails max_fails times within this period, it is marked unavailable for the duration of fail_timeout.

The default max_fails is 1. In the example it is set to 3, meaning the server is considered down after three consecutive failures.

7. Assign a backup server in Nginx load‑balancer pool

In the following example the fifth server is marked as a backup using the backup keyword.

upstream crmdev {
    server 192.168.101.1:8080 max_fails=3 fail_timeout=30s;
    server 192.168.101.2:8080;
    server 192.168.101.3:8080 weight 2;
    server 192.168.101.4:8080;
    server 192.168.101.5:8080 backup;
}

The backup server (192.168.101.5) will only receive traffic if all other four servers are unavailable.