Master Nginx: From Basics to Advanced Load Balancing, Caching and High Availability

Introduction

Early monolithic deployments can become a bottleneck as traffic grows; introducing load balancing with Nginx improves high availability, performance, and scalability.

1. Nginx Concept Overview

Nginx is a lightweight, high‑performance HTTP reverse proxy and generic proxy supporting TCP, UDP, SMTP, HTTPS, etc. It shares the event‑driven, multiplexed model with Redis, offering low resource consumption and high concurrency (e.g., 50,000 connections).

2. Nginx Environment Setup

# Create installation directory
mkdir /soft && mkdir /soft/nginx && cd /soft/nginx
# Download source
wget https://nginx.org/download/nginx-1.21.6.tar.gz
# Extract
tar -xvzf nginx-1.21.6.tar.gz
# Install dependencies
yum -y install gcc zlib zlib-devel pcre pcre-devel openssl openssl-devel
# Configure and compile
./configure --prefix=/soft/nginx/
make && make install

After installation, edit

conf/nginx.conf

to set the listening port and server name.

3. Reverse Proxy Load Balancing

upstream nginx_boot {
    server 192.168.0.100:8080 weight=100 max_fails=2 fail_timeout=30s;
    server 192.168.0.101:8090 weight=200 max_fails=2 fail_timeout=30s;
}
server {
    location / {
        proxy_pass http://nginx_boot;
        proxy_set_header Host $host;
        proxy_set_header X-Real-IP $remote_addr;
        proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
    }
}

Requests are first handled by Nginx, then distributed to the backend services according to the defined weights (e.g., 2:1 for ports 8080 and 8090).

4. Static/Dynamic Separation

location ~ .*\\.(html|htm|gif|jpg|jpeg|bmp|png|ico|txt|js|css) {
    root /soft/nginx/static_resources;
    expires 7d;
}

Static assets are served directly by Nginx, reducing backend load by more than 50% for typical web pages.

5. Resource Compression

http {
    gzip on;
    gzip_types text/plain application/javascript text/css application/xml image/jpeg image/gif image/png;
    gzip_comp_level 5;
    gzip_vary on;
    gzip_buffers 16 8k;
    gzip_disable "MSIE [1-6]\\.";
    gzip_http_version 1.1;
    gzip_min_length 2k;
    gzip_proxied off;
}

Enabling gzip reduces a 230 KB JavaScript file to about 69 KB, dramatically improving transfer speed.

6. Buffer Settings

http {
    proxy_connect_timeout 10;
    proxy_read_timeout 120;
    proxy_send_timeout 10;
    proxy_buffering on;
    client_body_buffer_size 512k;
    proxy_buffers 4 64k;
    proxy_buffer_size 16k;
    proxy_busy_buffers_size 128k;
    proxy_temp_path /soft/nginx/temp_buffer;
}

Buffers smooth the speed mismatch between client and backend connections.

7. Cache Mechanism

# Cache directory and zone
proxy_cache_path /soft/nginx/cache levels=1:2 keys_zone=hot_cache:128m inactive=3d max_size=2g;
server {
    location / {
        proxy_cache hot_cache;
        proxy_cache_valid 200 206 304 301 302 1d;
        proxy_cache_valid any 30m;
        proxy_cache_key $host$uri$is_args$args;
        proxy_cache_min_uses 3;
        proxy_cache_lock on;
        proxy_cache_lock_timeout 3s;
        proxy_no_cache $cookie_nocache $arg_nocache $arg_comment;
        add_header Cache-Status $upstream_cache_status;
    }
}

Cache is populated only after a resource is requested three times, preventing useless entries.

8. IP Whitelist / Blacklist

# BlocksIP.conf (blacklist)
deny 192.177.12.222;
deny 192.177.44.201;
deny 127.0.0.0/8;

# WhiteIP.conf (whitelist)
allow 192.177.12.222;
allow 192.177.44.201;
allow 127.45.0.0/16;
deny all;

Include the appropriate file in the

http

,

server

or

location

block to apply site‑wide or endpoint‑specific restrictions.

9. Cross‑Domain (CORS) Configuration

location / {
    add_header Access-Control-Allow-Origin *;
    add_header Access-Control-Allow-Credentials "true";
    add_header Access-Control-Allow-Methods "GET,POST,OPTIONS,PUT";
    add_header Access-Control-Allow-Headers *;
    add_header Access-Control-Expose-Headers "Content-Length,Content-Range";
    if ($request_method = 'OPTIONS') {
        add_header Access-Control-Max-Age 1728000;
        add_header Content-Type "text/plain; charset=utf-8";
        add_header Content-Length 0;
        return 204;
    }
}

This enables browsers to make cross‑origin AJAX calls safely.

10. Anti‑Hotlinking

location ~ .*\\.(html|htm|gif|jpg|jpeg|bmp|png|ico|txt|js|css) {
    valid_referers none blocked server_names;
    if ($invalid_referer) { return 403; }
    root /soft/nginx/static_resources;
    expires 7d;
}

Requests without a valid

Referer

header are denied, protecting assets from unauthorized embedding.

11. Large File Transfer

client_max_body_size 2g;
client_header_timeout 60s;
proxy_read_timeout 600s;
proxy_send_timeout 600s;

Adjust these values according to the expected file size and network conditions.

12. SSL Certificate Configuration

# HTTPS server block
server {
    listen 443 ssl;
    server_name www.example.com;
    ssl_certificate /soft/nginx/certificate/example.pem;
    ssl_certificate_key /soft/nginx/certificate/example.key;
    ssl_session_timeout 5m;
    ssl_ciphers ECDHE-RSA-AES128-GCM-SHA256:ECDHE:ECDH:AES:HIGH:!NULL:!aNULL:!MD5:!ADH:!RC4;
    ssl_protocols TLSv1 TLSv1.1 TLSv1.2;
    ssl_prefer_server_ciphers on;
    location / { ... }
}
# Redirect HTTP to HTTPS
server {
    listen 80;
    server_name www.example.com;
    return 301 https://$host$request_uri;
}

After configuring the certificate files, the site is accessible via HTTPS and all HTTP traffic is redirected.

13. High Availability with Keepalived

# /etc/keepalived/keepalived.conf (master)
global_defs {
    router_id 192.168.12.129
}
vrrp_instance VI_1 {
    state MASTER
    interface eth0
    virtual_router_id 121
    priority 100
    nopreempt
    advert_int 1
    authentication { auth_type PASS; auth_pass 1111; }
    track_script { check_nginx_pid_restart }
    virtual_ipaddress { 192.168.12.111 }
}
# Script check_nginx_pid_restart.sh
#!/bin/sh
nginx_number=$(ps -C nginx --no-header | wc -l)
if [ $nginx_number -eq 0 ]; then
    /soft/nginx/sbin/nginx -c /soft/nginx/conf/nginx.conf
    sleep 1
    if [ $(ps -C nginx --no-header | wc -l) -eq 0 ]; then
        systemctl stop keepalived.service
    fi
fi

The VIP (192.168.12.111) moves to the backup node automatically when the master fails, ensuring uninterrupted service.

14. Performance Optimizations

Enable keepalive connections:

keepalive 32;

and

keepalive_timeout 60s;

Turn on zero‑copy:

sendfile on;

Configure TCP options:

tcp_nodelay on;

for low‑latency services or

tcp_nopush on;

for high‑throughput static delivery.

Set

worker_processes auto;

and increase

worker_rlimit_nofile 20000;

Bind workers to CPUs:

worker_cpu_affinity auto;

Use epoll and raise

worker_connections 10240;

Conclusion

The guide covered Nginx fundamentals, deployment steps, load balancing, static‑dynamic separation, compression, buffering, caching, security controls, large‑file handling, SSL, high availability with Keepalived, and a set of proven performance tweaks. Applying these practices yields a robust, scalable, and high‑performance web infrastructure.