Unlock Nginx Mastery: Load Balancing, Caching, SSL, and High‑Availability Explained

Introduction

Early monolithic deployments could handle low traffic, but as business grows the single‑node architecture becomes a bottleneck, leading to frequent crashes and system paralysis. Introducing load balancing solves these problems by providing high availability, high performance, and easy scalability.

Why Choose Nginx

Nginx is a lightweight, high‑performance HTTP reverse proxy and supports many protocols (TCP, UDP, SMTP, HTTPS). Like Redis, it uses an event‑driven, multiplexed model, allowing a single instance to handle tens of thousands of concurrent connections.

Environment Setup

Steps to compile and install Nginx from source include creating a directory, downloading the tarball, installing dependencies (gcc, pcre, zlib, openssl), configuring with ./configure --prefix=/soft/nginx/, building with make && make install, and verifying the installation.

# mkdir /soft/nginx && cd /soft/nginx
wget https://nginx.org/download/nginx-1.21.6.tar.gz
tar -xvf nginx-1.21.6.tar.gz
cd nginx-1.21.6
./configure --prefix=/soft/nginx/
make && make install

Reverse Proxy Load Balancing

Define an upstream group with weighted servers and configure a location block to proxy requests.

upstream nginx_boot {
    server 192.168.0.100:8080 weight=100 max_fails=2 fail_timeout=30s;
    server 192.168.0.101:8090 weight=200 max_fails=2 fail_timeout=30s;
}
server {
    location / {
        proxy_pass http://nginx_boot;
        proxy_set_header Host $host;
        proxy_set_header X-Real-IP $remote_addr;
        proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
    }
}

Static/Dynamic Separation

Serve static assets directly from Nginx to reduce backend load. Example location rule:

location ~* \.(html|htm|gif|jpg|jpeg|bmp|png|ico|txt|js|css)$ {
    root /soft/nginx/static_resources;
    expires 7d;
}

Resource Compression

Enable gzip compression for text‑based resources to save bandwidth and improve response time.

http {
    gzip on;
    gzip_types text/plain application/javascript text/css application/xml image/jpeg image/gif image/png;
    gzip_comp_level 5;
    gzip_vary on;
    gzip_min_length 2k;
}

Buffering

Configure proxy buffering to smooth differences between client and upstream speeds.

http {
    proxy_buffering on;
    client_body_buffer_size 512k;
    proxy_buffers 4 64k;
    proxy_buffer_size 16k;
    proxy_busy_buffers_size 128k;
    proxy_temp_path /soft/nginx/temp_buffer;
}

Caching

Set up proxy cache with a shared memory zone, define cache key, validity, and control stale behavior.

proxy_cache_path /soft/nginx/cache levels=1:2 keys_zone=hot_cache:128m inactive=3d max_size=2g;
server {
    location / {
        proxy_cache hot_cache;
        proxy_cache_valid 200 206 304 301 302 1d;
        proxy_cache_valid any 30m;
        proxy_cache_key $host$uri$is_args$args;
        proxy_cache_min_uses 3;
        proxy_cache_lock on;
        proxy_cache_lock_timeout 3s;
        add_header Cache-Status $upstream_cache_status;
    }
}

IP Black/White List

Control access using allow and deny. Separate files can be included for large lists.

# BlocksIP.conf
deny 192.177.12.222;
deny 192.177.44.201;
# WhiteIP.conf
allow 192.177.12.222;
allow 192.177.44.201;
allow 127.45.0.0/16;
deny all;

Cross‑Domain (CORS) Support

Add the necessary headers to allow any origin, credentials, and common methods. Handle preflight OPTIONS requests.

location / {
    add_header 'Access-Control-Allow-Origin' '*';
    add_header 'Access-Control-Allow-Credentials' 'true';
    add_header 'Access-Control-Allow-Methods' 'GET,POST,OPTIONS,PUT';
    add_header 'Access-Control-Allow-Headers' '*';
    if ($request_method = 'OPTIONS') {
        add_header 'Access-Control-Max-Age' 1728000;
        add_header 'Content-Type' 'text/plain; charset=utf-8';
        add_header 'Content-Length' 0;
        return 204;
    }
}

Anti‑Hotlinking

Use valid_referers to block requests without a proper Referer header.

location ~* \.(html|htm|gif|jpg|jpeg|bmp|png|ico|txt|js|css)$ {
    valid_referers none blocked server_names www.example.com;
    if ($invalid_referer) { return 403; }
    root /soft/nginx/static_resources;
    expires 7d;
}

Large File Transfer

Adjust client and proxy timeouts and body size to handle big files.

client_max_body_size 500m;
client_body_timeout 120s;
proxy_read_timeout 300s;
proxy_send_timeout 300s;

SSL Configuration

Configure HTTPS server block with certificate and key files, enable modern TLS versions and ciphers.

server {
    listen 443 ssl;
    server_name www.example.com;
    ssl_certificate /soft/nginx/certificate/example.pem;
    ssl_certificate_key /soft/nginx/certificate/example.key;
    ssl_session_timeout 5m;
    ssl_ciphers ECDHE-RSA-AES128-GCM-SHA256:ECDHE:!NULL:!MD5:!RC4;
    ssl_protocols TLSv1 TLSv1.1 TLSv1.2;
    ssl_prefer_server_ciphers on;
    location / { ... }
}
# Redirect HTTP to HTTPS
server {
    listen 80;
    server_name www.example.com;
    return 301 https://$host$request_uri;
}

High Availability with Keepalived

Deploy two Nginx nodes with Keepalived providing a virtual IP (VIP). The master monitors Nginx and restarts it if it crashes; the VIP fails over to the backup node on failure.

# /etc/keepalived/keepalived.conf
global_defs {
    router_id 192.168.12.129;
}
vrrp_script check_nginx {
    script "/soft/scripts/check_nginx.sh";
    interval 3;
    weight -20;
}
vrrp_instance VI_1 {
    state MASTER;
    interface eth0;
    virtual_router_id 121;
    priority 100;
    nopreempt;
    advert_int 1;
    authentication {
        auth_type PASS;
        auth_pass 1111;
    }
    track_script { check_nginx }
    virtual_ipaddress { 192.168.12.111 }
}

Performance Optimizations

Key tuning items include enabling keepalive connections, zero‑copy sendfile on, configuring tcp_nodelay or tcp_nopush based on latency vs throughput needs, setting worker_processes auto, increasing worker_rlimit_nofile, using worker_cpu_affinity auto, and selecting the epoll event model with a high worker_connections limit.

keepalive_timeout 60s;
keepalive_requests 100;
sendfile on;
# For low latency
tcp_nodelay on;
# For high throughput
tcp_nopush on;
worker_processes auto;
worker_rlimit_nofile 20000;
worker_cpu_affinity auto;
events { use epoll; worker_connections 10240; }

Conclusion

The guide covered Nginx installation, load balancing, static asset handling, compression, buffering, caching, access control, CORS, anti‑hotlinking, large file support, SSL, high‑availability with Keepalived, and a set of performance‑tuning knobs, providing a solid foundation for building robust, scalable backend services.