Mastering Concurrency: Optimize Nginx, HAProxy & Keepalived for High‑Performance Servers

Preface

Understanding Concurrency

Connection and Request

Connection : TCP connection between client and server. HTTP/1.1 enables Keep‑Alive, allowing multiple HTTP requests over one TCP connection.

Request : An HTTP request sent by the client over a TCP connection.

Concurrent Connections

Concurrent connections refer to the number of active TCP connections a server handles at a given moment, independent of how many requests each connection carries.

If a client opens a TCP connection and keeps it open, it counts as one concurrent connection.

Even if the client sends many requests per second on that connection, it still counts as a single concurrent connection.

Request Processing Capacity

The server’s ability to handle requests is measured in Requests Per Second (RPS). Within a TCP connection:

Single concurrent connection : More requests per second increase load, but the concurrent connection count stays the same.

Multiple concurrent connections : Both total request count and connection count increase.

Configuration and Monitoring

Nginx configuration :

Connection count is determined by worker_processes and worker_connections.

Request handling efficiency depends on Nginx’s event‑driven architecture.

HAProxy configuration :

Connection count is set by the maxconn parameter.

Request capacity can be tuned via queues and buffers.

Example Scenario

Assume an Nginx server with the following configuration:

worker_processes 4;
events {
    worker_connections 1024;
}

The maximum concurrent connections equal worker_processes × worker_connections = 4 × 1024 = 4096. If each connection generates 10 requests per second, the total request rate reaches 4096 × 10 = 40960 RPS.

Viewing Concurrency and Requests

Real‑time Monitoring

Monitoring active connections and request counts helps understand server load.

Nginx

Enable the stub_status module:

location /nginx_status {
    stub_status;
    allow 127.0.0.1;
    deny all;
}

Sample output:

Active connections: 291
server accepts handled requests
802024 802024 1604037
Reading: 12 Writing: 43 Waiting: 236

Explanation of each metric is provided, indicating active load, total connections accepted, handled, total requests, and the state of reading, writing, and waiting connections.

How to Use These Metrics

Active connections : High values may require scaling Nginx instances or adjusting configuration.

accepts / handled : Discrepancies can indicate rejected or failed connections.

requests : Rising request counts may necessitate Nginx tuning or backend optimization.

Reading vs. Writing : Helps identify whether the server is I/O‑bound.

Waiting : Large waiting counts suggest many Keep‑Alive connections; consider lowering the Keep‑Alive timeout.

Concurrency Calculation

For Nginx:

max_concurrent_connections = worker_processes × worker_connections

For HAProxy: maxconn defines the limit, but the operating system must allow enough file descriptors.

System Resource Limits

Each connection consumes a file descriptor. Ensure the OS limits are high enough.

Temporary limit (session‑only):

ulimit -n 65535

Permanent limit (edit /etc/security/limits.conf):

* soft nofile 65535
* hard nofile 65535

Kernel parameters (edit /etc/sysctl.conf):

net.core.somaxconn=10240
net.ipv4.tcp_max_tw_buckets=5000
net.ipv4.tcp_max_syn_backlog=5000
net.ipv4.tcp_fin_timeout=30
net.ipv4.ip_local_port_range = 1024 65535

Check the global file descriptor limit:

cat /proc/sys/fs/file-max
6815744

HAProxy Maxconn Configuration

Assuming the system limits are set, HAProxy can be configured as:

global
    maxconn 100000
    ...

defaults
    maxconn 100000
    timeout http-keep-alive 120s
    timeout connect 1000ms
    timeout client 600ms
    timeout server 600ms
    timeout check 5s

Keepalived Configuration

! Configuration File for keepalived
global_defs {
    notification_email {
        [email protected]
        [email protected]
        [email protected]
    }
    router_id haproxy-2
}
vrrp_script chk_haproxy {
    user root
    script "/etc/keepalived/check_haproxy.sh"
    interval 2
    weight -10
}
vrrp_instance VI_1 {
    state BACKUP
    interface eth0
    virtual_router_id 51
    priority 110
    nopreempt
    advert_int 1
    authentication {
        auth_type PASS
        auth_pass 88888888
    }
    virtual_ipaddress {
        192.168.0.190
    }
    track_script {
        chk_haproxy
    }
}

ab Load‑Testing Tool

Install:

yum install httpd-tools -y   # CentOS
apt install apache2-utils   # Debian/Ubuntu

Run a test:

ab -kc 1000 -n 100000 http://127.0.0.1/

Typical output includes server software, active connections, request rate, latency statistics, and percentile response times.

Monitoring with Prometheus

Prometheus can scrape Nginx’s stub_status metrics. To estimate requests per minute, multiply the measured RPS by 60 and compare against expected capacity (e.g., 1000 RPS → 60 000 requests/minute). Adjust thresholds based on observed load (low: 1 000‑5 000, medium: 5 000‑20 000, high: 20 000‑60 000 requests/minute).

These graphs illustrate the difference in request rates and latency between a single Nginx instance and a proxy‑backed setup.