Master Concurrency: Nginx & HAProxy Config, Monitoring & Performance Tuning

Preface

Understanding Concurrency Before Optimization

Connection and Request

Connection : the TCP link between client and server. HTTP/1.1+ enables Keep‑Alive by default, allowing multiple HTTP requests to reuse a single TCP connection.

Request : an individual HTTP request sent by the client over a TCP connection.

Concurrent Connections

Concurrent connections refer to the number of active TCP connections a server handles at a given moment, independent of the number of requests per connection.

If a client establishes a TCP connection and keeps it open, it counts as one concurrent connection.

Even if the client sends multiple requests per second on that connection, it still counts as a single concurrent connection.

Request Processing Capacity

Server request handling ability is measured by request rate, usually expressed as Requests Per Second (RPS). In a TCP connection:

Single concurrent connection : multiple requests per second increase the request count, but the concurrent connection count stays the same.

Multiple concurrent connections : each connection may send multiple requests, increasing both total request count and total connection count.

Configuration and Monitoring

Nginx configuration :

Connection count is determined by worker_processes and worker_connections .

Request handling benefits from Nginx's event‑driven architecture.

HAProxy configuration :

Connection count is set by the maxconn parameter.

Request handling can be tuned via queues and buffers.

Example Scenario

Assume an Nginx server with the following configuration:

worker_processes 4;
events {
    worker_connections 1024;
}

The maximum concurrent connections equal worker_processes × worker_connections, i.e., 4 × 1024 = 4096.

If each connection sends 10 requests per second, the total request rate reaches 4096 × 10 = 40960 RPS.

Monitoring Concurrency and Requests

Real‑time Monitoring

Monitoring active connections and request counts helps assess server load.

Nginx

Enable the stub_status module to view current metrics:

location /nginx_status {
    stub_status;
    allow 127.0.0.1;
    deny all;
}

Sample output:

Active connections: 291
server accepts handled requests
 802024 802024 1604037
Reading: 12 Writing: 43 Waiting: 236

Metric Details

Active connections : currently processing connections (reading, writing, Keep‑Alive).

server accepts handled requests : accepts : total connections accepted. handled : connections successfully processed. requests : total requests processed (often larger than handled because of Keep‑Alive).

Reading : connections reading request headers.

Writing : connections sending response data.

Waiting : idle Keep‑Alive connections awaiting the next request.

How to Use This Information

Active connections : a high value may indicate overload; consider scaling Nginx instances or adjusting configuration.

accepts vs. handled : disparity suggests connection rejections or failures.

requests : rising values may require Nginx tuning or backend optimization.

Reading/Writing : help identify whether the server is I/O‑bound on input or output.

Waiting : many idle Keep‑Alive connections may warrant a shorter timeout.

Concurrency Calculation

For Nginx:

max concurrent connections = worker_processes × worker_connections

.

For HAProxy: ensure the operating system supports the required number of file descriptors, as each connection consumes one descriptor.

System Resource Limits

Temporary limit (session‑only): ulimit -n 65535 Permanent limit (edit /etc/security/limits.conf):

* soft nofile 65535
* hard nofile 65535

Kernel parameters (edit /etc/sysctl.conf):

net.core.somaxconn=10240
net.ipv4.tcp_max_tw_buckets=5000
net.ipv4.tcp_max_syn_backlog=5000
net.ipv4.tcp_fin_timeout=30
net.ipv4.ip_local_port_range = 1024 65535

Check total file descriptors the system can open:

cat /proc/sys/fs/file-max
6815744

HAProxy Maximum Connections

Assuming the system limits are configured, set global maxconn accordingly (e.g., 100000, not exceeding the file‑descriptor limit).

global
    maxconn 100000
    chroot /usr/local/haproxy
    stats socket /var/lib/haproxy/haproxy.sock mode 600 level admin
    uid 188
    gid 188
    daemon
    nbthread 4
    cpu-map 1/1 0
    cpu-map 1/2 1
    cpu-map 1/3 2
    cpu-map 1/4 3

defaults
    option http-keep-alive
    option forwardfor
    option redispatch
    maxconn 100000
    mode http
    timeout http-keep-alive 120s
    timeout connect 1000ms
    timeout client 600ms
    timeout server 600ms
    timeout check 5s

listen stats
    mode http
    bind 0.0.0.0:9999
    stats enable
    stats uri /haproxy-status
    stats auth haadmin:123456

listen nginx
    balance roundrobin
    bind 0.0.0.0:8079
    server web1 192.168.0.120:8078 check inter 3000 fall 2 rise 3
    server web2 192.168.0.61:8078 check inter 3000 fall 2 rise 3

keepalived Configuration

! Configuration File for keepalived

global_defs {
   notification_email {
     [email protected]
     [email protected]
     [email protected]
   }
   router_id haproxy-2
}

vrrp_script chk_haproxy {
    user root
    script "/etc/keepalived/check_haproxy.sh"
    interval 2
    weight -10
}

vrrp_instance VI_1 {
    state BACKUP
    interface eth0
    virtual_router_id 51
    priority 110
    nopreempt
    advert_int 1
    authentication {
        auth_type PASS
        auth_pass 88888888
    }
    virtual_ipaddress {
        192.168.0.190
    }
    track_script {
        chk_haproxy
    }
}

ab Stress‑Testing Tool

Install:

yum install httpd-tools -y   # on RHEL/CentOS
apt install apache2-utils   # on Debian/Ubuntu

Run a test: ab -kc 1000 -n 100000 http://127.0.0.1/ Typical output includes server software, hostname, port, document length, concurrency level, total transferred bytes, requests per second, time per request, and connection time statistics.

Installing Disk Monitoring Tool

yum install sysstat

Concurrency Test Comparison

Comparing a single Nginx instance with a proxy‑based setup shows lower per‑request latency and higher throughput for the proxy configuration.

How to Set Up Monitoring

Prometheus can scrape Nginx metrics to calculate requests per minute. Define a suitable range based on benchmark results (e.g., 1 000 RPS → 60 000 requests/min) and adjust alerts accordingly.

Reasonable Load Ranges

Low load : 1 000 – 5 000 requests/minute

Medium load : 5 000 – 20 000 requests/minute

High load : 20 000 – 60 000 requests/minute

Use Prometheus query results to verify actual traffic falls within these intervals and to assess whether the server resources are sufficient.

Original source: https://www.cnblogs.com/rtnb/p/18238885

(© original author, all rights reserved)