Mastering High‑Performance Socket Programming with OpenResty and Nginx

Socket Programming

In Linux socket programming, handling massive connection requests requires non‑blocking I/O and I/O multiplexing. The kernel APIs select, poll, and epoll provide this capability, and since Linux 2.6 the epoll interface has become the foundation for high‑performance servers such as Nginx. For high‑performance services the focus is on caching and asynchronous non‑blocking I/O rather than language speed.

Cache hierarchy: memory > SSD > mechanical disk; local machine > network; intra‑process > inter‑process.

The goal of a cache system is to achieve the highest hit rate inside a process, maximizing overall efficiency. Asynchronous non‑blocking I/O lets the server continue serving other clients while waiting for slow resources like databases or network devices.

OpenResty

OpenResty is a high‑performance web platform based on Nginx and Lua. It bundles a rich set of Lua libraries and third‑party modules, enabling rapid development of dynamic web applications that can handle 10K‑1M concurrent connections on a single machine. OpenResty runs Lua scripts inside Nginx’s non‑blocking I/O model, providing consistent high‑performance responses for HTTP clients and backend services.

OpenResty fundamentally combines Nginx with LuaJIT, reshaping the development model for high‑performance servers.

In Nginx’s master‑worker architecture, the master process only handles initialization and worker management. Each worker runs its own Lua VM; when a request is assigned to a worker, a coroutine is created inside that VM. Coroutines have isolated global variables, similar to threads but with cooperative scheduling.

Tests show OpenResty’s performance is comparable to Nginx’s C modules and can even surpass them.

OpenResty Architecture

Load Balancing – LVS + HAProxy forward traffic to core Nginx instances, achieving load distribution.

Single‑Machine Closed Loop – All required data can be obtained from the local server without external network calls.

Distributed Closed Loop – Single‑machine loops face two main problems:

1. Data inconsistency (e.g., lack of master‑slave replication).

2. Storage bottlenecks (disk or memory limits).

Solutions: use master‑slave or distributed centralized storage for consistency, and shard data by business key to alleviate storage limits.

Access Gateway – The entry point that receives traffic and performs tasks such as routing, authentication, and rate limiting.

OpenResty Environment Setup

Prerequisites: install perl, libpcre, and libssl libraries.

# Check required libraries
$ sudo ldconfig -v
# Install required libraries
$ sudo apt install libpcre3-dev libssl-dev perl make build-essential curl libreadline-dev libncurses5-dev

1. Download and extract OpenResty

$ wget https://openresty.org/download/ngx_openresty-1.13.6.1.tar.gz
$ tar -zxvf ngx_openresty-1.13.6.1.tar.gz
$ mv openresty-1.13.6.1 openresty
$ cd openresty
$ ./configure

2. Compile and install (default to /usr/local/openresty)

# Build and install
$ sudo make && make install
$ cd /usr/local/openresty

3. Start Nginx

$ sudo /usr/local/openresty/nginx/sbin/nginx
$ ps -ef | grep nginx
$ service nginx status

4. Resolve common start‑up errors

nginx: [emerg] bind() to 0.0.0.0:80 failed (98: Address already in use)
nginx: [emerg] still could not bind()

The error indicates port 80 is already occupied. Use netstat -ntlp | grep 80 to find the process and kill it with sudo killall -9 nginx.

After fixing, edit /etc/nginx/nginx.conf to adjust listen directives, then reload:

$ sudo vim /etc/nginx/nginx.conf
listen 80;
listen [::]:80 ipv6only=on default_server;
$ sudo nginx -s reload

Test with curl 127.0.0.1 or a browser.

OpenResty Quick Start

1. Create a working directory

$ mkdir -p ~/openresty/test/logs ~/openresty/test/conf
$ vim ~/openresty/test/conf/nginx.conf
# worker_processes 1;
# error_log logs/error.log;
events { worker_connections 10224; }
http {
    server {
        listen 8001;
        location / {
            default_type text/html;
            content_by_lua_block {
                ngx.say("hello world")
            }
        }
    }
}

Run and test:

$ nginx -p ~/openresty/test
$ curl 127.0.0.1:8001
hello world

To serve a Lua file:

$ vim nginx.conf
location /test { content_by_lua_file "lua/test.lua"; }
$ mkdir lua && vim lua/test.lua
local args = ngx.req.get_uri_args()
local salt = args.salt
if not salt then
    ngx.exit(ngx.HTTP_BAD_REQUEST)
end
local md5str = ngx.md5(ngx.time() .. salt)
ngx.say(md5str)
$ sudo /usr/local/openresty/nginx/sbin/nginx -s reload
$ curl -i 127.0.0.1/test?salt=lua

If errors occur, check /usr/local/openresty/nginx/logs/error.log for messages such as missing Lua files.

Windows: view Nginx processes

λ tasklist /fi "imagename eq nginx.exe"
... (output omitted) ...
λ taskkill /im nginx.exe /f

Author information and decorative end sections have been omitted for brevity.