How to Multiplex Services on a Single Port with Nginx and HAProxy

Port Reuse Scenarios and Uses

Port reuse allows multiple services to share a single port, useful when IP/port resources are limited, simplifying client configuration, and improving load balancing and high availability.

Protocol Characteristics

Port reuse relies on protocol features; by inspecting the first few bytes of a packet you can identify the protocol and route accordingly. Example: MySQL handshake starts with

0x490000000a

, but MySQL cannot be multiplexed because the first packet is sent by the server after the TCP handshake.

Common protocols have identifiable prefixes, e.g., HTTP GET =

0x47 0x45 0x54

, POST =

0x50 0x4F 0x53

, PUT =

0x50 0x55 0x54

, DELETE =

0x44 0x45 0x4C

, etc.

Using Nginx for Port Reuse

With OpenResty (Nginx + LuaJIT) you can read the first three bytes of the payload in a

preread_by_lua_block

and set a variable to select the appropriate backend (http, redis, ssh).

stream {
    lua_shared_dict protocol_cache 10m;
    upstream redis_backend { server localhost:6379; }
    upstream http_backend { server localhost:80; }
    upstream ssh_backend { server localhost:22; }
    lua_add_variable $backend;
    server {
        listen 3000;
        preread_by_lua_block {
            local sock = ngx.req.socket()
            local data, err = sock:peek(3)
            if not data then
                ngx.log(ngx.ERR, 'receive data failed: ', err)
                return ngx.exit(ngx.ERROR)
            end
            if data == '\x47\x45\x54' or data == '\x50\x4F\x53' or data == '\x50\x55\x54' or data == '\x44\x45\x4C' or data == '\x4F\x50\x54' or data == '\x48\x45\x41' or data == '\x43\x4F\x4E' or data == '\x54\x52\x41' then
                ngx.var.backend = 'http_backend'
            elseif data == '\x2a\x32\x0d' then
                ngx.var.backend = 'redis_backend'
            elseif data == '\x53\x53\x48' then
                ngx.var.backend = 'ssh_backend'
            end
        }
        proxy_pass $backend;
    }
}

Using HAProxy for Port Reuse

HAProxy can inspect the payload with

acl

rules and route to different backends based on the first bytes.

frontend main_front
    bind *:3000
    mode tcp
    tcp-request inspect-delay 5s
    acl is_http req.payload(0,3) -m bin 474554 504f53 505554 44454c 4f5054 484541 434f4e 545241
    acl is_redis req.payload(0,3) -m bin 535348
    acl is_ssh req.payload(0,3) -m bin 535348
    use_backend http_backend if is_http
    use_backend redis_backend if is_redis
    use_backend ssh_backend if is_ssh
    default_backend another_backend
backend http_backend
    mode tcp
    server server1 localhost:80
backend redis_backend
    mode tcp
    server server1 localhost:6379
backend ssh_backend
    mode tcp
    server server1 localhost:22
backend another_backend
    mode tcp
    server server1 localhost:9000

Conclusion

Port reuse enhances flexibility and availability in resource‑constrained environments. By analyzing protocol signatures, you can multiplex services on a single port, with Nginx offering Lua‑based customization and HAProxy providing a straightforward configuration.