Understanding Linux TCP Socket Internals and Port Range Configuration

The article begins by showing how to enlarge the local port range used by the kernel:

echo "5000 65000" > /proc/sys/net/ipv4/ip_local_port_range

It then demonstrates how to raise the maximum number of open file descriptors for the whole system and for each user:

// modify the system-wide file descriptor limit to 200,000
echo 200000 > /proc/sys/fs/file-max

// set per‑process limits in /etc/sysctl.conf and /etc/security/limits.conf
fs.nr_open=210000
*  soft  nofile  200000
*  hard  nofile  200000

Note: the hard limit in limits.conf cannot exceed nr_open , so nr_open must be increased first, preferably in sysctl.conf .

The core of the article examines the kernel data structure sock_common defined in include/net/sock.h :

struct sock_common {
    union {
        __addrpair skc_addrpair; // IP pair
        struct {
            __be32 skc_daddr;
            __be32 skc_rcv_saddr;
        };
    };
    union {
        __portpair skc_portpair; // port pair
        struct {
            __be16 skc_dport;
            __u16  skc_num;
        };
    };
    ...
}

When a network packet reaches the NIC it passes through DMA, hard‑interrupt, soft‑interrupt processing and finally lands in the socket’s receive queue.

The entry point for TCP packet handling is tcp_v4_rcv (in net/ipv4/tcp_ipv4.c ), which extracts the TCP and IP headers and looks up the corresponding socket:

int tcp_v4_rcv(struct sk_buff *skb) {
    ...
    th = tcp_hdr(skb);   // get TCP header
    iph = ip_hdr(skb);   // get IP header
    sk = __inet_lookup_skb(&tcp_hashinfo, skb, th->source, th->dest);
    ...
}

The lookup is performed by __inet_lookup (in include/net/inet_hashtables.h ), which first tries to find an established socket:

static inline struct sock *__inet_lookup(struct net *net,
                                         struct inet_hashinfo *hashinfo,
                                         const __be32 saddr, const __be16 sport,
                                         const __be32 daddr, const __be16 dport,
                                         const int dif)
{
    u16 hnum = ntohs(dport);
    struct sock *sk = __inet_lookup_established(net, hashinfo,
                                                saddr, sport, daddr, hnum, dif);
    return sk ? : __inet_lookup_listener(net, hashinfo, saddr, sport,
                                         daddr, hnum, dif);
}

The helper __inet_lookup_established builds a combined port value, computes a hash, and walks the hash bucket list to find a matching socket:

struct sock *__inet_lookup_established(struct net *net,
                                      struct inet_hashinfo *hashinfo,
                                      const __be32 saddr, const __be16 sport,
                                      const __be32 daddr, const u16 hnum,
                                      const int dif)
{
    const __portpair ports = INET_COMBINED_PORTS(sport, hnum);
    unsigned int hash = inet_ehashfn(net, daddr, hnum, saddr, sport);
    unsigned int slot = hash & hashinfo->ehash_mask;
    struct inet_ehash_bucket *head = &hashinfo->ehash[slot];
    begin:
    sk_nulls_for_each_rcu(sk, node, &head->chain) {
        if (sk->sk_hash != hash)
            continue;
        if (likely(INET_MATCH(sk, net, acookie, saddr, daddr, ports, dif))) {
            if (unlikely(!atomic_inc_not_zero(&sk->sk_refcnt)))
                goto begintw;
            if (unlikely(!INET_MATCH(sk, net, acookie, saddr, daddr, ports, dif))) {
                sock_put(sk);
                goto begin;
            }
            goto out;
        }
    }
}

The macro INET_MATCH compares the packet’s source/destination addresses and ports with the socket’s stored values, as well as the bound device and network namespace:

#define INET_MATCH(__sk, __net, __cookie, __saddr, __daddr, __ports, __dif) \
    ((inet_sk(__sk)->inet_portpair == (__ports)) && \
     (inet_sk(__sk)->inet_daddr == (__saddr)) && \
     (inet_sk(__sk)->inet_rcv_saddr == (__daddr)) && \
     (!(__sk)->sk_bound_dev_if || \
      ((__sk)->sk_bound_dev_if == (__dif))) && \
     net_eq(sock_net(__sk), (__net)))

These mechanisms together ensure that a TCP connection is uniquely identified by the four‑tuple (source IP, source port, destination IP, destination port) and that the kernel can reliably locate the correct socket even when many connections share the same port on the client side.

The article concludes with practical advice: to increase the maximum number of concurrent TCP connections on a client, either assign multiple IP addresses to the client or connect to many different servers, but avoid mixing the two approaches because binding to a specific IP changes the kernel’s port‑selection strategy.

Experimental results show that, with the described kernel tweaks, a client can handle well over a million concurrent TCP connections, dispelling the common fear of the 65,535‑port limit.