Demystifying TCP Socket Lifecycle: From Creation to Closure

This article explains the operations on sockets at each stage of the TCP connection process, aiming to help readers without network programming background understand what a socket is and its role. Please point out any errors.

1. Background

1. Complete socket format {protocol, src_addr, src_port, dest_addr, dest_port}.

This is commonly called the socket five‑tuple. The protocol field indicates whether the connection is TCP or UDP; the remaining fields specify source address, source port, destination address, and destination port.

2. The TCP protocol stack maintains two socket buffers: send buffer and recv buffer.

Data to be sent over a TCP connection is first copied into the send buffer, either from a user‑space application buffer or from a kernel buffer, using the send() (or write()) function. The data is then transferred to the network card, typically via DMA, without CPU involvement.

When receiving data, the network card copies incoming packets into the recv buffer (also via DMA), after which the recv() function copies the data into the application buffer.

The overall process is illustrated in the figure below:

3. Two kinds of sockets: listening socket and connected socket.

A listening socket is created by a server process after reading its configuration file, calling socket() to obtain a descriptor, then bind() to associate it with a specific address and port, and finally listen() to start listening for incoming connections.

A connected socket is returned by accept() after the three‑way handshake completes; the server can then use this socket to communicate with the client.

For clarity, some code uses listenfd for the listening descriptor and connfd for the connected descriptor.

2. Detailed analysis of the connection process

The following diagrams show the sequence of events:

2.1 socket() function

The socket() function creates an endpoint for communication and returns a descriptor that can later be bound with bind().

2.2 bind() function

The server parses the configuration file to obtain the address and port to listen on, then calls bind() to associate the previously created socket descriptor with that address‑port pair.

After binding, the socket has a source address and source port (from the server's perspective). Combined with the protocol type from the configuration, the five‑tuple is partially formed: {protocal,src_addr,src_port} Multiple instances can be created by repeating the socket() + bind() sequence.

2.3 listen() and connect() functions

The listen() function puts a bound socket into LISTEN state, making it ready to accept incoming TCP connections.

The connect() function is used by a client to initiate a three‑way handshake with a listening socket, supplying the destination address and port as well as its own source address and port, thereby completing the five‑tuple on both sides.

2.3.1 In‑depth analysis of listen()

When multiple sockets are listened on, the process/thread typically uses select(), poll(), or epoll() to monitor them. The listening thread blocks on these calls until a SYN packet arrives, wakes up, copies the SYN into the application buffer, sends a SYN+ACK, and creates an entry in the incomplete‑connection queue (state SYN_RECV). Subsequent ACK processing moves the entry to the completed‑connection queue (state ESTABLISHED).

The listen() function maintains two queues: an incomplete‑connection queue and a completed‑connection queue. The backlog parameter (pre‑Linux 2.2) set the total length; modern kernels use /proc/sys/net/ipv4/tcp_max_syn_backlog for the incomplete queue and /proc/sys/net/core/somaxconn for the completed queue.

When the completed‑connection queue is full, further connections are blocked. The netstat command shows the socket buffer statistics: man netstat explanation:

Recv‑Q    Established: The count of bytes not copied by the user program connected to this socket. Listening: Since Kernel 2.6.18 this column contains the current syn backlog.

Send‑Q    Established: The count of bytes not acknowledged by the remote host. Listening: Since Kernel 2.6.18 this column contains the maximum size of the syn backlog.

For a listening socket, Recv‑Q shows the current SYN backlog (number of connections in the completed queue) and Send‑Q shows the maximum SYN backlog size. For an established TCP connection, Recv‑Q shows the amount of data still in the recv buffer, and Send‑Q shows the amount of data not yet acknowledged by the peer.

Example output of netstat -tnl and ss -tnl:

[root@xuexi ~]# netstat -tnl
Active Internet connections (only servers)
Proto Recv‑Q Send‑Q Local Address           Foreign Address         State
tcp       0      0 0.0.0.0:22              0.0.0.0:*               LISTEN
...

[root@xuexi ~]# ss -tnl
State      Recv‑Q Send‑Q               Local Address:Port      Peer Address:Port
LISTEN     0      128                     *:22                    *:*
...

Note that netstat does not display the maximum size of the completed queue, so ss is preferred for checking queue capacity.

2.3.2 Impact of SYN flood

If a client spoofs its source address, the server may repeatedly retransmit SYN+ACK without receiving an ACK, consuming CPU and buffer resources and potentially leading to a SYN flood attack. Mitigations include reducing queue lengths, limiting retransmissions, increasing timeouts, or using SYN cookies.

2.4 accept() function

The accept() function removes the first entry from the completed‑connection queue and returns a new socket descriptor (often called connfd) for subsequent data transfer, while the original listening socket continues to listen.

In a prefork model, each child process both listens and works; after accept() it handles the client on the new socket. In a worker/event model, a dedicated listening thread accepts connections and hands them to worker threads, improving concurrency.

If the completed‑connection queue is empty, accept() blocks unless the socket is set to non‑blocking mode, in which case it returns EWOULDBLOCK or EAGAIN. Event‑driven I/O (select/poll/epoll) or signal‑driven I/O can also be used.

Synchronously, a single thread handles one connection at a time; asynchronously, a thread can handle multiple connections concurrently.

2.5 send() and recv() functions

send()

copies data from the application buffer to the send buffer; recv() copies data from the recv buffer to the application buffer. They may block if buffers are full or empty, unless the socket is non‑blocking, in which case they return EWOULDBLOCK / EAGAIN. Monitoring with select(), poll(), or epoll is recommended.

2.6 close() and shutdown() functions

The generic close() function decrements the file‑descriptor reference count; the socket is fully closed only when the count reaches zero, after which the four‑way termination proceeds.

The shutdown() function actively terminates the connection, optionally disabling further reads, writes, or both, and also triggers the FIN exchange.

3. Address/Port reuse techniques

Normally, a given address‑port pair can be bound by only one socket. Linux provides SO_REUSEADDR and SO_REUSEPORT options to allow address and port reuse, enabling multiple sockets to listen on the same address‑port and distribute incoming connections via round‑robin.

Each reusable listening socket is called a “listener bucket”. In a worker or event model with multiple processes, each process can have its own listening thread on the same address‑port, reducing contention for the listening lock.

However, on a single‑CPU system the benefit may be limited, and excessive reuse can increase CPU overhead due to context switches. Proper CPU affinity and the number of reuse instances should be considered.

Source: http://www.cnblogs.com/f-ck-need-u/p/7623252.html