Analyzing Excessive TIME_WAIT Connections on a MySQL Server and Effective Mitigation

Recently a customer’s MySQL server displayed an unusually high number of TCP connections stuck in the TIME_WAIT state, consistently ranging between 3000 and 4000 connections as shown in monitoring graphs.

Inspection of the operating system’s TCP table confirmed that all TIME_WAIT connections originated from the application server (IP 234) to the MySQL server (IP 161) on the database port 4125. This raised the question why the server side, rather than the client side, held so many TIME_WAIT sockets.

Initial hypotheses such as MySQL closing idle connections due to wait_timeout , network anomalies, or client crashes were ruled out because MySQL connection counts remained low, error logs showed no “Aborted connection” or packet‑reading errors, and no abnormal shutdown messages were observed.

A temporary change of the kernel parameter net.ipv4.tcp_tw_recycle from 0 to 1 caused the TIME_WAIT count to drop dramatically, confirming that kernel TCP settings affect the symptom. However, documentation warns that tcp_tw_recycle is unsafe, so the parameter was reverted to 0.

To understand the root cause, a one‑way packet capture was performed on the MySQL server using tcpdump and analyzed with Wireshark. The capture revealed a large volume of FIN and SYN packets, indicating that the application server frequently opened short‑lived connections, performed quick queries, and then closed them repeatedly.

Further analysis showed two possible TCP termination sequences: (1) the client initiates the close, the server acknowledges, and the client sends the final FIN; (2) the client initiates the close, but the server actively terminates the connection. Both scenarios were observed.

The conclusion is that when many short connections are created by the client, the MySQL server’s OS can accumulate a substantial number of TIME_WAIT sockets because the server sometimes performs the active close. The recommended solution is to replace numerous short connections with a smaller number of persistent (long‑lived) connections on the application side.

The MySQL Internals Manual also documents the server’s behavior during client session termination, which aligns with the packet‑capture findings.