Understanding MySQL Slave Net Timeout, Master Heartbeat Period, and Their Impact on Relay Log Accumulation
The article analyzes a MySQL replication issue where a low slave_net_timeout combined with a larger MASTER_HEARTBEAT_PERIOD causes frequent zombie dump threads, leading to massive small relay‑log files, and provides detailed explanations, code references, and step‑by‑step remediation procedures.
This article presents a real‑world MySQL replication case in which a large number of tiny relay‑log files (about 2,600) were generated due to mismatched timeout settings between the slave and the master.
Case description : The slave’s slave_net_timeout was set to 10 seconds while the master’s MASTER_HEARTBEAT_PERIOD defaulted to 30 seconds. When the master sent no events, the slave’s I/O thread timed out, disconnected, and repeatedly re‑connected, creating new relay logs that could not be cleaned up.
Parameter analysis : slave_net_timeout defines the idle timeout for the slave I/O thread; since MySQL 5.7.7 it defaults to 60 s (previously 3600 s). If MASTER_HEARTBEAT_PERIOD is not explicitly set, it is derived as slave_net_timeout / 2 . The heartbeat ensures a periodic event is sent from master to slave to keep the connection alive.
Root cause : Three conditions trigger the issue: (1) MASTER_HEARTBEAT_PERIOD > slave_net_timeout ; (2) the master is idle, producing no events; (3) existing replication lag. Under these conditions the master kills the “zombie” dump thread, the slave reconnects, and a new relay log is created each time.
Solution : Set slave_net_timeout to at least twice the value of MASTER_HEARTBEAT_PERIOD (or adjust the heartbeat period accordingly) and restart the replication threads. Example commands:
set global slave_net_timeout=10; stop slave sql_thread; stop slave; start slave io_thread;After adjusting the parameters, the relay‑log accumulation stops.
Implementation details : The article walks through the source‑code paths where slave_net_timeout is applied (via mysql_options and mysql_real_connect ), how MASTER_HEARTBEAT_PERIOD is set during CHANGE MASTER , and how the dump thread uses the heartbeat to send periodic events. Key snippets include:
case MYSQL_OPT_CONNECT_TIMEOUT:
mysql->options.connect_timeout = *(uint*)arg;
break; char query_format[] = "SET @master_heartbeat_period= %s";
char query[sizeof(query_format) - 2 + sizeof(llbuf)]; if (mi->heartbeat_period != 0.0) {
set_timespec_nsec(&ts, mi->heartbeat_period);
ret = mysql_bin_log.wait_for_update_bin_log(m_thd, &ts);
if (ret == ETIMEDOUT) send_heartbeat_event(log_pos);
}The article also provides a simulation script that reproduces the problem by stopping the slave SQL thread, setting a low slave_net_timeout , and observing the generation of relay logs every ~10 seconds.
Finally, a flowchart from the author’s book “Deep Understanding of MySQL Master‑Slave Principles” is shown to illustrate the dump‑thread lifecycle.
Aikesheng Open Source Community
The Aikesheng Open Source Community provides stable, enterprise‑grade MySQL open‑source tools and services, releases a premium open‑source component each year (1024), and continuously operates and maintains them.
How this landed with the community
Was this worth your time?
0 Comments
Thoughtful readers leave field notes, pushback, and hard-won operational detail here.