Diagnosing MySQL Memory Spikes Using perf and mmap Monitoring
This tutorial demonstrates how to reproduce a MySQL memory‑spike bug, monitor the process with Linux perf to capture mmap system calls, and analyze the resulting perf.out to identify which SQL statements trigger large memory allocations, while also discussing the method’s limitations.
In this experiment we reproduce a known MySQL bug (ID 99382) that causes a sudden memory increase, then use Linux perf to trace mmap system calls and collect a perf.data file.
We first create a test database, then run the provided SQL script that reproduces the memory‑spike; MySQL memory usage rises from about 181 MB upward.
During script execution we simultaneously run a perf command to record mmap events, generating a perf.data file that is later converted to a readable perf.out .
Tip: MySQL does not request memory directly from Linux; it asks glibc, which maintains its own pool and calls mmap (or brk ) when the pool is exhausted. Monitoring mmap therefore reveals when glibc needs additional memory, indirectly showing MySQL’s memory pressure.
Analyzing perf.out shows each mmap call with three key pieces of information:
Thread ID – identifies which thread requested the allocation, allowing us to map it to a specific SQL statement.
Allocation size – the amount of memory requested.
Stack trace – the call stack at the moment of allocation.
Using this data we determined that thread 29735 allocated memory while executing a CREATE VIEW operation.
The method works well for sudden memory spikes but has limitations: it only observes the second stage (glibc requesting memory from Linux) and may miss the first stage (glibc’s internal pool pressure) in cases of gradual memory growth, potentially leading to incorrect attribution.
When applying this technique, be aware of these constraints and consider complementary monitoring for slow‑growth scenarios.
Aikesheng Open Source Community
The Aikesheng Open Source Community provides stable, enterprise‑grade MySQL open‑source tools and services, releases a premium open‑source component each year (1024), and continuously operates and maintains them.
How this landed with the community
Was this worth your time?
0 Comments
Thoughtful readers leave field notes, pushback, and hard-won operational detail here.