MySQL OOM Caused by Unused Hugepages on a CentOS 7.1 Host
The article explains how a MySQL instance on a CentOS 7.1 VM repeatedly suffered connection failures due to out‑of‑memory (OOM) events triggered by a mis‑configured 40 GB hugepage allocation that remained reserved for MySQL even after the large‑page setting was disabled.
The author, a DBA team member, reports intermittent MySQL connection failures that occur a few times per day. Investigation reveals the issue is related to the host's traditional hugepage configuration.
Environment: MySQL 5.6.25 running on a VMware virtual machine with CentOS 7.1, 32 CPU cores, 64 GB RAM, and innodb_buffer_pool_size=48G .
Investigation steps:
Checked MySQL uptime and found recent restarts.
Monitored memory and CPU usage: free showed 61 GB used, top showed MySQL RES around 20 GB, and numa indicated low free memory per node.
Examined system logs and discovered OOM events matching the reported outages.
Observed that during OOM, MySQL's RSS (aron‑rss) was about 22 GB.
Reviewed /proc/meminfo and confirmed the values matched those from free and top .
Found that the host had 20 000 hugepages configured (2 MB each), reserving roughly 40 GB of memory.
Although the hugepages were allocated, they were not actively used because MySQL's large‑page option had been disabled. The hugepage reservation still consumed memory, leaving only about 22 GB for MySQL and causing the system to run out of memory under load.
Further checks showed that the hugetlb_shm_group file pointed to the MySQL user group (uid=27), confirming the hugepages were intended for MySQL.
MySQL configuration indeed contained a large‑page setting, but it was turned off. The mismatch between the host's hugepage reservation and MySQL's actual usage led to OOM and the observed connection interruptions.
Conclusion: The host retained 40 GB of hugepages reserved for MySQL even after the large‑page option was disabled, causing memory exhaustion and OOM kills of the MySQL process.
Follow‑up actions: Removing the unnecessary hugepage settings freed the reserved memory, preventing further OOM events.
Recommendations:
MySQL hosts generally do not need hugepages; let MySQL manage its buffer pool internally.
Before deployment, verify that no unused hugepages are allocated on the host to avoid hidden memory consumption.
Reference links:
https://docs.oracle.com/database/121/UNXAR/appi_vlm.htm#UNXAR391
https://kerneltalks.com/services/what-is-huge-pages-in-linux/
https://dev.mysql.com/doc/refman/5.6/en/large-page-support.html
shell> /proc/sys/vm/hugetlb_shm_group
27
shell> id 27
uid=27(mysql) gid=27(mysql) groups=27(mysql)Aikesheng Open Source Community
The Aikesheng Open Source Community provides stable, enterprise‑grade MySQL open‑source tools and services, releases a premium open‑source component each year (1024), and continuously operates and maintains them.
How this landed with the community
Was this worth your time?
0 Comments
Thoughtful readers leave field notes, pushback, and hard-won operational detail here.