Investigating the NUMA Trap with a Large Redis Instance on a Dual‑Node Server

Our infrastructure team provides a cloud Redis platform where users can request arbitrary memory sizes for instances. This raised the question: on a 64 GB physical host, can a single Redis instance be allocated 50 GB without adverse effects?

Research suggested that a single‑process memory allocation should not exceed the memory of a single NUMA node; otherwise the kernel may resort to swapping on that node, leading to severe performance degradation for memory‑intensive workloads such as Redis, MySQL, or MongoDB.

To verify this, we first examined the server’s NUMA topology using numactl --hardware , which revealed two nodes (node 0 and node 1), each with 12 CPUs and roughly 32 GB of RAM. The zone_reclaim_mode was set to 1 , meaning memory reclamation occurs only within the local node.

Step 1 – Baseline Memory Check

# top
Mem: 65961428k total, 26748124k used, 39213304k free
Swap: 8388600k total, 0k used, 8388600k free
# cat /proc/zoneinfo | grep "pages free"
Node 0, zone   Normal
  pages free     4651908
Node 1, zone   Normal
  pages free     4773314

We then launched a Redis server with its maxmemory set to 50 GB, exceeding the size of a single node. After loading data, the process consumed about 46 GB of resident memory, but memory was still allocated across both nodes, and no swap was observed because the kernel distributed allocations.

Step 2 – Enforcing NUMA Affinity

By binding the Redis process to node 0 using numactl --cpunodebind=0 --membind=0 , we forced all memory requests to come from the same node.

numactl --cpunodebind=0 --membind=0 /path/to/redis-server /path/to/redis.conf

Monitoring with top showed the process staying on node 0, and the free pages on that node quickly dropped to a few thousand. Swap usage rose sharply, and the Redis process was eventually killed by the OOM killer, confirming the classic “NUMA trap”.

Conclusion

The experiment proves that a NUMA trap exists: when a process is bound to a single node and requests more memory than that node can provide, the kernel resorts to swapping on that node, dramatically degrading performance. In typical deployments without explicit NUMA binding, Redis can safely use the total physical memory of the host, but allocating an entire machine’s memory to a single Redis instance is discouraged because it wastes CPU cores and reduces overall system efficiency.

For workloads demanding extreme performance, manually binding CPU and memory to the same NUMA node can reduce memory‑access latency and improve QPS, but developers must be aware of the risk of hitting the NUMA trap.