Resolving IP Address Exhaustion and Pod Limits in Amazon EKS with AWS‑CNI
This article explains why EKS clusters can run out of pod IP addresses or have low pod density per node, describes how AWS ENI and address‑prefix features affect IP allocation, and provides practical steps such as secondary CIDR allocation and subnet reservation to overcome these limits.
When running an Amazon EKS cluster you may encounter two common problems: the pool of IP addresses assigned to pods is exhausted, and each node can host only a few pods because of ENI limits.
Understanding how networking works inside a Kubernetes node is essential. The kubelet delegates container creation to the container runtime, connects containers to the CNI network, and mounts volumes via CSI. Each pod gets its own Linux network namespace and is attached to a bridge created by the CNI plugin.
Many CNI implementations use a single bridge, but AWS‑CNI is different: each EC2 instance can have multiple Elastic Network Interfaces (ENIs), each with a limited number of IP addresses. For example, an m5.large instance can attach up to three ENIs, each providing ten private IPs, allowing a maximum of 27 pods (after reserving one IP per ENI).
A recent AWS change introduces address prefixes , allowing ENIs to allocate blocks of IP addresses (slots) instead of single IPs. By default a slot contains 16 IPs; with ten slots an ENI can provide 160 IPs. Using the same m5.large instance, three ENIs with ten slots each yield up to 432 pod IPs, dramatically increasing the pod limit.
However, this increase brings new challenges: slots consume IP space faster and can cause fragmentation. The Kubernetes scheduler still prefers spreading pods across nodes, so a cluster with five nodes and a five‑replica deployment may allocate 75 IPs while using only five.
You can assign a secondary CIDR to the EKS VPC.
You can reserve a portion of the subnet IP space exclusively for ENI slots.
These steps help mitigate IP exhaustion and improve pod density when using AWS‑CNI.
System Architect Go
Programming, architecture, application development, message queues, middleware, databases, containerization, big data, image processing, machine learning, AI, personal growth.
How this landed with the community
Was this worth your time?
0 Comments
Thoughtful readers leave field notes, pushback, and hard-won operational detail here.