How to Overcome CPU Throttling and NUMA Bottlenecks in Cloud‑Native Containers

Background

In the cloud‑native era, applications run as containers on shared host machines. As hardware improves, container density increases, leading to severe CPU contention, cross‑NUMA memory accesses, and degraded application performance. Proper host‑CPU allocation is essential for optimal service quality.

Kubelet CPU Allocation Strategies

Kubernetes uses request and limit to describe container CPU needs. The scheduler uses request to place Pods, while Kubelet enforces limit at runtime. By default Kubelet relies on the Linux CFS bandwidth controller (CFS) to cap CPU usage. For multi‑core nodes, the static policy lets Guaranteed‑QoS Pods exclusively bind to CPU cores, avoiding context‑switch overhead.

CPU Throttle Problem

The CFS controller uses cfs_period (typically 100 ms) and cfs_quota (e.g., a limit of 2 CPU sets cfs_quota=200ms). When a container exceeds its quota, the kernel throttles it, which is reflected in the CPU Throttle Rate metric. Short‑interval monitoring (100 ms) often reveals spikes that are invisible in 1‑second averages, causing latency spikes and RT jitter.

An example shows a container with CPU Limit=2 receiving four requests within one CFS period. Although overall utilization stays low, the burst consumes the entire 200 ms quota, forcing later threads to wait for the next period and elongating response time.

NUMA Topology Impact

On NUMA machines, CPUs and memory are divided into sockets. Accessing remote memory incurs higher latency. Blindly allocating CPUs across sockets harms latency‑sensitive workloads. Binding a container to a single socket (single‑numa‑node policy) improves cache locality and reduces cross‑socket traffic.

CPU Burst Mechanism

Alibaba Cloud contributed the CPU Burst kernel feature. When a container's real CPU usage is below its cfs_quota, unused time is stored in cfs_burst. If the container later needs more CPU, the kernel allows consumption of the stored burst time, effectively bypassing the hard limit.

ACK (Alibaba Cloud Container Service) fully supports CPU Burst. For kernels without native support, ACK monitors throttle status and dynamically raises the container's limit to emulate the same effect.

Benchmarks with Apache HTTP Server show that enabling CPU Burst reduces the p99 response time and eliminates CPU throttling while keeping overall utilization stable.

Topology‑Aware Scheduling in ACK

ACK extends the Kubernetes scheduling framework with a topology‑aware scheduler and flexible binding policies. It works for all QoS classes, can be enabled per‑Pod, and selects the optimal node‑CPU topology across the cluster.

Evaluations on Intel (104‑core) and AMD (256‑core) servers demonstrate performance gains of 22%–43% for CPU‑intensive, memory‑intensive workloads when topology‑aware scheduling is applied.

Conclusion

CPU Burst eliminates hard throttling for bursty workloads, while topology‑aware scheduling reduces cross‑NUMA latency and improves cache affinity. Neither technique is a universal silver bullet; their effectiveness depends on workload characteristics, QoS class, and overall cluster density. Combining both can yield the best results for latency‑sensitive, CPU‑bound applications.