How Baidu Boosted CPU Utilization by Up to 80% with Offline Mixed‑Tenant Scheduling

1. What Is Offline Mixed‑Tenant?

Mixed‑tenant (混部) technology classifies workloads into online (latency‑sensitive, long‑running) and offline (batch, non‑latency‑sensitive) categories. By placing both on the same physical resources and using isolation and scheduling controls, resource utilization can be maximized while preserving service stability.

2. Why Is Resource Utilization Low?

Online clusters often run at ~20% CPU utilization because resources are over‑provisioned for peak traffic (the “tidal” effect) and redundant replicas are kept for disaster recovery. Additionally, data‑center planning separates online and offline farms, leading to uneven load distribution.

By pooling offline jobs into the online resource pool, Baidu can reclaim idle CPU cycles and raise overall utilization.

3. Baidu Cloud‑Native Mixed‑Tenant Details

With the rapid growth of the Kubernetes ecosystem, Baidu operates thousands of clusters that suffer from low utilization. Baidu’s solution adds a native‑offline mixed‑tenant layer that requires zero intrusion into existing K8s components.

3.1 How Is Resource Reuse Achieved?

Kubernetes traditionally allocates resources statically based on pod request values, leaving a large gap between request and actual usage. Baidu introduces a dynamic resource view for offline workloads: the offline view sees the host quota minus the resources already consumed by online pods.

Formulas (ideal case):

High‑priority (online) static allocation: ∑High request + ∑Medium request ≤ Host Quota Low‑priority (offline) available quota:

Low Quota = Host Quota – ∑High used – ∑Medium used

Offline jobs are scheduled onto this reclaimed quota, improving overall CPU usage.

3.2 Priority Management

Baidu extends the native K8s QoS model (Guaranteed, Burstable, BestEffort) with three priority levels: high and medium for online services, low for offline jobs. Each level can be further subdivided.

3.3 Isolation Mechanisms

CPU : Baidu uses cpuset binding to keep latency‑critical online pods on the same NUMA node, reducing cross‑node latency. NUMA‑aware scheduling further balances load across nodes.

Offline Scheduler : A dedicated scheduler runs after the online scheduler, ensuring online tasks always win resource contention.

Memory : A background page‑cache reclamation daemon runs per container, with separate high/low watermarks for online and offline workloads, prioritizing offline cache reclamation.

Network : Container‑level bandwidth limits and traffic tagging are enforced via Cgroup extensions.

eBPF‑Based Dynamic Policies : Custom eBPF programs are hot‑loaded from user space to apply fine‑grained isolation without kernel restarts, enabling rapid response to high‑sensitivity workloads.

3.4 High‑Performance Offline Scheduler

The offline scheduler can process up to 5,000 scheduling operations per second (≈5k ops) while throttling binding actions to 1,500 ops to protect etcd and the cluster.

3.5 Resource Profiling (画像)

Predictive resource profiling forecasts online usage for the next hour, allowing the scheduler to place offline jobs only when sufficient capacity is guaranteed, thus avoiding pre‑emptions and improving offline job stability.

Both online and offline schedulers benefit from profiling: online scheduling improves service availability by avoiding hotspots, while offline scheduling boosts cluster throughput and reduces job queue times.

4. Future Outlook

Baidu’s mixed‑tenant clusters now span hundreds of thousands of nodes, raising CPU utilization by 40‑80% and saving tens of thousands of servers. Future work includes deeper kernel programmability with eBPF, broader support for heterogeneous resources (e.g., GPUs), container‑VM fusion for high‑density nodes, and multi‑cloud mixed‑tenant strategies that incorporate spot instances for cost‑effective elasticity.