Redesigning Agent Infrastructure to Support 40K+ Collaborative Agents and Multi‑Model Teams

At the recent Open Computing Technology Conference (OCTS26) in Beijing, Inspur unveiled a series of AI infrastructure products, including the industry’s first CPU‑native liquid‑cooled rack server and the Yuan‑Brain SD200 supernode, aimed at the emerging agent era.

Market forecasts indicate rapid growth: IDC predicts a 139% CAGR for the global agent market from 2025‑2030, while Gartner expects 40% of enterprise applications to integrate agents this year, rising to over 30% by 2028. This shift drives a fundamental redesign of compute resources.

Software trends show large models such as Kimi, DeepSeek, and GLM evolving toward native agent capabilities, and enterprise frameworks like ChatGPT Work and Workbuddy enabling hundreds of agents to cooperate on complex tasks. Consequently, token consumption grows exponentially, requiring hundreds of CPU cores per request.

Analysis reveals that each agent operates as a small CPU sandbox handling logic management, resource scheduling, and system coordination—tasks ill‑suited to GPU parallel matrix operations. Studies show CPU‑bound stages can account for up to 90.6% of end‑to‑end latency, elevating CPU importance in agent workloads.

To meet these demands, data‑center CPU capacity must expand dramatically. Traditional AI servers have a CPU‑to‑GPU ratio of 1:4‑1:8, but agent‑centric systems need dense CPU clusters alongside GPUs. Leading internet firms are already allocating new CPU server purchases to agent services.

The new liquid‑cooled rack server redefines CPU computing: a single cabinet supports up to 384 heterogeneous CPUs, enabling over 40 000 agents to run concurrently. Its power draw can reach megawatt levels, far exceeding conventional 40‑50 kW racks.

Key technical breakthroughs include:

CPU compute system reconstruction : a 0.5U thin node design packs 16 CPUs in 2U space, breaking the traditional CPU‑GPU split.

Standardized compute modules : an open OCM liquid‑cooling architecture that seamlessly supports X86, ARM, and other CPU families, offering high performance under load and large memory/bandwidth for long‑context scenarios.

Full‑component liquid cooling : decouples and planarizes cooling for CPU, memory, NICs, and optical modules, eliminating hoses, cables, and fans, improving reliability and energy efficiency.

The cabinet also adopts 800 V high‑voltage DC power, preparing for megawatt‑class deployments where 380 V would require impractically thick copper cables.

The upgraded Yuan‑Brain SD200 supernode delivers a single‑token generation time of 4.77 ms on the 4‑trillion‑parameter Kimi K2.6 model—a 35% reduction versus its previous version—thanks to hardware fabric optimizations and software techniques such as multi‑token prediction and JIT inference.

Multi‑model fusion is highlighted as a core strategy: combining outputs from several advanced models (Kimi, DeepSeek, GLM, MiniMax) improves robustness. Benchmarks show fusion models achieving 53.9% optimal scores on the DRACO benchmark, and 97.2% / 90.8% on AIME 2026 math reasoning and GPQA high‑difficulty QA, respectively, surpassing single‑model baselines.

System‑level capabilities include supporting a single model with up to 4 trillion parameters, parallel deployment of multiple such models, and integration with the Yuan‑Brain EPAI platform for unified API calls, automatic task distribution, and result fusion.

For enterprise customers, an SD200 Enterprise edition offers a 16‑card scale‑up domain with unified addressing and sub‑40% token latency, TB‑scale unified memory, and the ability to run trillion‑parameter models for deep‑context understanding and complex logic reasoning, lowering the barrier to high‑performance agent deployment.

Looking ahead, the second half of 2026 is expected to see large‑scale rollout of supernode solutions, driving down per‑token costs and enabling AI to permeate entire enterprise workflows. The evolution from large models to agents signals a broader infrastructure transformation, where token generation becomes a finely partitioned pipeline—prefill and decode stages split and matched to the most suitable chip architectures for optimal efficiency.