Cloud Network Reconstruction for AI Scale: 7 SIGCOMM/NSDI Papers from Tencent Cloud

Introduction

As large-model training enters the 10,000-GPU cluster era, AI compute expansion is breaking the design boundaries of traditional cloud networks. Single-node bandwidth is jumping from hundreds of Gbps to Tbps; RDMA is replacing TCP as the core transport for GPU interconnects; heterogeneous hardware (DPUs, programmable switches, FPGAs) is being deployed at scale in cloud and AI gateways; and network fault localization and fast recovery at massive scale directly constrain compute availability. Tencent Cloud addresses these infrastructure challenges with seven papers accepted at CCF-A conferences (five at SIGCOMM 2025, one at NSDI 2026, one at NSDI 2027).

Reducing AI Network Cost Anxiety: Low Overhead, High Performance, Native Management

DistDPU: Disaggregated DPU Architecture

Traditional deployments separate three networks: high-bandwidth for training, business network for deployment/storage, and virtualization for tenants. AI scenarios demand unified delivery. The common approach adds DPUs with external controllers, but each DPU carries underutilized management compute and memory. Training traffic is high-volume and stable, while control-plane overhead stays constant, leading to wasted resources.

DistDPU splits the DPU into an Orchestration Module (management) and Execution Modules (high-speed forwarding). Management is centralized; adding bandwidth only adds Execution Modules. The architecture is interconnect-agnostic (PCIe, NVLink). It supports up to 3.6 Tbps per node, reduces per-node hardware cost by 77.1% and power by 59.3% versus multi-DPU stacking, and has run in production for over 20 months serving 10,000+ GPUs.

Pegasus: Bare-Metal AI Cloud Data Center Network

Pegasus refines the Execution Module side: one DPU virtualizes multiple RNICs, reducing redundant DPU configuration. DPU and RNICs connect via the server's native PCIe; control, flow-table deployment, and telemetry stay in-server, eliminating a dedicated management network. Two-level flow tables on DPU and RNIC handle address translation and tenant isolation, achieving multi-tenant isolation without performance loss. Per-packet load balancing and credit-based congestion control run on RNIC hardware. As of February 2026, the network covers 8,000 GPUs, yielding 7.5% training speedup for 70B dense models, 4.8% for 7B MoE, 53.2% for LLaMA 7B, 3.7% inference throughput gain, and 7.2% reduction in average request completion time.

SkyRDMA: Fully Offloaded Cloud RDMA Virtualization

RDMA bypasses the OS kernel for direct data transfer, but virtualization in multi-tenant clouds degrades performance and causes interference. SkyRDMA moves the entire virtualization layer to the NIC. Multi-root PCIe socket-direct RNICs let a single NIC connect multiple CPU sockets, avoiding cross-socket traffic detours. The NIC carries mixed RDMA, VPC, cloud-disk, and virtualization traffic with hierarchical QoS at both the NIC port and PCIe transport. The vRDMA driver splits into a lightweight tenant-side front end and a heavy back end on the NIC's embedded SoC; back-end upgrades are transparent to tenants. Experiments show near-bare-metal performance; QoS isolation reduces tail latency under background interference by up to 6.2×; control-plane hot updates take 2.09–3.27 ms with zero downtime; existing RDMA applications migrate without code changes.

Upgrading Cloud Network Core Performance: Handling Traffic Surge and Density Explosion

XFir: Accelerating New-Flow Setup on Host Servers

Traditional cloud workloads (AI agents, e-commerce search/ads/recommendation, gaming) increase traffic, while CPU core counts double yearly, inflating per-server session counts. Short-connection-heavy workloads (flash sales, game matches, microservice calls) amplify new-flow setup pressure; some large customers require hundreds of thousands to millions of new connections per second per server. The DPU's software new-flow path becomes the bottleneck.

XFir offloads new-flow setup to the DPU's built-in cloud network co-processor, redesigning the data path and lookup structures while retaining software programmability for policy agility. It achieves >776k CPS (connections per second) per server, 11.7 µs slow-path latency, a 4.8× CPS improvement and 69.2% latency reduction over the prior Fornax design, using only one DPU per host.

Dorado: Scaling SmartNIC Session Tables on Commodity DDR

SmartNIC session tables store connection state for hardware acceleration. Observations across ~12,000 hosts showed 12.1% of hosts exceed 1 million entries, peaking at 16 million—an order of magnitude above typical industry assumptions. Using expensive SRAM or HBM for these tables is cost-prohibitive at million-server scale.

Dorado places the entire session table in cheap commodity DDR, redesigning the table structure, processing pipeline, and memory-access scheduling to match high-speed memory performance. With fewer hardware resources, it delivers 33% higher packet processing rate, 80% lower memory cost, supports 16 million entries at 50 Mpps line rate, and has been deployed on millions of servers for over three years.

Minute-Level Fault Localization, Sub-Second Recovery: Building a "Super Cloud"

CubeTrace: Microscopic Network Tracing for Heterogeneous Cloud Gateways

Cloud gateways combine software servers, programmable switches, and FPGAs; failures require cross-device diagnosis. CubeTrace standardizes processing stages across heterogeneous hardware into uniform trace units, recording every hop a flow traverses. It pinpoints packet loss and latency to specific functional stages. Running over 300 trace units in production gateways with <1% memory overhead, it reduces fault localization time from hours/days to minutes.

Harp: Efficient Failure Detection and Rerouting for VPC Availability

Harp leverages switch multipath forwarding to maintain a set of viable physical paths per host pair with continuous probing. On failure, traffic switches to a healthy path in sub-seconds. The solution is pure software, independent of specific hardware or transport protocols. Harp cuts VPC network interruption time by 78.71% to 99.97% and has been deployed in Tencent Cloud for over two years.

Conclusion

These seven papers collectively address the triad of cost, performance, and fault recovery for AI-scale cloud networks. The common thread: offload high-frequency forwarding to NICs and DPUs, keep control and diagnosis in software, enabling simultaneous scale-out, cost control, and stability improvements. This work reshapes the cost structure of compute, training/inference efficiency, and failure impact. As compute scale grows, how cloud networks are reorganized becomes a new competitive variable among cloud providers.

Paper links:

DistDPU: https://dl.acm.org/doi/10.1145/3789240.3829161 Pegasus: https://dl.acm.org/doi/10.1145/3789240.3829186 XFir: https://dl.acm.org/doi/10.1145/3789240.3829131 Dorado: https://dl.acm.org/doi/10.1145/3789240.3829121 CubeTrace: https://dl.acm.org/doi/10.1145/3789240.3829165 Harp:

https://www.usenix.org/conference/nsdi26/presentation/hu-jiayu