Why Nvidia’s Rubin CPX GPU Could Revolutionize Long-Context AI Inference

Nvidia Announces Rubin CPX GPU for Long‑Context Inference

On 9 September 2025 Nvidia introduced the Rubin CPX GPU, built on the Rubin architecture to boost token‑per‑second rates during the inference phase of large language models. The chip delivers 30 petaFLOPS at NVFP4 precision and 128 GB of GDDR7 memory, deliberately avoiding costly HBM.

The design separates the context‑processing (prefill) stage, which is compute‑intensive, from the generation stage, which is memory‑bandwidth‑limited. By off‑loading the compute‑heavy context stage to Rubin CPX, the system reduces the “compute wall” bottleneck while keeping the “memory wall” manageable with GDDR7.

Rubin CPX works together with Nvidia Vera CPUs and standard Rubin GPUs to form a complete inference pipeline. The new hardware is integrated into the third‑generation Oberon‑based Vera Rubin (VR) server family, expanding the VR NVL144 chassis into three configurations:

VR NVL144 (no CPX) – 18 compute trays, each with 4 R200 GPUs (counted as 8 dies) and 2 Vera CPUs.

VR NVL144 CPX – each compute tray adds 8 Rubin CPX GPUs (ratio 1 Vera CPU : 2 Rubin GPU : 4 Rubin CPX GPU).

VR NVL144 + VR CPX (dual‑rack) – a scale‑out solution that connects a separate VR CPX rack via InfiniBand or Ethernet, allowing flexible allocation of prefill and decode resources.

The VR platform also upgrades interconnects: NVLink5 provides 18 ports per GPU with four differential pairs per port (≈200 Gbps per lane), totaling 1.8 TB/s bidirectional bandwidth. Future NVLink6 could double per‑GPU bandwidth to 3.6 TB/s by increasing differential pairs, albeit with higher copper‑cable requirements.

By decoupling context processing from generation, Rubin CPX aims to improve inference efficiency and return on investment. Nvidia estimates that each $100 M of hardware can generate roughly $5 B in token revenue through high‑throughput long‑context workloads.

Overall, the Rubin CPX GPU and the third‑generation Oberon servers push power‑density limits, necessitate upgraded power delivery and enhanced liquid‑cooling solutions.