GPU Architecture in the AI Era: From Specific‑Domain Designs to 3D/AI Fusion

In June 2018, John Hennessy and David Patterson highlighted three insights about computer architecture: software advances drive architectural innovation, evolving hardware/software interfaces create opportunities, and market demand resolves architectural debates. The author adds a fourth point—that the winning architecture in competition will further stimulate software evolution.

Following the Hennessy/Patterson talk, the market has largely validated the third insight, selecting the GPU as the dominant architecture for the AI revolution. This article explores whether GPUs can continue to thrive in the emerging golden age of computer architecture.

01 Specific‑Domain Architecture – Hennessy and Patterson introduced the concept of specific‑domain architectures (DSA). GPUs, originally DSA for 3D graphics, have become the de‑facto “CPU” for AI because they accelerate neural‑network workloads. The push toward general‑purpose GPU (GPGPU) reflects attempts to reuse GPU resources beyond 3D rendering.

AI‑focused DSAs are now challenging GPGPU, as AI workloads demand tensor acceleration that is unnecessary for traditional 3D pipelines, leading to a tension between AI‑centric and 3D‑centric GPU designs.

02 GPU Hardware/Software Interface – The GPU’s success stems from its programmable hardware/software interface, which enables developers to control individual work items (vertices, pixels) without fixed‑function loops. This flexibility supports both graphics and AI workloads and has driven the integration of a unified compute resource pool that can be shared across stages, improving load balancing and enabling GPGPU.

The interface also allows the addition of specialized co‑processors (e.g., texture units, special function units) that are allocated based on average usage, enhancing throughput without altering the core hardware.

03 3D Tensor Acceleration – Traditional 3D rendering pipelines are dominated by post‑processing (≈90% of render time). AI‑driven techniques such as NVIDIA’s DLSS 2.0, AI‑based denoising for ray tracing, and AI‑enhanced anti‑aliasing and super‑resolution illustrate how tensor acceleration can improve 3D performance. If AI‑based post‑processing becomes mainstream, tensor acceleration could become the primary driver for GPU usage in 3D, reducing the need for separate AI and 3D DSAs.

04 3D/AI Fusion – The article questions whether fixed‑function hardware for 3D rendering should be removed when GPUs are used for AI. Neural rendering, which treats the 3D scene as a learnable neural representation (e.g., NeRF, GRAF), blurs the line between graphics and AI. This approach integrates differentiable rendering into the training loop, making gradient computation a part of inference.

Digital twins further demand high‑fidelity, mutable virtual worlds, pushing GPUs toward fully differentiable pipelines that can serve both AI training and realistic rendering.

05 Conclusion – In the AI‑driven paradigm, 3D rendering becomes an essential component of AI training loops, and gradient descent extends from cloud‑side training to on‑device inference. The convergence of neural and traditional rendering will rely on GPUs’ dual‑personality, ultimately cementing GPUs as the preferred architecture for the next era of computer systems.