How Do GPUs Power Modern Rendering? A Deep Dive into Architecture and Optimization

GPU Architecture Overview

Modern GPUs consist of a large external global memory (DRAM) and many compute units called Stream Multiprocessors (SMs) . SMs are grouped into Graphics Processing Clusters (GPCs) and Texture/Processor Clusters (TPCs) . Different generations (e.g., Maxwell, Turing) vary in the number of GPCs, SMs per GPC, and TPCs per GPC.

Memory Hierarchy

Access latency (fastest → slowest):

Registers – 1 cycle

Shared memory – 1‑32 cycles

L1 cache – 1‑32 cycles

L2 cache – 32‑64 cycles

Texture/constant cache – 400‑600 cycles

Global memory (VRAM) – 400‑600 cycles

SM (Stream Multiprocessor) Internals

An SM contains the following functional blocks (Fermi‑class example):

PolyMorph Engine – polygon deformation and vertex fetching.

Vertex Fetch – reads indexed vertex data from global memory.

Tessellator – DX11‑style surface subdivision.

Stream Output – DX10 stream‑output support.

Viewport Transform – converts vertices to clip space.

Attribute Setup – interpolates vertex attributes for pixel processing.

Core (ALU) – 32 scalar cores per SM.

Warp Scheduler – issues a single instruction to a warp (32 threads).

Instruction Cache & Dispatch Units – feed decoded instructions to the cores.

Special Function Units (SFU) – execute expensive math (pow, sin, cos, log, etc.).

Load/Store (LD/ST) – move data between shared memory, registers and global memory.

Register File – per‑thread register storage.

L1 Cache – may be shared with shared memory or texture cache depending on architecture.

Uniform Cache – constant data cache.

Texture Units & Texture Cache – fetch texels; each unit can sample multiple texels per cycle.

Interconnect Network / Crossbar – routes data between GPCs and other blocks.

Unified Shader Architecture

Early GPUs had separate vertex and pixel shader pipelines, causing load imbalance. Modern GPUs use a unified shader model where each core can execute any shader stage, improving utilization and power efficiency.

SIMT and Warps

GPUs follow the Single Instruction, Multiple Threads (SIMT) model. A warp (32 threads) receives the same instruction from the Warp Scheduler. If a thread stalls (e.g., waiting for memory), the scheduler can switch to another ready warp.

Divergent control flow (if‑else) masks out threads that do not follow the taken path, wasting cycles.

Hardware‑Level Rendering Pipeline

The CPU builds draw calls and writes them into a FIFO PushBuffer . The driver copies the buffer into a RingBuffer that feeds the GPU front‑end. The Primitive Distributor assigns primitives to GPCs, which then rasterize, clip, cull, and perform early‑Z before invoking pixel shaders.

Render Architectures

Immediate Mode Rendering (IMR) – each draw call writes directly to the framebuffer, resulting in high bandwidth usage.

Tile‑Based Rendering (TBR) – the screen is divided into tiles; intermediate results stay on‑chip (L1/L2) and are flushed to the framebuffer only after the tile is complete, reducing bandwidth and power.

Tile‑Based Deferred Rendering (TBDR) – extends TBR with per‑pixel sorting (e.g., PowerVR HSR) to improve overdraw handling.

Key Optimization Techniques

Keep register usage low to maintain a high number of active warps; each additional register reduces the maximum warp count and hurts latency hiding.

Leverage SIMD and co‑issue to combine low‑dimensional operations into a single instruction, increasing ALU utilization.

Avoid heavy branching; divergent branches cause mask‑out behavior and waste cycles.

Minimize calls to SFU‑only functions (pow, sin, cos, log) because SFUs are limited.

Enable early‑Z whenever possible; avoid alpha‑test, alpha‑blend, or manual depth writes that disable early‑Z.

Batch draw calls and keep vertex counts modest on mobile; excessive draw calls increase on‑chip FrameData size and may overflow on‑chip memory.

Clear or discard render textures when they are no longer needed to free tile memory.

Shader Execution Details

Shaders run on SM cores. The compiler determines the number of registers each thread needs. The Register File provides those registers; the total number of registers per SM limits the number of concurrent threads.

Example: an SM with 32 768 registers and a shader requiring 256 registers per thread can host 128 threads, i.e., 4 warps (128 / 32).

When a warp encounters a long‑latency operation (e.g., memory load), the Warp Scheduler swaps to another ready warp, keeping the cores busy.

Register Vector Utilization

GPU ALUs operate on 4‑component vectors (SIMD). Using full vectors reduces instruction count. Example:

float4 c = a + b; // SIMD addition, one instruction

Without SIMD the same operation would require four scalar adds:

ADD c.x, a.x, b.x
ADD c.y, a.y, b.y
ADD c.z, a.z, b.z
ADD c.w, a.w, b.w

Shader code can pack multiple logical values into a single vector to save registers, e.g., Unity’s TRANSFORM_TEX macro packs scale (xy) and bias (zw) into one float4.

#define TRANSFORM_TEX(tex,name) (tex.xy * name##_ST.xy + name##_ST.zw)

Co‑Issue

When two low‑dimensional instructions are issued in the same cycle, the hardware can co‑issue them as a single instruction, increasing ALU throughput. This works only when operands are not read‑after‑write within the same cycle.

Control Flow on GPU vs CPU

CPU pipelines use branch prediction to guess the outcome of a conditional and continue fetching instructions speculatively. GPUs lack branch prediction; they instead use mask‑out execution. All threads in a warp execute the same instruction, but threads whose condition is false are masked out and do not perform the operation, though they still occupy the cycle.

Consequences:

Heavy if‑else or variable‑length loops waste cycles because only a subset of threads performs useful work.

Uniform control flow (same branch outcome for all threads) yields full utilization.

Tile‑Based Rendering Details

In TBR/TBDR, the screen is split into tiles (e.g., 16×16 pixels). Each tile’s color and depth buffers reside in on‑chip memory. The Work Distribution Crossbar (WDC) assigns tiles to GPCs. After all primitives for a tile are processed, the tile’s data is written back to the global framebuffer in a single burst, dramatically reducing memory bandwidth.

Early‑Z in Tile Architectures

Early‑Z tests are performed on‑chip, preventing overdraw before the tile is written out.

Alpha‑test, alpha‑blend, manual depth writes, or disabled depth testing break early‑Z, causing higher bandwidth and power usage.

Practical Guidance for Shader Authors

Limit register count to keep warp occupancy high.

Prefer vector operations and co‑issueable instruction patterns.

Avoid divergent branches; restructure algorithms to be data‑parallel.

Use early‑Z friendly techniques: render opaque geometry first, then alpha‑tested, then alpha‑blended objects.

On mobile, batch draw calls and reuse vertex buffers to keep FrameData size manageable.

Clear or discard unused render targets to free tile memory.

Summary

This summary captures the essential GPU hardware components (global memory, caches, SM internals), the unified shader execution model, SIMT/warp scheduling, and the differences between immediate‑mode and tile‑based rendering pipelines. It also outlines concrete optimization strategies—register budgeting, SIMD/co‑issue usage, early‑Z, and draw‑call batching—that are critical for achieving high performance on both desktop and mobile GPUs.