Explore CUDA Toolkit 13.1: CUDA Tile, Green Contexts, and Performance Boosts

Release Overview

NVIDIA announced the official release of CUDA Toolkit 13.1, calling it the largest update in two decades, bringing a suite of new features aimed at improving GPU programming for AI and high‑performance computing.

Key New Features

CUDA Tile : a tile‑based programming model that abstracts specialized hardware such as Tensor Cores, allowing developers to write algorithms at a higher level than traditional SIMT.

Green Contexts : lightweight, concurrently schedulable contexts exposed via the Runtime API, enabling fine‑grained GPU resource partitioning.

cuBLAS Precision Simulation : double‑ and single‑precision simulation capabilities.

New CUDA Programming Guide : a completely rewritten guide for beginners and advanced users.

CUDA Tile Details

CUDA Tile is the core update of 13.1. It introduces a tile‑based programming model that lets developers specify data blocks (tiles) instead of individual threads. The compiler and runtime automatically map tiles to the optimal thread configuration, abstracting away low‑level details of Tensor Cores and ensuring compatibility with future GPU architectures.

Two components support Tile programming:

CUDA Tile IR : a new virtual instruction set architecture for NVIDIA GPUs.

cuTile Python : a domain‑specific language for writing tile‑based kernels in Python.

Current limitations: Tile support is limited to NVIDIA Blackwell GPUs (compute capability 10.x and 12.x). Future releases will broaden architecture support, and NVIDIA plans to add a C++ implementation.

Why Tile Programming?

Traditional SIMT requires developers to manage fine‑grained thread execution, which becomes complex when targeting multiple GPU architectures. Tile programming raises the abstraction level, letting developers focus on high‑level mathematical operations while the compiler handles optimal thread mapping. This is especially beneficial for AI workloads that heavily use tensors and specialized hardware like Tensor Cores.

Green Contexts and Split API

Green Contexts provide lightweight execution environments that can be allocated a specific number of Streaming Multiprocessors (SMs). They enable priority scheduling for latency‑sensitive code by isolating resources. The new split() API lets developers create SM partitions with a single call, reducing false dependencies between contexts.

Typical usage example:

nvcc -fdevice-sanitize=memcheck -o myapp myapp.cu

compute-sanitizer --tool memcheck myapp

CUDA Multi‑Process Service (MPS) Enhancements

Memory Locality Optimization Partition (MLOPart) : creates multiple logical devices on a single GPU, each with a subset of SMs and memory, currently supporting Blackwell B200 and B300 series.

Static SM Partitioning : provides deterministic SM allocation for MPS clients on Ampere (compute capability 8.0) and newer GPUs, using the -S or --static-partitioning flag.

Developer Tools Updates

Nsight Compute now includes a “Result Type” column to differentiate Tile kernels from SIMT kernels and a “Tile Statistics” section summarizing tile dimensions and pipeline utilization. Source pages can map metrics back to high‑level cuTile kernel source.

Math Library Improvements

cuBLAS : experimental API for grouped GEMM on Blackwell GPUs, supporting FP8, BF16, and FP16 with up to 4× speed‑up in MoE‑style workloads.

cuSPARSE : new SpMVOp API with performance gains, supporting CSR format, 32‑bit indices, double precision, and custom suffixes.

cuFFT : new device API for generating cuFFTDx code blocks, improving performance for FFT operations.

Performance benchmarks show significant acceleration on Blackwell GPUs for batched SYEV, GEEV, and GEMM operations, with speed‑ups ranging from 1.5× to 2× depending on matrix size.

CUDA Core Compute Library (CCCL) Enhancements

CCCL 3.1 adds two deterministic floating‑point options, allowing developers to trade determinism for performance. New overloads let many CUB algorithms skip the temporary storage query/allocate/free pattern, simplifying API usage.

References

CUDA programming guide – Green Contexts: https://docs.nvidia.com/cuda/cuda-programming-guide/04-special-topics/green-contexts.html

CUDA Tile resource page: https://developer.nvidia.com/cuda/tile

CUDA Toolkit download: https://developer.nvidia.com/cuda-downloads