How to Enable GPU Acceleration in Docker on Linux

Prerequisites

To run deep‑learning workloads on a Linux server with a GPU inside Docker, the container must have access to the host GPU driver and CUDA libraries.

1. Install the NVIDIA driver on the host

Download the driver installer from the NVIDIA website (any Linux distribution is supported). Ensure the kernel headers and a C compiler are present before running the installer: sudo apt install -y gcc kernel-dev Download and execute the driver package (example for version 450.80.02):

wget https://www.nvidia.com/content/DriverDownload-March2009/confirmation.php?url=/tesla/450.80.02/NVIDIA-Linux-x86_64-450.80.02.run&lang=us&type=Tesla
chmod +x NVIDIA-Linux-x86_64-450.80.02.run
sudo ./NVIDIA-Linux-x86_64-450.80.02.run

Verify the installation: nvidia-smi The command should display the GPU model, driver version, and usage statistics.

2. Install the CUDA Toolkit

CUDA provides the GPU runtime and development libraries required by most deep‑learning frameworks. Download the appropriate version from the CUDA Toolkit archive (e.g., https://developer.nvidia.com/cuda-toolkit-archive) and run the installer following the same pattern as the driver.

After installation, add the CUDA binaries to the system PATH and load the changes:

echo 'export PATH=/usr/local/cuda/bin:$PATH' | sudo tee /etc/profile.d/cuda.sh
source /etc/profile

3. Install nvidia-docker2

nvidia-docker2

supplies the Docker runtime components that automatically bind the host GPU into containers. The package consists of:

libnvidia-container

nvidia-container-toolkit

nvidia-container-runtime

Installation steps (Ubuntu/Debian example):

# Detect the distribution identifier (e.g., ubuntu20.04)
distribution=$( . /etc/os-release; echo "$ID$VERSION_ID" )
# Add the NVIDIA package repository GPG key
curl -s -L https://nvidia.github.io/nvidia-docker/gpgkey | sudo apt-key add -
# Add the repository list for the detected distribution
curl -s -L https://nvidia.github.io/nvidia-docker/$distribution/nvidia-docker.list | sudo tee /etc/apt/sources.list.d/nvidia-docker.list
# Install the package and restart Docker
sudo apt-get update
sudo apt-get install -y nvidia-docker2
sudo systemctl restart docker

4. Verify Docker GPU access

Run a CUDA base image and query the GPU from inside the container:

docker run --rm --gpus all nvidia/cuda:10.2-base nvidia-smi

If the output matches the host nvidia-smi result, the Docker runtime can see the GPU.

5. Test with deep‑learning frameworks

TensorFlow (inside a container that includes TensorFlow):

import tensorflow as tf
print(tf.config.list_physical_devices('GPU'))

PyTorch:

import torch
print(torch.cuda.is_available())

Both commands should report the GPU(s) as available.

6. Common Docker run patterns

Use all GPUs

docker run --rm --gpus all nvidia/cuda nvidia-smi
# Equivalent legacy syntax
docker run --rm --runtime=nvidia -e NVIDIA_VISIBLE_DEVICES=all nvidia/cuda nvidia-smi

Expose specific GPUs (e.g., devices 1 and 2)

docker run --rm --gpus "device=1,2" nvidia/cuda nvidia-smi
# Legacy syntax
docker run --rm --runtime=nvidia -e NVIDIA_VISIBLE_DEVICES=1,2 nvidia/cuda nvidia-smi

These steps provide a minimal, functional environment for GPU‑accelerated computing inside Docker. Further work can extend the setup to Kubernetes GPU scheduling or multi‑node training.