Enable GPU Support in Kubernetes with Containerd and NVIDIA Runtime

Kubernetes previously used Docker via a dockershim component, but the dockershim is now deprecated. Kubernetes can request GPU resources for pods, enabling deep‑learning and blockchain workloads.

1. Install NVIDIA driver

Install the NVIDIA driver on the host, preferably using the script from the official NVIDIA website; ensure gcc and kernel headers are present.

Install gcc and kernel‑dev: sudo apt install gcc kernel-dev -y Download driver from the official site.

Select the operating system and version, then download.

Example download command:

$ wget https://www.nvidia.com/content/DriverDownload-March2009/confirmation.php?url=/tesla/450.80.02/NVIDIA-Linux-x86_64-450.80.02.run&type=Tesla

Install the driver:

$ chmod +x NVIDIA-Linux-x86_64-450.80.02.run && ./NVIDIA-Linux-x86_64-450.80.02.run

Verify installation with nvidia-smi (output similar to the screenshot below).

2. Install CUDA toolkit

CUDA (Compute Unified Device Architecture) is NVIDIA's parallel computing platform.

Download the appropriate CUDA version from NVIDIA and add it to the PATH:

$ echo 'export PATH=/usr/local/cuda/bin:$PATH' | sudo tee /etc/profile.d/cuda.sh
$ source /etc/profile

3. Install nvidia-container-runtime

The nvidia‑container‑runtime adds a hook that mounts GPU devices when the NVIDIA_VISIBLE_DEVICES environment variable is set.

Set the repository and GPG key:

$ curl -s -L https://nvidia.github.io/nvidia-container-runtime/gpgkey | sudo apt-key add -
$ curl -s -L https://nvidia.github.io/nvidia-container-runtime/$(. /etc/os-release; echo $ID$VERSION_ID)/nvidia-container-runtime.list | sudo tee /etc/apt/sources.list.d/nvidia-container-runtime.list

Install the runtime:

$ apt install nvidia-container-runtime -y

Configure Containerd to use NVIDIA runtime

If /etc/containerd does not exist, create it, then generate the default configuration and modify it to use the NVIDIA runtime.

$ mkdir /etc/containerd
$ containerd config default > /etc/containerd/config.toml
... edit /etc/containerd/config.toml to set runtime_type and runtime to nvidia-container-runtime ...
$ systemctl restart containerd

4. Deploy NVIDIA device plugin

Apply the device‑plugin manifest:

$ kubectl apply -f https://raw.githubusercontent.com/NVIDIA/k8s-device-plugin/v0.7.1/nvidia-device-plugin.yml

Check the plugin logs to confirm it is running (sample log shown below).

5. Test GPU inside a pod

Pull a CUDA image and run nvidia-smi with --gpus 0 using ctr to verify host GPU visibility.

$ ctr images pull docker.io/nvidia/cuda:9.0-base
$ ctr run --rm -t --gpus 0 docker.io/nvidia/cuda:9.0-base nvidia-smi nvidia-smi

Create a pod manifest ( gpu-pod.yaml) that requests one GPU:

apiVersion: v1
kind: Pod
metadata:
  name: cuda-vector-add
spec:
  restartPolicy: OnFailure
  containers:
  - name: cuda-vector-add
    image: "k8s.gcr.io/cuda-vector-add:v0.1"
    resources:
      limits:
        nvidia.com/gpu: 1

Deploy the pod and verify it completes successfully, then view the pod logs to see the CUDA vector‑add test output.

$ kubectl apply -f ./gpu-pod.yaml
$ kubectl get pod
$ kubectl logs cuda-vector-add

The logs show successful GPU computation, confirming that Kubernetes can schedule and run GPU‑accelerated workloads. Note that current Kubernetes scheduling is at the card level and GPU resources are exclusive to a single container.