A working NVIDIA GPU on Ubuntu comes down to three things: the kernel driver, the CUDA toolkit, and a test program that proves both. This guide installs the NVIDIA drivers and the CUDA toolkit on Ubuntu 26.04 and 24.04 from a clean system, using the open kernel modules that are now the only flavor in NVIDIA’s repo, and ends with a compiled CUDA program running on the card.
The path is NVIDIA’s own apt repository: one keyring, one driver package, one toolkit package. After the core install you get the parts people usually hunt for separately: whether the driver has to match the toolkit, Ubuntu’s own ubuntu-drivers packages, two CUDA versions side by side, GPU access in Docker, a PyTorch check, and a clean uninstall. Built and run on fresh Ubuntu 26.04.1 and 24.04.5 machines with an NVIDIA A10G in September 2026, on driver 615.71.09 and CUDA 13.4.2.
Prerequisites
You need an NVIDIA card and sudo on the box. Still picking hardware? See how much VRAM an LLM needs and the best GPU for LLMs breakdown first. On Fedora, the RPM Fusion akmod route is in our NVIDIA drivers on Fedora guide.
- An NVIDIA GPU, Turing or newer (GTX 16, RTX 20 and up, A and H series). Older cards need the legacy branch in the table below.
- Ubuntu 26.04 LTS or 24.04 LTS, x86_64.
- About 8 GB free disk. The toolkit alone unpacks to 5.2 GB.
- One reboot.
Add the CUDA repository
The driver builds its kernel module through DKMS, so it needs a compiler and headers for the running kernel:
sudo apt update
sudo apt install -y build-essential linux-headers-$(uname -r)NVIDIA publishes one repository per Ubuntu release. Detect yours so the same commands work on both:
distro=$(. /etc/os-release && echo "ubuntu${VERSION_ID/./}")
echo "$distro"That prints ubuntu2604 or ubuntu2404. It is the only thing that changes between the two releases. Add the signed repository with the keyring package:
wget https://developer.download.nvidia.com/compute/cuda/repos/${distro}/x86_64/cuda-keyring_1.1-1_all.deb
sudo dpkg -i cuda-keyring_1.1-1_all.deb
sudo apt updateCheck what the repo now offers. Both releases resolved the same driver and toolkit builds:
apt-cache policy nvidia-open cuda-toolkit | grep -E '^\S|Candidate'On 26.04 and 24.04 alike:
nvidia-open:
Candidate: 615.71.09-2ubuntu1
cuda-toolkit:
Candidate: 13.4.2-1Install the NVIDIA driver on Ubuntu
Check your GPU generation first. This is the one decision that trips people up, because the current driver branch is open modules only. NVIDIA stopped shipping the proprietary module flavor in its repo, and cuda-drivers is now a transitional package that pulls nvidia-open:
| GPU generation | Driver | Toolkit |
|---|---|---|
| Turing and newer (GTX 16, RTX 20/30/40/50, A and H series) | nvidia-open | cuda-toolkit (CUDA 13) |
| Maxwell, Pascal, Volta on 24.04 | cuda-drivers-580 from the NVIDIA repo | cuda-toolkit-12-9 |
| Maxwell, Pascal, Volta on 26.04 | Ubuntu’s nvidia-driver-580 | No CUDA 12 in the 26.04 repo; stay on 24.04 for these cards |
The 580 branch is the last one that supports pre-Turing cards, and CUDA 13 dropped them as compile targets. Everything below assumes Turing or newer. Install the driver:
sudo apt install -y nvidia-openThe nvidia-open install pulled 117 packages on 26.04 and 131 on 24.04, and took under three minutes on both, most of it the DKMS build. It also drops /usr/lib/modprobe.d/nvidia-graphics-drivers.conf, which blacklists nouveau for you. The old “blacklist nouveau by hand” step is gone; on the test machines nouveau was loaded before the install and absent after the reboot.
Secure Boot: DKMS signs the modules with a machine key at /var/lib/shim-signed/mok/MOK.der, and on a Secure Boot machine the first reboot shows a blue MOK screen where you pick Enroll MOK and type the password apt asked for. Our lab boxes had Secure Boot off, so we did not walk that screen. If you got no prompt, or turn Secure Boot on later, queue the key by hand with sudo mokutil --import /var/lib/shim-signed/mok/MOK.der and reboot. Check your state:
mokutil --sb-stateSecureBoot disabled means nothing to enroll. Reboot so the module loads:
sudo rebootBack in, check the card:
nvidia-smiThe A10G shows up with 23 GB free. The newer header prints a separate KMD Version (kernel module) and CUDA UMD Version (user-mode driver) where older drivers printed one Driver Version and one CUDA Version:
Confirm it is the open module and not the proprietary one:
cat /proc/driver/nvidia/versionOpen Kernel Module is the proof, and the second line names the compiler DKMS used (GCC 15 on 26.04, GCC 13 on 24.04):
NVRM version: NVIDIA UNIX Open Kernel Module for x86_64 615.71.09 Release Build (dvs-builder@U22-I3-F05-2-4) Fri Sep 4 22:33:33 UTC 2026
GCC version: gcc version 15.2.0 (Ubuntu 15.2.0-16ubuntu1)Install the CUDA toolkit on Ubuntu
The driver runs the GPU. The toolkit gives you nvcc, the CUDA libraries and the headers. The meta package tracks the newest release and keeps whatever driver you already picked:
sudo apt install -y cuda-toolkitAbout 2m40s on the test boxes. It lands in /usr/local/cuda-13.4, with /usr/local/cuda as an update-alternatives symlink. It does not touch your PATH, so nvcc is “command not found” until you add it. The library line uses a guarded expansion so an unset LD_LIBRARY_PATH does not leave an empty entry (which makes the loader search the current directory):
echo 'export PATH=/usr/local/cuda/bin:$PATH' >> ~/.bashrc
echo 'export LD_LIBRARY_PATH=/usr/local/cuda/lib64${LD_LIBRARY_PATH:+:$LD_LIBRARY_PATH}' >> ~/.bashrc
source ~/.bashrcCheck the compiler:
nvcc --versionRelease and build line:
nvcc: NVIDIA (R) Cuda compiler driver
Copyright (c) 2005-2026 NVIDIA Corporation
Built on Tue_Sep_01_08:45:21_PDT_2026
Cuda compilation tools, release 13.4, V13.4.92
Build cuda_13.4.r13.4/compiler.38855100_0Verify with a real CUDA program
nvidia-smi proves the driver. Compiling and running NVIDIA’s CUDA samples proves the toolkit and the whole chain. They build with CMake:
sudo apt install -y cmake git
git clone --depth 1 https://github.com/NVIDIA/cuda-samples.git
cd cuda-samples
cmake -B build
cmake --build build --target deviceQuery vectorAdd -j$(nproc)Configure plus build took under a minute. GCC 15 on 26.04 compiled cleanly, no -ccbin workaround. Pinned an older toolkit? Clone the matching samples tag instead, since a shallow clone carries no tags (for example git clone --depth 1 --branch v13.3). Run both programs:
./build/cpp/1_Utilities/deviceQuery/deviceQuery
./build/cpp/0_Introduction/vectorAdd/vectorAdddeviceQuery reports compute capability 8.6 and 10240 CUDA cores on the A10G and ends with Result = PASS. The current vectorAdd uses unified memory and prints Unified Memory: CPU and GPU answers match. Older write-ups tell you to look for Test PASSED; that string is gone from this sample.
That’s it. Driver, toolkit and card are talking. Both releases ran this exact sequence.
| Item | Ubuntu 26.04 | Ubuntu 24.04 |
|---|---|---|
| NVIDIA repo path | repos/ubuntu2604 | repos/ubuntu2404 |
| Host compiler DKMS used | GCC 15 | GCC 13 |
Packages pulled by nvidia-open | 117 | 131 |
Ubuntu archive nvidia-cuda-toolkit | 12.4 | 12.0 |
| Driver + toolkit from NVIDIA repo | Identical builds on both | |
Does the NVIDIA driver have to match the CUDA version?
No, not within the same major version. A CUDA 13.x toolkit runs on an older 13.x driver (580 or newer). That is NVIDIA’s minor version compatibility, and we tested it rather than taking it on faith. The 24.04 box ran Ubuntu’s own 595 driver, which reports CUDA 13.2, with the 13.4 toolkit from NVIDIA’s repo on top. Both samples passed:
Two limits apply. Code that JIT-compiles PTX needs the newer driver, and a call into a feature the older driver lacks returns cudaErrorCallRequiresNewerDriver. The version field in the nvidia-smi header (labelled “CUDA Version” on 595, “CUDA UMD Version” on 615) is the newest CUDA the driver supports, not the toolkit you installed; nvcc --version tells you that. Crossing a major version (a CUDA 13 toolkit on a 12-era driver) is where things break. Upgrade the driver there; the cuda-compat packages only help on datacenter GPUs.
Use Ubuntu’s own NVIDIA driver packages instead
Ubuntu ships its own driver builds with prebuilt, signed kernel modules. No DKMS compile, and no MOK dance on Secure Boot. Pick one stack: if you already installed nvidia-open from NVIDIA’s repo, run the purge in the uninstall section first, which is what we did on the 24.04 box. The tool is ubuntu-drivers, which cloud images leave out:
sudo apt install -y ubuntu-drivers-common
ubuntu-drivers devicesOn 24.04 it listed branches from 470-server up to 610 and marked 595-open as recommended:
== /sys/devices/pci0000:00/0000:00:1e.0 ==
modalias : pci:v000010DEd00002237sv000010DEsd0000152Fbc03sc02i00
vendor : NVIDIA Corporation
model : GA102GL [A10G]
driver : nvidia-driver-580 - distro non-free
driver : nvidia-driver-470-server - distro non-free
driver : nvidia-driver-580-server-open - distro non-free
driver : nvidia-driver-580-open - distro non-free
driver : nvidia-driver-595-open - distro non-free recommended
driver : nvidia-driver-595 - distro non-free
driver : nvidia-driver-595-server - distro non-free
driver : nvidia-driver-610 - distro non-free
driver : nvidia-driver-580-server - distro non-free
driver : nvidia-driver-610-open - distro non-free
driver : nvidia-driver-595-server-open - distro non-free
driver : xserver-xorg-video-nouveau - distro free builtinFor a headless compute box, use the --gpgpu flag:
sudo ubuntu-drivers install --gpgpuHere’s the catch. That picked nvidia-headless-no-dkms-595-server-open, which loads the kernel module but does not install nvidia-smi. After the reboot, /proc/driver/nvidia/version showed the 595 open module and nvidia-smi was “command not found”. Add the utils package for the same branch:
sudo apt install -y nvidia-utils-595-serverSkip Ubuntu’s nvidia-cuda-toolkit package. It is 12.4 on 26.04 and 12.0 on 24.04, a full major release behind. Pair the Ubuntu driver with cuda-toolkit from NVIDIA’s repo, as in the compatibility test above.
Run two CUDA versions side by side
Some framework builds want an older toolkit. Versioned packages install next to each other:
sudo apt install -y cuda-toolkit-13-3 # https://developer.nvidia.com/cuda-toolkit-archive
update-alternatives --display cudaBoth register with the cuda alternative, and the newest wins on priority:
cuda - auto mode
link best version is /usr/local/cuda-13.4
link currently points to /usr/local/cuda-13.4
link cuda is /usr/local/cuda
/usr/local/cuda-13.3 - priority 133
/usr/local/cuda-13.4 - priority 134Point /usr/local/cuda at the older one. Your PATH already uses the symlink, so nvcc follows:
sudo update-alternatives --set cuda /usr/local/cuda-13.3
nvcc --version | grep releaseThe switch is instant:
update-alternatives: using /usr/local/cuda-13.3 to provide /usr/local/cuda (cuda) in manual mode
Cuda compilation tools, release 13.3, V13.3.73Back to the newest with sudo update-alternatives --auto cuda.
Give Docker containers the GPU
The NVIDIA Container Toolkit ships in the same CUDA repo you already added, so there is no extra repo to wire up. Containers never need the driver inside them; the toolkit mounts the host driver in at run time:
sudo apt install -y docker.io nvidia-container-toolkit
sudo nvidia-ctk runtime configure --runtime=docker
sudo systemctl restart docker--gpus all already worked with Ubuntu’s Docker package before the configure step. Run it anyway: it registers the nvidia runtime in /etc/docker/daemon.json, which Compose files using runtime: nvidia expect. Test with NVIDIA’s base image:
sudo docker run --rm --gpus all nvidia/cuda:13.4.1-base-ubuntu26.04 nvidia-smi --query-gpu=name,driver_version,memory.total --format=csv # https://hub.docker.com/r/nvidia/cuda/tagsThe container sees the host’s card and driver:
name, driver_version, memory.total [MiB]
NVIDIA A10G, 615.71.09, 23028 MiBMind the tag. 13.4.0-base does not exist on Docker Hub, and docker run fails with “not found”. Pick a real one from the tags page.
Check PyTorch sees the GPU
PyTorch wheels bundle their own CUDA runtime, so they only need the driver. The fastest check uses uv, with no virtualenv to set up:
curl -LsSf https://astral.sh/uv/install.sh | sh
export PATH="$HOME/.local/bin:$PATH"
uv run --python 3.13 --with torch python -c 'import torch; print(torch.__version__, torch.cuda.is_available(), torch.cuda.get_device_name(0))'The first run downloads the wheel. After that the check takes seconds. True plus the card name means you’re live:
The +cu130 suffix is the CUDA runtime inside the wheel. An older runtime on a newer driver is plain backward compatibility, which NVIDIA drivers guarantee as long as they still support the GPU.
Kernel updates and DKMS
Nothing to do. When a new kernel lands, the DKMS hook rebuilds the NVIDIA module for it during the apt run. Installing linux-generic on the 26.04 box showed it happen:
/etc/kernel/header_postinst.d/dkms:
Autoinstall of module nvidia/615.71.09 for kernel 7.0.0-34-generic (x86_64)
Building module(s)................................. done.
Signing module /var/lib/dkms/nvidia/615.71.09/build/nvidia.kodkms status then lists one line per kernel. If a kernel is missing there, its headers were not installed, so install linux-headers-<version> for it and the build runs.
nvidia/615.71.09, 7.0.0-1012-aws, x86_64: installed
nvidia/615.71.09, 7.0.0-34-generic, x86_64: installedErrors you will see
“NVIDIA-SMI has failed because it couldn’t communicate with the NVIDIA driver”
You ran nvidia-smi after the driver install but before the reboot. The module is built but not loaded. Reboot. If it persists after a reboot, check dkms status for your running kernel and mokutil --sb-state for an unenrolled key.
“nvcc: command not found”
The toolkit is installed but /usr/local/cuda/bin is not on PATH. Add the two export lines from the toolkit step.
“nvidia-smi: command not found” after ubuntu-drivers install
The --gpgpu headless package skips the utilities. Install nvidia-utils-<branch>-server matching the branch it picked.
“No rule to make target ‘bandwidthTest'”
bandwidthTest is no longer in the current cuda-samples tree (it was removed as of the CUDA Samples 12.9 release in favor of the separate nvbandwidth tool), so older guides that build it fail at this step. deviceQuery and vectorAdd are enough to prove the install.
Uninstall the NVIDIA driver and CUDA
One purge removes the NVIDIA-repo driver, toolkit, libraries and the repo keyring. On the 24.04 box it left no nvidia or cuda package and no /usr/local/cuda* behind. If you used ubuntu-drivers instead, add '^linux-modules-nvidia-.*' to the list:
sudo apt purge -y '^nvidia-.*' '^libnvidia-.*' '^cuda-.*' '^libcublas.*' '^libcufft.*' '^libcurand.*' '^libcusolver.*' '^libcusparse.*' '^libnpp.*' '^libnvjitlink.*'
sudo apt autoremove -y --purgeReboot afterwards so nouveau takes the card back.
The whole install, in order
Fresh box to a GPU that compiles CUDA: about six minutes of apt plus one reboot.
sudo apt update
sudo apt install -y build-essential linux-headers-$(uname -r)
distro=$(. /etc/os-release && echo "ubuntu${VERSION_ID/./}")
wget https://developer.download.nvidia.com/compute/cuda/repos/${distro}/x86_64/cuda-keyring_1.1-1_all.deb
sudo dpkg -i cuda-keyring_1.1-1_all.deb
sudo apt update
sudo apt install -y nvidia-open
sudo reboot
# after the reboot:
sudo apt install -y cuda-toolkit
echo 'export PATH=/usr/local/cuda/bin:$PATH' >> ~/.bashrc
echo 'export LD_LIBRARY_PATH=/usr/local/cuda/lib64${LD_LIBRARY_PATH:+:$LD_LIBRARY_PATH}' >> ~/.bashrc
source ~/.bashrc
nvidia-smi
nvcc --versionNext step for most people: run a local LLM with Ollama, serve models with vLLM, or build llama.cpp with CUDA. Running a fleet of these? Monitor the GPUs with DCGM and Grafana, and see the local AI workstation build for the hardware around the card.
