How to install the CUDA Toolkit on Ubuntu 24.04 / 26.04?

Updated: 28 September 2026 · Applies to: CUDA Toolkit 13.4 on Ubuntu 24.04 / 26.04 LTS

The CUDA Toolkit contains the nvcc compiler and the development libraries that you need to compile CUDA programs. You do not need it to run PyTorch, Ollama or most prebuilt AI tools; they only need the NVIDIA driver. Install the toolkit when you build CUDA code yourself or when a project asks for nvcc.

Before you start

The NVIDIA driver must be installed and working: nvidia-smi must print a table (see How to install the NVIDIA driver on Ubuntu 24.04 / 26.04 for a GPU server?). Since CUDA 13.4 the toolkit package installs the toolkit only and does not install a driver.
Your driver must be new enough for the toolkit you install: CUDA 13.x needs driver 580 or newer.

Install the toolkit

Add NVIDIA's repository. Use ubuntu2404 on Ubuntu 24.04 and ubuntu2604 on Ubuntu 26.04:

wget https://developer.download.nvidia.com/compute/cuda/repos/ubuntu2404/x86_64/cuda-keyring_1.1-1_all.deb
sudo dpkg -i cuda-keyring_1.1-1_all.deb
sudo apt update

Install the toolkit:
```
sudo apt install -y cuda-toolkit
```
NVIDIA recommends the cuda-toolkit package for most cases. To pin one release, use its versioned name, for example cuda-toolkit-13-4.
Add the compiler to your PATH for the current session and for future logins:
```
export PATH=$PATH:/usr/local/cuda-13.4/bin
echo 'export PATH=$PATH:/usr/local/cuda-13.4/bin' >> ~/.bashrc
```
Replace 13.4 with the version that ls /usr/local | grep cuda shows.
Check the compiler:
```
nvcc --version
```

Common problems

nvcc: command not found: the PATH line is missing. Repeat step 3 and open a new SSH session.
The driver is too old for the toolkit: programs stop with a message that the CUDA driver version is insufficient. Update the driver first (How to install the NVIDIA driver on Ubuntu 24.04 / 26.04 for a GPU server?).
nvidia-smi shows a different CUDA version than nvcc --version: this is normal. nvidia-smi shows the newest CUDA that the driver supports, and nvcc shows the toolkit you installed. The toolkit version must not be newer than what the driver supports.

Frequently asked questions

Do I need the toolkit for PyTorch?
No. PyTorch's pip packages bring their own CUDA libraries. See How to install PyTorch with GPU (CUDA) support and test it?.

Can I keep two CUDA versions on one server?
Yes. Versioned packages such as cuda-toolkit-13-4 install into their own folder under /usr/local. Point PATH at the one you want to use.

Official documentation: NVIDIA CUDA installation guide for Linux.

Need a GPU server, or a hand with the setup?

GPU dedicated servers: NVIDIA GPU servers for AI training and inference; our engineers can install the driver, CUDA, PyTorch or a private LLM and hand it over ready to use.
Private LLM installation: we install Ollama, the GPU driver and a chat interface on your own server.

Prefer a hand with the setup? Our engineers can do it for you: Hire an Expert, or use our on-demand server management.