Manually install CUDA - Elastic GPU Service - Alibaba Cloud Documentation Center

Background

CUDA (Compute Unified Device Architecture) is a general-purpose parallel computing architecture developed by NVIDIA. It leverages the processing power of a graphics processing unit (GPU) to solve complex computational problems. It includes the CUDA instruction set architecture (ISA) and a parallel computing engine within the GPU. You can use CUDA with C, C++, and Fortran to develop high-performance programs that run on CUDA-enabled processors. CUDA also provides interfaces for various languages, such as Java and Python.

Prerequisites

You have created a GPU-accelerated instance. For more information, see Create a GPU-accelerated instance.
Make sure that the driver version on the GPU-accelerated instance is compatible with the CUDA version you want to install. For more information, see CUDA Toolkit and Corresponding Driver Versions.

Procedure

CUDA supports both the Windows and Linux operating systems. This topic uses CUDA Toolkit 12.1.1 as an example to demonstrate how to install CUDA on GPU-accelerated instances that run Linux and Windows.

Linux

Obtain the CUDA installation package.

Go to the CUDA Toolkit Archive page.
Select the CUDA version that corresponds to your driver.

This topic uses CUDA Toolkit 12.1.1 as an example.

On the official NVIDIA Developer CUDA Toolkit Archive page, locate your target version, such as CUDA Toolkit 12.1.1 (April 2023), and click the corresponding download link.

Get the download URL for the CUDA installation package.

Select the required parameters, such as the operating system, architecture, and version.

Parameter	Example value
Operating system	Linux
Architecture	x86_64
Distribution	Ubuntu
Version	20.04
Installer type	runfile (local)

Copy the base installer commands from the page.

$ wget https://developer.download.nvidia.com/compute/cuda/12.1.1/local_installers/cuda_12.1.1_530.30.02_linux.run
$ sudo sh cuda_12.1.1_530.30.02_linux.run

Install the CUDA package.

Connect to the GPU-accelerated instance using one of the following methods.

Connection method	Instructions
Workbench	Log on to a Linux instance by using Workbench
VNC	Connect to an instance by using VNC

Use the wget command and paste the download address from Step 1 to download the CUDA installation package.
```
wget https://developer.download.nvidia.com/compute/cuda/12.1.1/local_installers/cuda_12.1.1_530.30.02_linux.run
```
Important
The download may take 5 to 10 minutes.
Run the following commands to install the CUDA package.
Note
- For a GPU-accelerated compute-optimized instance, you can install the Tesla driver manually or install the bundled driver.
- For a vGPU-accelerated instance, you cannot install the bundled driver during the CUDA installation. You must manually install the NVIDIA GRID driver. For more information, see Install an NVIDIA GRID driver.
- Interactive installation
```
sudo chmod +x cuda_12.1.1_530.30.02_linux.run
```
```
sudo ./cuda_12.1.1_530.30.02_linux.run
```
  After you enter accept at the prompt, choose whether to install the bundled driver based on your scenario:
  - If you have a GPU-accelerated compute-optimized instance with an existing driver, press the Enter key to deselect the Driver option as shown in the following example. Then, select Install to continue.
    
    Important
    If you do not deselect the Driver option, the bundled driver overwrites the existing driver on the instance.
```
CUDA Installer
- [ ] Driver
    [ ] 530.30.02
+ [X] CUDA Toolkit 12.1
  [X] CUDA Demo Suite 12.1
  [X] CUDA Documentation 12.1
- [ ] Kernel Objects
    [ ] nvidia-fs
  Options
  Install
Up/Down: Move | Left/Right: Expand | 'Enter': Select | 'A': Advanced options
```
  - If you have a GPU-accelerated compute-optimized instance and want to use the bundled driver, first verify its compatibility with your instance's GPU model. For more information, see Download a Tesla driver (Linux). If the driver is compatible, keep the default selection (the Driver option is selected) and then select Install to continue.
  - For a vGPU-accelerated instance, an NVIDIA GRID driver is required. First, check if a compatible driver is already installed. For more information, see Install an NVIDIA GRID driver. If not, install the driver before proceeding. During the CUDA installation, deselect the Driver option by pressing the Enter key and then select Install to continue.
- Silent installation
```
sudo chmod +x cuda_12.1.1_530.30.02_linux.run
```
```
sudo ./cuda_12.1.1_530.30.02_linux.run --toolkit --samples --silent
```
  Important
  If you use silent installation, the bundled driver is not automatically installed. You must manually install the required driver for your GPU-accelerated instance.
(Optional) Restart the GPU-accelerated instance:
```
reboot
```

Configure the CUDA environment variables:

echo 'export PATH=/usr/local/cuda/bin:$PATH' | sudo tee /etc/profile.d/cuda.sh

source /etc/profile

Verify the CUDA installation.

Run the nvcc -V command to check if CUDA is installed correctly.

root@iZxxxZ:~# nvcc -V
nvcc: NVIDIA (R) Cuda compiler driver
Copyright (c) 2005-2023 NVIDIA Corporation
Built on Mon Apr__3_17:16:06_PDT_2023
Cuda compilation tools, release 12.1, V12.1.105
Build cuda_12.1.r12.1/compiler.32688072_0

Test the CUDA Samples to verify the installation:

cd /usr/local/cuda-12.1/extras/demo_suite

./deviceQuery

If the output is Result=PASS, the CUDA installation is successful.

deviceQuery, CUDA Driver = CUDART, CUDA Driver Version = 12.1, CUDA Runtime Version = 12.1, NumDevs = 1, Device0 = Tesla T4
Result = PASS

Windows

Log on to a GPU-accelerated compute-optimized instance that runs Windows.

Connect to the GPU-accelerated instance using one of the following methods:

Connection method	Instructions
Workbench	Log on to a Windows instance by using Workbench
VNC	Connect to an instance by using VNC

Select the CUDA version to install.

Go to the CUDA Toolkit Archive page.
Select the CUDA version that corresponds to your driver.

First, verify that the driver bundled with the CUDA package is compatible with your instance's GPU model. For more information, see Download a Tesla driver (Windows). This topic uses CUDA Toolkit 12.1.1 as an example.