Elastic GPU Service:Error when starting a container image: docker: Error response from daemon: could not select device driver "" with capabilities: [[gpu]]

Problem description

After you install Docker on an Elastic GPU Service instance, the following error occurs when you run the docker run --gpus all [Image Name] command to start a container:

Cause

The NVIDIA Container Toolkit allows Docker to access GPU resources. This error occurs because the NVIDIA Container Toolkit is not installed on the Elastic GPU Service instance. Without the toolkit, Docker cannot select a GPU device.

Solution

1. Confirm that the NVIDIA GPU driver is installed on the GPU-accelerated instance.

   Note

   GPU-accelerated instances do not have pre-installed drivers. You must install the required driver. If the NVIDIA GPU driver is not installed, Docker cannot access the GPU device.

   nvidia-smi

   If the output shows the driver version, the NVIDIA GPU driver is installed. If not, see Install a Tesla driver or Install a GRID driver.

   

2. Confirm that Docker is installed on the GPU-accelerated instance.

   sudo docker -v

   If the output shows the Docker version, Docker is installed. If not, see Install and use Docker and Docker Compose.

   

3. Install the NVIDIA Container Toolkit.

   This step provides examples for CentOS, Alibaba Cloud Linux, and Ubuntu. For installation commands for other operating systems, see Installing the NVIDIA Container Toolkit.

   • Alibaba Cloud Linux 3/CentOS 8

     # Configure the repository.
     curl -s -L https://nvidia.github.io/libnvidia-container/stable/rpm/nvidia-container-toolkit.repo | \
       sudo tee /etc/yum.repos.d/nvidia-container-toolkit.repo
     
     # Install the toolkit.
     sudo yum install -y nvidia-container-toolkit
     
     # Restart the Docker service.
     sudo systemctl restart docker

     If the installation fails, try the following method.

     sudo dnf config-manager --add-repo=https://developer.download.nvidia.com/compute/cuda/repos/rhel8/x86_64/cuda-rhel8.repo
     sudo yum install -y nvidia-container-toolkit

   • Ubuntu

     # Configure the repository.
     curl -fsSL https://nvidia.github.io/libnvidia-container/gpgkey | sudo gpg --dearmor -o /usr/share/keyrings/nvidia-container-toolkit-keyring.gpg \
       && curl -s -L https://nvidia.github.io/libnvidia-container/stable/deb/nvidia-container-toolkit.list | \
         sed 's#deb https://#deb [signed-by=/usr/share/keyrings/nvidia-container-toolkit-keyring.gpg] https://#g' | \
         sudo tee /etc/apt/sources.list.d/nvidia-container-toolkit.list
     
     sudo apt-get update
     
     # Install the toolkit.
     sudo apt-get install -y nvidia-container-toolkit
     
     # Restart the Docker service.
     sudo systemctl restart docker

     If the installation fails, try the following method.

     distribution=$(. /etc/os-release;echo $ID$VERSION_ID | sed -e 's/\.//g')
     wget https://developer.download.nvidia.com/compute/cuda/repos/$distribution/x86_64/3bf863cc.pub
     sudo apt-key add 3bf863cc.pub
     echo "deb https://developer.download.nvidia.com/compute/cuda/repos/$distribution/x86_64 /" | sudo tee /etc/apt/sources.list.d/cuda.list
     sudo apt-get update
     sudo apt-get install -y nvidia-container-toolkit

4. Verify that the NVIDIA Container Toolkit is installed.

   • CentOS/Alibaba Cloud Linux

     sudo rpm -qa | grep nvidia-container-toolkit

   • Ubuntu

     sudo dpkg -l | grep nvidia-container-toolkit

   If the output shows the NVIDIA Container Toolkit version, the toolkit is installed correctly.

   

5. Run docker run --gpus all [Image Name] to verify that the issue is resolved.