Elastic Compute Service:How do I install the NVMe driver for a custom image?

NVMe is a high-speed interface protocol for solid-state storage such as flash-based SSDs. NVMe delivers faster response and higher bandwidth than traditional driver protocols, such as Small Computer System Interface (SCSI) and virtio-blk. For more information about NVMe, see NVMe protocol.

Procedure

1. Create an ECS instance from an existing custom image to transfer data of the custom image to the instance.

   For more information, see Create an instance by using a custom image.

   Important

   You are charged for the intermediate ECS instance that you create. After you create a custom image from the intermediate ECS instance, we recommend that you release the intermediate ECS instance at the earliest opportunity. For more information, see Release an instance.

2. Connect to the new ECS instance.

   For more information, see Connection method overview.

3. Check whether the NVMe driver is installed on the ECS instance. If the NVMe driver is not installed, install the driver.

   Automatic configuration by using Cloud Assistant

   Cloud Assistant provides the ecs_nvme_config plug-in to help you quickly complete the NVMe driver configuration based on the operating system.

   Operating systems supported by the ecs_nvme_config plug-in

   • The ecs_nvme_config plug-in supports the following operating systems:

     • Alibaba Cloud Linux

     • Anolis OS

     • CentOS and CentOS Stream

     • Debian

     • Ubuntu

     • openSUSE

     • SUSE Linux Enterprise Server (SLES)

     • Red Hat Enterprise Linux

     • Fedora

     • Rocky Linux

     • AlmaLinux

   • The ecs_nvme_config plug-in does not support the following operating system versions:

     • CentOS and Red Hat Enterprise Linux: versions earlier than CentOS 6.6 and Red Hat Enterprise Linux 6.6

     • Debian: versions earlier than Debian 9

     • Ubuntu: versions earlier than Ubuntu 16

     • openSUSE: versions earlier than openSUSE 42

     • SLES: versions earlier than SLES 11.4

   1. Run the following command to check whether Cloud Assistant Agent is installed on the instance, and if Cloud Assistant Agent is installed, check whether Cloud Assistant Agent provides the ecs_nvme_config plug-in:

      acs-plugin-manager --list

      

      • If no Cloud Assistant Agent information is returned, you must install Cloud Assistant Agent on the instance. For information about how to install Cloud Assistant Agent, see Install Cloud Assistant Agent.

      • If a list of Cloud Assistant plug-ins is returned and the ecs_nvme_config plug-in appears, proceed to the next step.

   2. Use the ecs_nvme_config plug-in to configure NVMe-related settings.

      1. Run the following command to use the ecs_nvme_config plug-in to check whether the instance has the NVMe module and whether the instance supports the NVMe module:

         sudo acs-plugin-manager --exec --plugin ecs_nvme_config --params --check

         • If the following message is returned, the NVMe driver is installed on the instance. You can create a custom image without the need to configure the NVMe driver.

           [SUCCESS]  Summary: Your image can Runnig on nvme instance

         • If a message similar to the following error message is returned, proceed to the next step.

           [ERROR]  1.initrd/initramfs not has nvme module, Please run acs-plugin-manager --exec --plugin ecs_nvme_config --params -f/--fix to enable nvme;

      2. Configure the NVMe driver and relevant parameters based on the error message in the command output.

         sudo acs-plugin-manager --exec --plugin ecs_nvme_config --params --fix

      3. After you configure the NVMe-related settings, run the following command to restart your instance:

         sudo reboot

      4. After you restart the instance, run the following command to check whether the NVMe module is configured:

         sudo acs-plugin-manager --exec --plugin ecs_nvme_config --params --check

         Sample success output:

         [OK]  1.initrd/initramfs already contain nvme module;
         
         [OK]  2.fstab file looks fine and does not contain any device names;
         
         [OK]  3.The nvme parameters already included.
         
         [SUCCESS]  Summary: Your image can Runnig on nvme instance

   Manual configuration in CentOS or Alibaba Cloud Linux

   1. Run the following command to check whether the kernel loaded the NVMe driver:

      cat /boot/config-`uname -r` | grep -i nvme | grep -v "^#"

      

      • If the CONFIG_BLK_DEV_NVME=y message is returned, the operating system can start on an instance of an instance family that supports NVMe. In this case, proceed to Step 3.d.

      • If the CONFIG_BLK_DEV_NVME=m message is returned, perform the following steps in sequence.

   2. Run the following command to check whether the initial Resource Access Management (RAM) file system (initramfs) contains the NVMe driver:

      sudo lsinitrd /boot/initramfs-`uname -r`.img | grep -i nvme | awk '{print $NF}'

      

      • If the command output shown in the preceding figure is returned, the operating system can start on an instance of an instance family that supports NVMe. In this case, proceed to Step 3.d.

      • If the command output shown in the preceding figure is not returned, perform the following steps in sequence.

   3. Run the following commands in sequence to configure the initramfs to support the NVMe driver:

      mkdir -p /etc/dracut.conf.d
      echo 'add_drivers+=" nvme nvme-core "' | sudo tee /etc/dracut.conf.d/nvme.conf > /dev/null
      sudo dracut -v -f

      Note

      If the dracut tool is not installed in your operating system, run the sudo yum -y install dracut command to install the dracut tool first.

   4. Add the NVMe-related io_timeout parameter to the GRand Unified Bootloader (GRUB) file.

      Note

      • The io_timeout parameter is added to the GRUB file to fix timeout errors related to the NVMe devices in the Linux operating system. If you set the io_timeout parameter to the maximum value, the operating system can continue to process I/O requests from NVMe devices, regardless whether a timeout occurs.

      • In most Linux distributions, the io_timeout parameter defaults to 30. Unit: seconds. In a new version of the kernel, set the io_timeout parameter to the maximum value of 4,294,967,295 seconds. In an earlier version of the kernel, set the parameter to 255 seconds.

      Method 1: Run the grubby command

      1. Check whether the operating system supports the grubby command.

         which grubby

         • If the path of the grubby command is returned in the command output, such as /usr/sbin/grubby, the operating system supports the grubby command. In this case, proceed to the next step.

         • If no information is returned, the operating system does not support the grubby command. In this case, use Method 2: Modify the GRUB configuration file.

      2. Run the grubby command to add the NVMe-related io_timeout parameter.

         sudo grubby --update-kernel=ALL --args="nvme_core.io_timeout=4294967295 nvme_core.admin_timeout=4294967295"

      Method 2: Modify the GRUB configuration file

      1. Run the following command to open the grub file:

         sudo vi /etc/default/grub

      2. Press the I key to enter Insert mode. At the end of the GRUB_CMDLINE_LINUX= line, add the nvme_core.io_timeout and nvme_core.admin_timeout parameters and set them both to 4294967295.

         The following figure shows an example on how to add the parameters.

         

         Note

         If the GRUB configuration file already contains the preceding parameter settings, you do not need to add the parameters again.

      3. Press the Esc key to exit Insert mode. Then, enter :wq and press the Enter key to save and close the file.

      4. Apply the GRUB configurations.

         Select a command based on the boot mode of the ECS instance.

         • Legacy BIOS boot mode

           sudo grub2-mkconfig -o /boot/grub2/grub.cfg

         • Unified Extensible Firmware Interface (UEFI) boot mode

           1. Run the following command to view the content of the GRUB configuration file:

              cat /boot/efi/EFI/centos/grub.cfg

           2. Select a command based on the output of the preceding command.

              • If the GRUB configuration file is stored in a directory different from the directory specified in the preceding command, the command output contains the new directory, such as /boot/grub/grub.cfg, as shown in the following figure. In this case, run the following command to allow the GRUB configurations to take effect.

                

                sudo grub2-mkconfig -o /boot/grub2/grub.cfg

              • If the GRUB configuration file is stored in the directory specified in the preceding cat command, run the following command to allow the GRUB configurations to take effect:

                sudo grub2-mkconfig -o /boot/efi/EFI/centos/grub.cfg

                Note

                For more information about the boot modes of ECS instances, see Boot modes of ECS instances.

   5. Run the following command again to check NVMe-related information:

      sudo lsinitrd /boot/initramfs-`uname -r`.img | grep -i nvme | awk '{print $NF}'

      If the following command output is returned, the configuration is complete and the operating system can start on an instance of an instance family that supports NVMe.

      

   Manual configuration in Ubuntu or Debian

   1. (Optional) Run the following command to view the NVMe drivers included in the initrd:

      lsinitramfs /boot/initrd.img-`uname -r` | grep -i nvme

      The following command output indicates that the NVMe driver is loaded in the initrd of the Ubuntu operating system.

   2. Add NVMe-related io_timeout parameters to the GRUB file.

      Note

      • The io_timeout parameter is added to the GRUB file to fix timeout errors related to the NVMe devices in the Linux operating system. If you set the io_timeout parameter to the maximum value, the operating system can continue to process I/O requests from NVMe devices, regardless whether a timeout occurs.

      • In most Linux distributions, the io_timeout parameter defaults to 30. Unit: seconds. In a new version of the kernel, set the io_timeout parameter to the maximum value of 4,294,967,295 seconds. In an earlier version of the kernel, set the parameter to 255 seconds.

      1. Run the following command to open the /etc/default/grub file:

         sudo vi /etc/default/grub

      2. Press the I key to enter Insert mode. On the GRUB_CMDLINE_LINUX= line, add the nvme_core.multipath, nvme_core.io_timeout, and nvme_core.admin_timeout parameters. Then, set nvme_core.multipath to n and nvme_core.io_timeout and nvme_core.admin_timeout both to 4294967295.

         The following figure shows the parameters.

         Note

         If the GRUB file already contains the preceding parameter settings, you do not need to add the parameters again.

      3. Press the Esc key to exit Insert mode. Then, enter :wq and press the Enter key to save and close the file.

   3. Apply the GRUB configurations.

      Run one of the following commands based on the boot mode of the ECS instance:

      • The following command is applicable to Ubuntu and Debian operating systems, regardless of the boot mode.

        sudo update-grub2

      • Legacy BIOS boot mode

        sudo grub-mkconfig -o /boot/grub/grub.cfg

      • UEFI boot mode

        sudo grub-mkconfig -o /boot/efi/EFI/debian/grub.cfg

      Note

      For information about the boot modes of ECS instances, see Boot modes of ECS instances.

4. Create a new custom image from the ECS instance that you created. The new custom image contains the NVMe driver.

   For more information, see Create a custom image from an instance.

5. Set the NVMe Drive parameter of the new custom image to Supported.

   For more information, see Modify the attributes and tags of an image.

6. (Optional) Use the custom image that you created to redeploy your business. For example, you can create an ECS instance from the custom image.

   For more information, see Create an instance by using a custom image. When you create an ECS instance, select an instance type that supports NVMe.

   Note

   After you redeploy your business, we recommend that you delete the old custom image at the earliest opportunity to prevent resource waste. For more information, see Delete a custom image.

References

• You can attach only Enterprise SSDs (ESSDs) and ESSD AutoPL disks to NVMe-based ECS instances. An ESSD or an ESSD AutoPL disk can be attached to multiple ECS instances to share data. For more information, see Enable multi-attach.

• If the io_timeout parameters of NVMe disks on an ECS instance are improperly configured, the NVMe disks become unavailable. For information about how to resolve the issue, see What do I do if an NVMe disk on a Linux ECS instance is unavailable due to an invalid I/O timeout parameter?