All Products
Search
Document Center

Elastic Compute Service:Adapt Linux custom images to NVMe-based system disks

Last Updated:Feb 26, 2024

Some Elastic Compute Service (ECS) instance families, such as ecs.g7se, ecs.c7se, and ecs.r7se, allow disks to be attached based on the Non-Volatile Memory Express (NVMe) protocol. If you want to use a custom image to create stable instances of such instance families, make sure that the custom image was created based on an operating system that contains required NVMe-related settings.

Background information

For information about instance families that allow disks to be attached based on the NVMe protocol, see Overview of instance families.

Before you configure NVMe-related settings on an instance and then use the instance to create a custom image, we recommend that you create snapshots for the system disk and data disks of the instance to back up data. For more information, see Create a snapshot for a disk.

This topic describes how to configure NVMe-related settings in the following operating systems:

Cloud Assistant of Alibaba Cloud provides a plug-in that can be used to automatically configure NVMe-related settings in supported operating systems. For more information, see Use the Cloud Assistant plug-in to automatically configure NVMe-related settings.

CentOS

  1. Connect to the CentOS instance from which you want to create a custom image.

    For more information, see Connection method overview.

  2. Configure NVMe-related settings based on the configurations of your operating system.

    1. Run the following command to check whether the kernel has loaded the NVMe driver:

      cat /boot/config-`uname -r` | grep -i nvme | grep -v "^#"

      A command output similar to the following one is returned. Check whether the value of CONFIG_BLK_DEV_NVME is y, which indicates that the operating system can be started on instance families that support the NVMe protocol. If yes, skip steps b and c and proceed to step d. If not, perform the following steps in sequence.centos1

    2. Run the following command to check whether the initial Resource Access Management (RAM) file system (initramfs) contains the NVMe driver:

      lsinitrd /boot/initramfs-`uname -r`.img | grep -i nvme | awk '{print $NF}'

      Check whether a command output similar to the following one is returned, which indicates that the operating system can be started on instance families that support the NVMe protocol. If yes, skip step c and proceed to step d. If not, perform the following steps in sequence.centos2

    3. Run the following commands in sequence to configure the initramfs to support the NVMe driver:

      mkdir -p /etc/dracut.conf.d
      echo 'add_drivers+=" nvme nvme-core nvme-fabrics nvme-fc nvme-rdma nvme-loop nvmet nvmet-fc nvme-tcp "' >/etc/dracut.conf.d/nvme.conf
      dracut -v -f
      Note

      If the dracut tool is not installed on your operating system, run the yum -y install dracut command to install the tool.

    4. Add NVMe-related timeout parameters in GRand Unified Bootloader (GRUB).

      1. Run the following command to open the grub file:

        vi /etc/default/grub
      2. Press the I key to enter Insert mode. On the GRUB_CMDLINE_LINUX= line, add the nvme_core.io_timeout and nvme_core.admin_timeout parameters and set them both to 4294967295,

        as shown in the following figure.centos3

        Note

        If the grub file already contains the preceding parameter settings, you do not need to add the parameters again.

      3. Press the Esc key to exit Insert mode. Then, enter :wq and press the Enter key to save and close the file.

    5. Apply the GRUB configurations.

      Run one of the following commands based on the boot mode of your operating system:

      • Legacy boot mode

        grub2-mkconfig -o /boot/grub2/grub.cfg
      • Unified Extensible Firmware Interface (UEFI) boot mode

        grub2-mkconfig -o /boot/efi/EFI/centos/grub.cfg
    6. Rerun the following command to check NVMe-related information:

      lsinitrd /boot/initramfs-`uname -r`.img | grep -i nvme | awk '{print $NF}'

      The following command output indicates that the operating system can be started on instance families that support the NVMe protocol.centos2

  3. Create a custom image from the CentOS instance.

    After you configure NVMe-related settings in the operating system of your CentOS instance, you can create a custom image from the instance. For information about how to create a custom image from an instance, see Create a custom image from an instance.

    Note

    For a custom image that you created in an on-premises environment, you can use the image import feature to import the image to Alibaba Cloud. For more information, see Import a Linux image.

    It takes time to create a custom image. Wait until the custom image is created and enters the Available state. Then, proceed to the next step.

  4. Create an instance from the custom image that you created.

    For information about how to create an instance from a custom image, see Create an ECS instance by using a custom image. When you create an instance from a custom image, select an instance type that meets your business requirements and allows disks to be attached based on the NVMe protocol.

  5. Connect to the new instance.

    For more information, see Connection method overview.

  6. Run the following commands in sequence on the new instance to check whether the NVMe-related settings work as expected.

    1. Run the following command to check the information of the disks that support the NVMe protocol:

      lsblk
    2. Run the following command to check whether the /proc/cmdline file contains NVMe-related timeout parameter settings:

      cat /proc/cmdline

      The following command output indicates that the custom image is adapted to the NVMe-based system disk.centos-g7se

Debian

  1. Connect to the Debian instance from which you want to create a custom image.

    For more information, see Connection method overview.

  2. Configure NVMe-related settings.

    1. Run the following command to check whether the initrd contains the NVMe driver:

      By default, the initrd in each Debian operating system loads the NVMe driver.

      lsinitramfs /boot/initrd.img-`uname -r` | grep -i nvme

      A command similar to the following one is returned.

    2. Add NVMe-related timeout parameters in GRUB.

      1. Run the following command to open the /etc/default/grub file:

        vi /etc/default/grub
      2. Press the I key to enter Insert mode. On the GRUB_CMDLINE_LINUX= line, add the nvme_core.multipath, nvme_core.io_timeout, and nvme_core.admin_timeout parameters. Then, set nvme_core.multipath to n and nvme_core.io_timeout and nvme_core.admin_timeout both to 4294967295,

        as shown in the following figure.debian2

        Note

        If the grub file already contains the preceding parameter settings, you do not need to add the parameters again.

      3. Press the Esc key to exit Insert mode. Then, enter :wq and press the Enter key to save and close the file.

    3. Apply the GRUB configurations.

      Run one of the following commands based on the boot mode of your operating system:

      • Legacy boot mode

        grub-mkconfig -o /boot/grub/grub.cfg
      • UEFI boot mode

        grub-mkconfig -o /boot/efi/EFI/debian/grub.cfg
      Note

      Alternatively, you can run the following command to execute the script provided by Debian to apply the GRUB configurations. The boot mode of the operating system is irrelevant to the script. You can use the script regardless of whether the boot mode of your operating system is Legacy or UEFI.

      update-grub2
  3. Create a custom image from the CentOS instance.

    After you configure NVMe-related settings in the operating system of your CentOS instance, you can create a custom image from the instance. For information about how to create a custom image from an instance, see Create a custom image from an instance.

    Note

    For a custom image that you created in an on-premises environment, you can use the image import feature to import the image to Alibaba Cloud. For more information, see Import a Linux image.

    It takes time to create a custom image. Wait until the custom image is created and enters the Available state. Then, proceed to the next step.

  4. Create an instance from the custom image that you created.

    For information about how to create an instance from a custom image, see Create an ECS instance by using a custom image. When you create an instance from a custom image, select an instance type that meets your business requirements and allows disks to be attached based on the NVMe protocol.

  5. Connect to the new instance.

    For more information, see Connection method overview.

  6. Run the following commands in sequence on the new instance to check whether the NVMe-related settings work as expected.

    1. Run the following command to check the information of the disks that support the NVMe protocol:

      lsblk
    2. Run the following command to check whether the /proc/cmdline file contains NVMe-related timeout parameter settings:

      cat /proc/cmdline

      The following command output indicates that the custom image is adapted to the NVMe-based system disk.debian3

Ubuntu

  1. Connect to the Ubuntu instance from which you want to create a custom image.

    For more information, see Connection method overview.

  2. Configure NVMe-related settings.

    1. Run the following command to check whether the initrd contains the NVMe driver:

      By default, the initrd in each Ubuntu operating system loads the NVMe driver.

      lsinitramfs /boot/initrd.img-`uname -r` | grep -i nvme

      A command similar to the following one is returned.ubuntu1

    2. Add NVMe-related timeout parameters in GRUB.

      1. Run the following command to open the /etc/default/grub file:

        vi /etc/default/grub
      2. Press the I key to enter Insert mode. On the GRUB_CMDLINE_LINUX= line, add the nvme_core.multipath, nvme_core.io_timeout, and nvme_core.admin_timeout parameters. Then, set nvme_core.multipath to n and nvme_core.io_timeout and nvme_core.admin_timeout both to 4294967295,

        as shown in the following figure.ubuntu2

        Note

        If the grub file already contains the preceding parameter settings, you do not need to add the parameters again.

      3. Press the Esc key to exit Insert mode. Then, enter :wq and press the Enter key to save and close the file.

    3. Apply the GRUB configurations.

      Run one of the following commands based on the boot mode of your operating system:

      • Legacy boot mode

        grub-mkconfig -o /boot/grub/grub.cfg
      • UEFI boot mode

        grub-mkconfig -o /boot/efi/EFI/ubuntu/grub.cfg
      Note

      Alternatively, you can run the following command to execute the script provided by Ubuntu to apply the GRUB configurations. The boot mode of the operating system is irrelevant to the script. You can use the script regardless of whether the boot mode of your operating system is Legacy or UEFI.

      update-grub2
  3. Create a custom image from the CentOS instance.

    After you configure NVMe-related settings in the operating system of your CentOS instance, you can create a custom image from the instance. For information about how to create a custom image from an instance, see Create a custom image from an instance.

    Note

    For a custom image that you created in an on-premises environment, you can use the image import feature to import the image to Alibaba Cloud. For more information, see Import a Linux image.

    It takes time to create a custom image. Wait until the custom image is created and enters the Available state. Then, proceed to the next step.

  4. Create an instance from the custom image that you created.

    For information about how to create an instance from a custom image, see Create an ECS instance by using a custom image. When you create an instance from a custom image, select an instance type that meets your business requirements and allows disks to be attached based on the NVMe protocol.

  5. Connect to the new instance.

    For more information, see Connection method overview.

  6. Run the following commands in sequence on the new instance to check whether the NVMe-related settings work as expected.

    1. Run the following command to check the information of the disks that support the NVMe protocol:

      lsblk
    2. Run the following command to check whether the /proc/cmdline file contains NVMe-related timeout parameter settings:

      cat /proc/cmdline

      The following command output indicates that the custom image is adapted to the NVMe-based system disk.ubuntu3

SUSE Linux Enterprise Server (SLES)

  1. Connect to the SLES instance from which you want to create a custom image.

    For more information, see Connection method overview.

  2. Configure NVMe-related settings based on the configurations of your operating system.

    1. Run the following command to check whether the kernel has loaded the NVMe driver:

      cat /boot/config-`uname -r` | grep -i nvme | grep -v "^#"

      A command output similar to the following one is returned. Check whether the value of CONFIG_BLK_DEV_NVME is y, which indicates that the operating system can be started on instance families that support the NVMe protocol. If yes, skip steps b and c and proceed to step d. If not, perform the following steps in sequence.suselinux1

    2. Run the following command to check whether the initrd contains the NVMe driver:

      lsinitrd /boot/initrd-`uname -r` | grep -i nvme | awk '{print $NF}'

      Check whether a command output similar to the following one is returned, which indicates that the operating system can be started on instance families that support the NVMe protocol. If yes, skip step c and proceed to step d. If not, perform the following steps in sequence.suselinux2

    3. Run the following commands in sequence to configure the initrd to support the NVMe driver:

      mkdir -p /etc/dracut.conf.d
      echo 'add_drivers+=" nvme nvme-core nvme-fabrics nvme-fc nvme-rdma nvme-loop nvmet nvmet-fc nvme-tcp "' >/etc/dracut.conf.d/nvme.conf
      dracut -v -f
      Note

      If the dracut tool is not installed on your operating system, run the zypper install -y dracut command to install the tool.

    4. Add NVMe-related timeout parameters in GRUB.

      1. Run the following command to open the grub file:

        vi /etc/default/grub
      2. Press the I key to enter Insert mode. On the GRUB_CMDLINE_LINUX= line, add the nvme_core.io_timeout and nvme_core.admin_timeout parameters and set them both to 4294967295,

        as shown in the following figure.suselinux3

        Note

        If the grub file already contains the preceding parameter settings, you do not need to add the parameters again.

      3. Press the Esc key to exit Insert mode. Then, enter :wq and press the Enter key to save and close the file.

    5. Apply the GRUB configurations.

      Run one of the following commands based on the boot mode of your operating system:

      • Legacy boot mode

        grub2-mkconfig -o /boot/grub2/grub.cfg
      • UEFI boot mode

        grub2-mkconfig -o /boot/efi/EFI/sles/grub.cfg
    6. Rerun the following command to check NVMe-related information:

      lsinitrd /boot/initrd-`uname -r` | grep -i nvme | awk '{print $NF}'

      The following command output indicates that the operating system can be started on instance families that support the NVMe protocol.suselinux2

  3. Create a custom image from the CentOS instance.

    After you configure NVMe-related settings in the operating system of your CentOS instance, you can create a custom image from the instance. For information about how to create a custom image from an instance, see Create a custom image from an instance.

    Note

    For a custom image that you created in an on-premises environment, you can use the image import feature to import the image to Alibaba Cloud. For more information, see Import a Linux image.

    It takes time to create a custom image. Wait until the custom image is created and enters the Available state. Then, proceed to the next step.

  4. Create an instance from the custom image that you created.

    For information about how to create an instance from a custom image, see Create an ECS instance by using a custom image. When you create an instance from a custom image, select an instance type that meets your business requirements and allows disks to be attached based on the NVMe protocol.

  5. Connect to the new instance.

    For more information, see Connection method overview.

  6. Run the following commands in sequence on the new instance to check whether the NVMe-related settings work as expected.

    1. Run the following command to check the information of the disks that support the NVMe protocol:

      lsblk
    2. Run the following command to check whether the /proc/cmdline file contains NVMe-related timeout parameter settings:

      cat /proc/cmdline

      The following command output indicates that the custom image is adapted to the NVMe-based system disk.suselinux4

Use the Cloud Assistant plug-in to automatically configure NVMe-related settings

Cloud Assistant provides the ecs_nvme_config plug-in to help you configure NVMe-related settings in operating systems. The ecs_nvme_config plug-in supports the following operating systems:

  • Alibaba Cloud Linux

  • Anolis OS

  • CentOS/CentOS Stream

  • Debian

  • Ubuntu

  • OpenSUSE

  • SUSE Linux Enterprise Server

  • Red Hat Enterprise Linux

  • Fedora

  • Rocky Linux

  • AlmaLinux

The ecs_nvme_config plug-in does not support the following operating system versions:

  • CentOS and Red Hat Enterprise Linux: versions earlier than CentOS 6.6 and Red Hat Enterprise Linux 6.6

  • Debian: versions earlier than Debian 9

  • Ubuntu: versions earlier than Ubuntu 16

  • openSUSE: versions earlier than openSUSE 42

  • SLES: versions earlier than SLES 11.4

  1. Connect to the instance from which you want to create a custom image.

    For more information, see Connection method overview.

  2. Run the following command to check whether Cloud Assistant Agent is installed on the instance and whether the installed Cloud Assistant Agent provides the ecs_nvme_config plug-in.

    acs-plugin-manager --list

    The following command output indicates that Cloud Assistant Agent is installed. If Cloud Assistant Agent is not installed on the instance, the command output contains no information about Cloud Assistant Agent. In this case, install Cloud Assistant Agent first. For more information, see Install Cloud Assistant Agent. 云助手插件

    If you find in the command output a list of Cloud Assistant plug-ins that exclude the ecs_nvme_config plug-in, the plug-in is unavailable in the current region and zone. In this case, use a method applicable to your operating system to manually configure NVMe-related settings.

    If Cloud Assistant Agent is installed and provides the ecs_nvme_config plug-in on your instance, proceed to the next step.

  3. Use the ecs_nvme_config plug-in to configure NVMe-related settings.

    1. Run the following command to use the ecs_nvme_config plug-in to check whether the instance has the NVMe module and whether the instance supports the NVMe module:

      acs-plugin-manager --exec --plugin ecs_nvme_config --params --check

      Command output description:

      • If the following SUCCESS message appears in the command output, you can create a custom image without the need to configure NVMe-related settings:

        [SUCCESS]  Summary: Your image can Runnig on nvme instance
      • If an [ERROR] message appears in the command output, proceed to the next step.

    2. Configure the NVMe driver and relevant parameters based on the error message in the command output.

      Sample error message:

      [ERROR]  1.initrd/initramfs not has nvme module, Please run acs-plugin-manager --exec --plugin ecs_nvme_config --params -f/--fix to enable nvme;

      Run the following command to configure NVMe-related settings based on the error message:

      acs-plugin-manager --exec --plugin ecs_nvme_config --params --fix
    3. After the NVMe-related settings are configured, run the following command to restart your instance:

      reboot
    4. After the instance is restarted, run the following command to check whether the NVMe module is configured:

      acs-plugin-manager --exec --plugin ecs_nvme_config --params --check

      Sample success output:

      [OK]  1.initrd/initramfs already contain nvme module;
      
      [OK]  2.fstab file looks fine and does not contain any device names;
      
      [OK]  3.The nvme parameters already included.
      
      [SUCCESS]  Summary: Your image can Runnig on nvme instance
  4. Create a custom image from the CentOS instance.

    After you configure NVMe-related settings in the operating system of your CentOS instance, you can create a custom image from the instance. For information about how to create a custom image from an instance, see Create a custom image from an instance.

    Note

    For a custom image that you created in an on-premises environment, you can use the image import feature to import the image to Alibaba Cloud. For more information, see Import a Linux image.

    It takes time to create a custom image. Wait until the custom image is created and enters the Available state. Then, proceed to the next step.

  5. Create an instance from the custom image that you created.

    For information about how to create an instance from a custom image, see Create an ECS instance by using a custom image. When you create an instance from a custom image, select an instance type that meets your business requirements and allows disks to be attached based on the NVMe protocol.

  6. Connect to the new instance.

    For more information, see Connection method overview.

  7. Run the following commands in sequence on the new instance to check whether the NVMe-related settings work as expected.

    1. Run the following command to check the information of the disks that support the NVMe protocol:

      lsblk
    2. Run the following command to check whether the /proc/cmdline file contains NVMe-related timeout parameter settings:

      cat /proc/cmdline

      The following command output indicates that the custom image is adapted to the NVMe-based system disk.centos-g7se