We recommend that you install cloud-init on a source server when you create a Linux custom image based on the server. This ensures that Elastic Compute Service (ECS) instances which run the custom image can initialize their system configurations. This topic describes how to install Alibaba Cloud cloud-init and the native cloud-init.

Background information

cloud-init is open source software used by cloud platforms to initialize the system for Linux virtual machines. All major public cloud platforms such as Alibaba Cloud, Amazon Web Services (AWS), Microsoft Azure, and OpenStack support cloud-init. Alibaba Cloud cloud-init initializes system configurations on instance startup and executes user data scripts. The configurations include Network Time Protocol (NTP), software repositories, hostnames, and SSH key pairs. For more information, see cloud-init Documentation.

By default, cloud-init is installed for all Alibaba Cloud public images. We recommend that you install Alibaba Cloud cloud-init on your Linux server based on which to create custom images in the following scenarios, so that instances created from the custom images can automatically initialize system configurations:
  • Linux servers that you plan to migrate to the cloud are not installed with cloud-init.
    Note Proceed with caution when you install Alibaba Cloud cloud-init on servers that you do not plan to migrate to the cloud.
  • Linux servers are installed with cloud-init of versions earlier than 0.7.9.
  • Alibaba Cloud ECS instances are not installed with cloud-init.

cloud-init versions

Different cloud platforms may support different versions of cloud-init. Select an appropriate version and configure an appropriate data source (datasource) for it. The latest version of Alibaba Cloud cloud-init is 19.1.10, and the data source is Aliyun.

Notice After cloud-init is installed, it automatically runs on system startup. If the installed version of cloud-init is not compatible with the operating system of the server or if the data source is not properly configured, cloud-init may not run normally and the instance may start slowly or may not start at all the next time you restart the instance. Therefore, you must select a later version of cloud-init and an appropriate data source such as Aliyun.
When you use cloud-init, take note of the following differences among versions:
  • 0.7.6a: the initial version of Alibaba Cloud cloud-init, which depends on Python 2.7. Some public images based on early versions of operating systems still support cloud-init 0.7.6a. If you need to install cloud-init 0.7.6a for your images, see the (Optional) Install Alibaba Cloud cloud-init 0.7.6a15 section.
    Note The Python community no longer provides technical support for Python 2.7. To avoid potential risks associated with dependency libraries, we recommend that you use later versions of cloud-init.
  • 0.7.9 and earlier: initial versions of the native cloud-init, which are not applicable to initializing ECS instances and must be upgraded.
  • 18: cloud-init 18 and later automatically initialize network configurations. The code for network configuration is BOOTPROTO=dhcp DEVICE=eth0 ONBOOT=yes STARTMODE=auto TYPE=Ethernet USERCTL=no. If you want to customize network configurations after you install cloud-init, see the (Optional) Customize network configurations section.
  • 19.1: Alibaba Cloud public images are being upgraded to be pre-installed with cloud-init 19.1, which depends on Python 3.6.

Check the version of cloud-init

  1. Log on to the source server.
  2. Run the following command to check whether cloud-init is installed:
    which cloud-init

    If the output contains no path information, cloud-init is not installed. Install Alibaba Cloud cloud-init first.

  3. Run the following command to check the version of cloud-init:
    cloud-init --version
    If the returned version is earlier than 0.7.9, install Alibaba Cloud cloud-init 19.1.10.
  4. Back up data on the server.

(Recommended) Install Alibaba Cloud cloud-init 19.1.10

Perform the following operations to download cloud-init 19.1.10, whose data source is Aliyun:

  1. Make sure that the python-pip dependency library is installed on the source server.
    In the examples, python3-pip is used. Run the following commands to install the python-pip dependency library for specific Linux distributions:
    • CentOS and Red Hat Enterprise Linux:
      yum -y install python3-pip
    • Ubuntu and Debian:
      apt-get -y install python3-pip
    • openSUSE and SUSE:
      zypper -n install python3-pip
  2. Run the following command to download Alibaba Cloud cloud-init:
    wget https://ecs-image-tools.oss-cn-hangzhou.aliyuncs.com/cloudinit/cloud-init-19.1.10.tgz
  3. Run the following command to decompress the cloud-init installation package to the current directory:
    tar -zxvf cloud-init-19.1.10.tgz
  4. Run the following commands to go to the cloud-init directory and install the dependency library:
    cd ./cloud-init-19.1.10
    pip3 install -r ./requirements.txt
  5. Run the following command to go to the tools directory of the cloud-init file:
    cd ./tools
  6. Run the following command to execute the deploy.sh script to install cloud-init:
    bash ./deploy.sh <issue> <major_version>
    The following table describes the parameters and their values in the deploy.sh script.
    Parameter Description Example
    <issue> The operating system type. Valid values: centos, redhat, rhel, debian, ubuntu, opensuse, and sles. The parameter values are case-sensitive. sles stands for SUSE or SUSE Linux Enterprise Server. centos
    <major_version> The major version number of the operating system. The major version number of CentOS 7.6 is 7.
    For example, if the current operating system is CentOS 7, you must run the bash ./deploy.sh centos 7 command.
  7. Check whether cloud-init is installed.
    If "description": "success" is returned, Alibaba Cloud cloud-init is installed. Alibaba Cloud cloud-init installed
The following section provides sample shell scripts that are used to install Alibaba Cloud cloud-init for different Linux distributions. Adapt the script to your operating system.
  • CentOS 7/8
    # Check whether the python3-pip dependency library is installed. If not, install it.
    if ! python3 -c 'import setuptools' >& /dev/null; then
      yum -y install python3-pip
    fi
    # Back up cloud-init of an earlier version.
    test -d /etc/cloud && mv /etc/cloud /etc/cloud-old
    # Download and decompress the Alibaba Cloud cloud-init installation package.
    wget https://ecs-image-tools.oss-cn-hangzhou.aliyuncs.com/cloud-init-19.1.10.tgz
    tar -zxvf ./cloud-init-19.1.10.tgz
    # Install cloud-init.
    issue_major=$( cat /etc/redhat-release | grep -Eo '[0-9]+\.?[0-9]+' | head -1 | awk -F'.' '{printf $1}')
    bash ./cloud-init-*/tools/deploy.sh centos "$issue_major"
  • Red Hat Enterprise Linux 7/8
    # Check whether the python3-pip dependency library is installed. If not, install it.
    if ! python3 -c 'import setuptools' >& /dev/null; then
      yum -y install python3-pip
    fi
    # Back up cloud-init of an earlier version.
    test -d /etc/cloud && mv /etc/cloud /etc/cloud-old
    # Download and decompress the Alibaba Cloud cloud-init installation package.
    wget https://ecs-image-tools.oss-cn-hangzhou.aliyuncs.com/cloud-init-19.1.10.tgz
    tar -zxvf ./cloud-init-19.1.10.tgz
    # Install cloud-init.
    issue_major=$( cat /etc/os-release | grep VERSION_ID | grep -Eo '[0-9]+\.?[0-9]+' | head -1 | awk -F'.' '{printf $1}')
    bash ./cloud-init-*/tools/deploy.sh rhel "$issue_major"
  • Ubuntu 16/18/20
    # Check whether the python3-pip dependency library is installed. If not, install it.
    if ! python3 -c 'import setuptools' >& /dev/null; then
      apt-get install python36 python3-pip -y
    fi
    # Back up cloud-init of an earlier version.
    test -d /etc/cloud && mv /etc/cloud /etc/cloud-old
    # Download and decompress the Alibaba Cloud cloud-init installation package.
    wget https://ecs-image-tools.oss-cn-hangzhou.aliyuncs.com/cloud-init-19.1.10.tgz
    tar -zxvf ./cloud-init-19.1.10.tgz
    # Install cloud-init.
    issue_major=$( cat /etc/os-release | grep VERSION_ID | grep -Eo '[0-9]+\.?[0-9]+' | head -1 | awk -F'.' '{printf $1}')
    bash ./cloud-init-*/tools/deploy.sh ubuntu "$issue_major"
  • Debian 9/10
    # Check whether the python3-pip dependency library is installed. If not, install it.
    if ! python3 -c 'import setuptools' >& /dev/null; then
      apt-get -y install python3-pip
    fi
    # Back up cloud-init of an earlier version.
    test -d /etc/cloud && mv /etc/cloud /etc/cloud-old
    # Download and decompress the Alibaba Cloud cloud-init installation package.
    wget https://ecs-image-tools.oss-cn-hangzhou.aliyuncs.com/cloud-init-19.1.10.tgz
    tar -zxvf ./cloud-init-19.1.10.tgz
    # Install cloud-init.
    issue_major=$( cat /etc/os-release | grep VERSION_ID | grep -Eo '[0-9]+\.?[0-9]+' | head -1 | awk -F'.' '{printf $1}')
    bash ./cloud-init-*/tools/deploy.sh debian "$issue_major"
  • SUSE 12/15
    # Check whether the python3-pip dependency library is installed. If not, install it.
    if ! python3 -c 'import setuptools'>& /dev/null; then
      zypper -n install python3-pip
    fi
    # Back up cloud-init of an earlier version.
    test -d /etc/cloud && mv /etc/cloud/etc/cloud-old
    # Download and decompress the Alibaba Cloud cloud-init installation package.
    wget https://ecs-image-tools.oss-cn-hangzhou.aliyuncs.com/cloud-init-19.1.10.tgz
    tar -zxvf ./cloud-init-19.1.10.tgz
    # Install cloud-init.
    issue_major=$( cat /etc/os-release | grep VERSION_ID | grep -Eo '[0-9]+\.?[0-9]+' | head -1 | awk -F'.' '{printf $1}')
    bash ./cloud-init-*/tools/deploy.sh sles "$issue_major"
  • OpenSUSE 15
    # Check whether the python3-pip dependency library is installed. If not, install it.
    if ! python3 -c 'import setuptools'>& /dev/null; then
      zypper -n install python3-pip
    fi
    # Back up cloud-init of an earlier version.
    test -d /etc/cloud && mv /etc/cloud/etc/cloud-old
    # Download and decompress the Alibaba Cloud cloud-init installation package.
    wget https://ecs-image-tools.oss-cn-hangzhou.aliyuncs.com/cloud-init-19.1.10.tgz
    tar -zxvf ./cloud-init-19.1.10.tgz
    # Install cloud-init.
    issue_major=$( cat /etc/os-release | grep VERSION_ID | grep -Eo '[0-9]+\.?[0-9]+' | head -1 | awk -F'.' '{printf $1}')
    bash ./cloud-init-*/tools/deploy.sh opensuse"$issue_major"

(Optional) Install Alibaba Cloud cloud-init 0.7.6a15

Some early operating systems such as CentOS 6, Debian 9, and SUSE Linux Enterprise Server 12 still support cloud-init 0.7.6a15.
Note By default, Alibaba Cloud public images of CentOS 6, Debian 9, and SUSE Linux Enterprise Server 12 are installed with cloud-init-0.7.6a15. If you want to test installed cloud-init-0.7.6a15, first run the mv /etc/cloud/cloud.cfg /etc/cloud/cloud.cfg_bak command to back up files in the images.
  1. Run the following command to check whether the operating system version is CentOS 6, Debian 9, or SUSE Linux Enterprise Server 12:
    cat /etc/issue
  2. Run the following commands to download and decompress the Alibaba Cloud cloud-init 0.7.6a15 installation package:
    wget https://ecs-image-tools.oss-cn-hangzhou.aliyuncs.com/cloud-init-0.7.6a15.tgz
    tar -zxvf cloud-init-0.7.6a15.tgz
  3. Run the following command to go to the tools directory of the cloud-init file:
    cd cloud-init-0.7.6a15/tools/
  4. Run the following command to install cloud-init:
    bash ./deploy.sh <issue> <major_version>
    The following table describes the parameters and their values in the deploy.sh script.
    Parameter Description Example
    <issue> The operating system type. Valid values: centos, debian, and sles. The parameter values are case-sensitive. sles stands for SUSE or SUSE Linux Enterprise Server. centos
    <major_version> The major version number of the operating system. The major version number of CentOS 6.5 is 6.
    For example, if the current operating system is CentOS 6, you must run the bash ./deploy.sh centos 6 command.

(Optional) Install the native cloud-init

  1. Make sure that the Git, Python, and python-pip dependency libraries are installed on the source server.
    In the examples, Git, Python 3.6, and python3-pip are used. Run the following commands to install Git, Python, and python-pip dependency libraries for specific Linux distributions:
    • CentOS and Red Hat Enterprise Linux:
      yum -y install git python36 python3-pip
    • Ubuntu and Debian:
      apt-get -y install git python36 python3-pip
    • openSUSE and SUSE:
      zypper -n install git python36 python3-pip
  2. Run the following command to download the cloud-init source code package from Git:
    git clone https://git.launchpad.net/cloud-init
  3. Run the following command to go to the cloud-init directory:
    cd ./cloud-init
  4. Run the following command to install all the dependency libraries:
    pip3 install -r ./requirements.txt
  5. Run the following command to install cloud-init:
    python3 setup.py install
  6. Modify the cloud.cfg configuration file.
    1. Run the following command to open the configuration file:
      vi /etc/cloud/cloud.cfg
      vi /etc/cloud/cloud.cfg
    2. Change the content that precedes cloud_init_modules: to the following content:
      # Example datasource config
      # The top level settings are used as module
      # and system configuration.
      # A set of users which may be applied and/or used by various modules
      # when a 'default' entry is found it will reference the 'default_user'
      # from the distro configuration specified below
      users:
         - default
      user:
          name: root
          lock_passwd: False
      # If this is set, 'root' will not be able to ssh in and they 
      # will get a message to login instead as the above $user
      disable_root: false
      # This will cause the set+update hostname module to not operate (if true)
      preserve_hostname: false
      syslog_fix_perms: root:root
      datasource_list: [ AliYun ]
      # Example datasource config
      datasource:
          AliYun:
              support_xen: false
              timeout: 5 # (defaults to 50 seconds)
              max_wait: 60 # (defaults to 120 seconds)
      #      metadata_urls: [ 'blah.com' ]
      # The modules that run in the 'init' stage
      cloud_init_modules:

(Optional) Customize network configurations

  1. After cloud-init is installed, open the /etc/cloud/cloud.cfg file.
    vim /etc/cloud/cloud.cfg
  2. Press the I key to enter the edit mode. Add the following disabled configuration before Example datasource config:
    network:
      config: disabled
    Note After the configuration is added, you must manage the network configurations in the /etc/sysconfig/network-scripts/ directory on your own.
    cloud-init-disable-config
  3. Press the Esc key to exit the edit mode. Enter :wq and press the Enter key to save and close the file.

(Optional) Keep the hostname and the /etc/hosts configuration file unchanged

By default, the hostname in the cloud-init configuration file is not retained. If you do not want the hostname and /etc/hosts configuration file of your instance to be modified, you can perform the following operations to modify the cloud-init configuration file.

  1. After cloud-init is installed, open the /etc/cloud/cloud.cfg file.
    vim /etc/cloud/cloud.cfg
  2. Press the I key to enter the edit mode and change preserve_hostname: false topreserve_hostname: true.
    hostname
  3. Press the Esc key to exit the edit mode. Enter :wq and press the Enter key to save and close the file.
    preserve_hostname: true prevents cloud-init from modifying the instance hostname and the /etc/hosts configuration file.

Troubleshooting

  • After I use Python 3 to install cloud-init, the required dependency libraries are absent. What do I do?

    The dependency libraries that are absent may vary based on images. You can use pip to install the libraries and then install cloud-init again.
  • cloud-init is running abnormally and an error message was returned. What do I do?

    If the default software package manager such as YUM and the pip manager have been installed with different versions of dependency libraries, library version conflicts may occur and cause cloud-init to run abnormally. We recommend that you download dependency libraries based on the error message.
    Error message Cause Troubleshooting command
    no setuptools module in python
    The python setuptools module is not installed. Python 3.6 is used in the following examples. Commands for different operating systems:
    • CentOS and Red Hat: yum -y install python3-pip
    • Ubuntu and Debian: apt-get -y install python3-pip
    • openSUSE and SUSE: zypper -n install python3-pip
    File "/root/cloud-init/cloudinit/log.py", line 19, in <module>
          import six
      ImportError: No module named six  )
    The six dependency library is not installed.
    pip3 install six
    File "/root/cloud-init/cloudinit/url_helper.py", line 20, in <module>
          import oauthlib.oauth1 as oauth1
      ImportError: No module named oauthlib.oauth1  )
    The oauthlib dependency library not installed.
    pip3 install oauthlib
    No uninstalled dependency libraries specified The error message is not mapped. Run the following command to install all dependency libraries that are displayed in the requirements.txt file of cloud-init:
    pip3 install -r requirements.txt
  • How do I make sure that a correct version of cloud init is installed?

    After you install cloud-init, you can run the cloud-init --version command to check the version of cloud-init. If the returned version is the one that you want to install, the correct version of cloud-init is installed. For example, you want to install Alibaba Cloud cloud-init 19.1.10. After you install cloud-init and run the cloud-init --version command, the command output contains 19.1.10, which indicates that cloud-init 19.1.10 is installed.
  • cloud-init runs abnormally on an instance after I install a new version of Python 3 on the instance. What do I do?

    If you install a new version of Python 3 such as Python 3.9 on your instance and create a symbolic link (such as ln -s /usr/bin/python3.9 /usr/bin/python3) for the Python 3 version to set it as the default version, cloud-init installed on the instance runs abnormally. For example, when you run the cloud-init --version command, an error is reported.
    $cloud-init --version
    Traceback (most recent call last):
      File "/usr/local/bin/cloud-init", line 33, in <module>
        sys.exit(load_entry_point('cloud-init==19.1.9', 'console_scripts', 'cloud-init')())
      File "/usr/local/bin/cloud-init", line 22, in importlib_load_entry_point
        for entry_point in distribution(dist_name).entry_points
      File "/usr/lib64/python3.9/importlib/metadata.py", line 524, in distribution
        return Distribution.from_name(distribution_name)
      File "/usr/lib64/python3.9/importlib/metadata.py", line 187, in from_name
        raise PackageNotFoundError(name)
    importlib.metadata.PackageNotFoundError: cloud-init
    You can use one of the following methods to solve this problem:
    • Method 1: Use the new version of Python 3 to re-install cloud-init.
    • Method 2: Modify the cloud-init executable to change the Python interpreter path to the path to an earlier Python 3 version. Python 3.6 is used in the following example. Perform the following operation to change the Python interpreter path in the cloud-init executable:
      1. Run the following command to open the cloud-init executable:
        vim   /usr/local/bin/cloud-init
      2. Press the I key to enter the edit mode. Then, at the beginning of the executable, change the content that follows #! to the path to Python 3.6.
        Example of the new content on the line that starts with #!:
        #!/usr/bin/python3.6
      3. Press the Esc key to exit the edit mode. Enter :wq and press the Enter key to save and close the file.
  • After I install cloud-init on an instance, the user data that was specified in instance metadata when I created the instance does not or cannot run. What do I do?

    1. Run the following command on the instance to check whether user data exists in instance metadata:
      curl http://100.100.100.200/latest/user-data
      Check results:
      • If user data exists in instance metadata, the output contains the user data. You must perform other operations to further troubleshoot the problem.
      • If user data does not exist in instance metadata, no output is returned.
    2. Identify the reason why the user data does not or cannot run.
      • Check whether the user data is in a valid format.

        User data must be in a valid format to be run by cloud-init. For example, the first line of the user data must start with #!. For more information, see Manage the user data of Linux instances.

      • Check whether you selected Security Hardening Mode when you created the instance.

        If you selected Security Hardening Mode, the components of cloud-init cannot be initialized, which affects instance configurations such as metadata and user data. For more information, see View instance metadata.

      • Check the execution result of the user data in the /var/log/cloud-init.log file of cloud-init and troubleshoot the problem based the returned error message.
        Example of the user data execution result in the log file:
         util.py[DEBUG]: Running command ['/var/lib/cloud/instance/scripts/part-001'] with allowed return codes [0] (shell=False, capture=False)
      • Check the standard output (stdout) and standard error (stderr) generated while the user data was running.
        In systemd (a system initialization tool of Linux), user data is run by the cloud-final.service system service. You can run the following command to check the stdout and stderr generated while the user data was running. Then, you can troubleshoot the problem based on the command output.
        journalctl -u cloud-final.service
      • If you still cannot identify the reason why the user data does not or cannot run after you perform the preceding operations, we recommend that you copy the user data to your computer and check whether the user data can be run on the computer.

What to do next

  • Linux servers that you plan to migrate to the cloud:

    You can migrate the servers by using the Cloud Migration tool or by importing the custom images. For more information, see Migrate a server to Alibaba Cloud by using the Cloud Migration tool or Import custom images.

  • Alibaba Cloud ECS instances that already run Linux custom images:
    You can restart the system to check whether cloud-init is installed. If cloud-init is installed, the system configures the hostname, software repositories, and NTP. For example, if cloud-init is installed, the network configuration file shows the following information:
    [root@iZbp1ios3psx4hoi******Z ~]# cat /etc/sysconfig/network-scripts/ifcfg-eth0
    # Created by cloud-init on instance boot automatically, do not edit.
    #
    BOOTPROTO=dhcp
    DEVICE=eth0
    ONBOOT=yes
    STARTMODE=auto
    TYPE=Ethernet
    USERCTL=no