All Products
Search
Document Center

Create a hybrid cloud cluster

Last Updated: Sep 17, 2021

A hybrid cloud cluster is deployed across hybrid environments. Its scheduling nodes and domain account nodes reside on a data center and its compute nodes reside on a data center and on the cloud. You can use on-premises high-performance computing clusters to scale out compute nodes on the cloud and schedule cloud resources and on-premises resources for high-performance computing. This topic describes how to create a hybrid cloud cluster in the Elastic High Performance Computing (E-HPC) console.

Prerequisites

Your data center is connected to Alibaba Cloud by using a VPN gateway, a Cloud Enterprise Network (CEN) instance, or an Express Connect circuit. For more information, see Create a CEN instance, Create a VPN gateway, and What is a connection over an Express Connect circuit?

Background information

The management nodes of the on-premises high-performance computing clusters have the following environment requirements:

  • Operating system: Linux CentOS 7.2, 7.3, 7.4, 7.5, 7.6, or 8.0

  • Scheduler type: the Deadline scheduler or a custom scheduler

  • Account management service: a custom account management service

Procedure

The procedure for creating and using a hybrid cloud cluster contain the following steps:

  1. Step 1: Create a hybrid cloud cluster

    When you create a hybrid cloud cluster, you need to configure the hardware settings, software settings, and basic settings of the compute nodes.

  2. Step 2: Scale out a hybrid cloud cluster

    After you create a hybrid cloud cluster, the cluster does not contain compute nodes. To use the cluster, you must scale it out.

    Before a scale-out, prepare a custom image based on your needs. The Deadline scheduler or a custom scheduler must be installed on the custom image. For more information, see Create a custom image from an instance.

Step 1: Create a hybrid cloud cluster

  1. Log on to the E-HPC console.

  2. In the upper-left corner, select the region.

  3. In the left-side navigation pane, click Cluster.

  4. On the Cluster page, click Create Hybrid Cluster.

  5. Configure hardware settings.

    1. In the Hardware Configurations step, configure the hardware settings.

      Parameter

      Description

      Availability Zone

      The zone of the compute nodes in the cluster.

      Pricing Model

      The billing method of the compute nodes in the cluster. The billing method does not apply to elastic IP addresses and NAS file systems.

      • Subscription: You can purchase or renew compute nodes by week, month, or year.

      • Pay-As-You-Go: You are charged for compute nodes by the hour.

      • Preemptible Instance: Preemptible instances are billed at lower prices compared with pay-as-you-go instances. The price of a preemptible instance fluctuates based on the supply and demand for its instance type. A preemptible instance is billed based on its actual duration of use.

      For more information, see Billing method overview.

      Compute Node

      Compute nodes are used to run high-performance computing jobs. The overall performance of an E-HPC cluster depends on the compute node settings.

      Preinstalled Scheduler

      You must turn on the switch.

      Schedulers are software that handle multiple jobs. A scheduler is deployed on an E-HPC cluster.

      Local Storage

      You must turn on the switch.

      If you turn on the switch, an on-premises file system is mounted on the cluster. This way, all cluster nodes can access data stored in the file system.

      Local Cluster Storage Mount Point

      The mount target of the on-premises file system.

      Mount Path

      The mount path of the on-premises file system.

      Local Path

      The path in which the on-premises file system is mounted on the nodes on the cloud.

    2. Click Advanced Configurations. In the Advanced Configurations section, specify parameters based on your needs.

      Parameter

      Description

      Resource Group

      The resource group where the cluster nodes reside. You can use the resource group to manage multiple cluster nodes that belong to your account in a centralized manner.

      VPC

      The virtual private cloud (VPC) where the cluster resides. Different VPCs do not affect each other. You can create and manage E-HPC clusters in a VPC.

      • If you do not specify these parameters, the first VPC and vSwitch in the VPC and vSwitch drop-down lists are selected by default. Make sure that the number of available IP addresses is greater than that of cluster nodes.

      • If no VPC and vSwitch are available, a VPC whose CIDR block is 192.168.0.0/16 and a vSwitch whose CIDR block is 192.168.0.0/20 are automatically created.

      You can also click Create VPC and Create vSwitch (for subnet) to create a VPC and a vSwitch. For more information, see Work with VPCs and Work with vSwitches.

      vSwitch

      Create Security Group

      Select whether to create a security group.

      • If you turn on the switch, you need to enter a new security group name in the Security Group Name field.

      • If you turn off the switch, you need to select an existing security group in the Select Security Group drop-down list.

      Storage

      Hybrid cloud clusters do not support shared storage.

  6. Click Next.

    Parameter

    Description

    Image Type

    An image provides the information that all cluster nodes require. Different image types apply to different operating systems. The operating systems of all the nodes in a cluster are the same.

    You can select Public Image, Custom Image, or Shared Image.

    Custom images have the following limits:

    • E-HPC allows you to modify only an image provided by Alibaba Cloud. You cannot upload an on-premises image.

    • A custom image that is created based on an E-HPC cluster cannot be used to scale out another E-HPC cluster. Otherwise, compute nodes may not run as expected after the current cluster is created.

    • If you modify the yum source configurations of the operating system in the custom image, the cluster cannot be created or scaled out.

    • The mount directory of the custom image cannot be the /home directory or /opt directory.

    Image

    An image provides the information that all cluster nodes require. The information includes the operating system, pre-installed software, and deployed business or application data of a cluster.

    Only CentOS and Windows Server images are supported. The image types displayed on the Create Hybrid Cloud page of the console prevail.

    Scheduler

    Schedulers are software that handle multiple jobs. A scheduler is deployed on an E-HPC cluster. Hybrid cloud clusters support only the Deadline scheduler and custom schedulers. PBS and Open Grid Scheduler are not supported.

  7. Click Next.

    Parameter

    Description

    Cluster Name

    The name of the cluster. The cluster name is displayed on the Cluster page.

    Logon Password

    The password of the cluster. This password is used when you remotely use SSH to access the logon node of the cluster. The username is root.

    Repeat Password

    Enter the password again.

  8. In the Configuration List section, check the settings that you configured. Read and select Alibaba Cloud International Website Product Terms of Service, and click OK.

    Navigate to the Cluster page. If the hybrid cloud cluster enters the Running state, it is created.

Step 2: Scale out a hybrid cloud cluster

  1. On the Cluster page, find the cluster that you want to scale out. Click Resize.

  2. In the Resize panel, complete the basic configurations.

    For more information, see Scale out a cluster. Set the following parameters:

    • Image Type: Select Custom Image.

    • Image: Select the custom image that you have created.

  3. Check the current settings. Read and select Alibaba Cloud International Website Product Terms of Service, and click Buy Now.

    After you scale out the cluster, you can query the statuses of scale-out nodes. In the left-side navigation pane, choose Resource Management > Nodes. On the Nodes page, select the cluster from the Cluster drop-down list and Compute Node from the Node Type drop-down list. If the nodes are in the Running state, the cluster is scaled out.