All Products
Search
Document Center

Create a hybrid cloud cluster

Last Updated: May 27, 2022

A hybrid cloud cluster is deployed across hybrid environments. Its scheduling nodes and domain account nodes reside on a data center and its compute nodes reside on a data center and on the cloud. You can use on-premises high-performance computing clusters to scale out compute nodes on the cloud, and schedule cloud resources and on-premises resources for high-performance computing. This topic describes how to create a hybrid cloud cluster in the Elastic High Performance Computing (E-HPC) console.

Prerequisites

Background information

The management nodes of the on-premises high-performance computing cluster have the following environment requirements:

  • Operating system: CentOS 7.2, 7.3, 7.4, 7.5, 7.6, or 8.0

  • Scheduler type: Deadline, PBS 18.1.1, Open Grid Scheduler (SGE), or a custom scheduler

    Note

    If your cluster uses PBS or SGE, its operating system must be CentOS 7.x.

  • Account management service: a custom account management service

Procedure

To create and use a hybrid cloud cluster, perform the following steps:

  1. Step 1: Create a hybrid cloud cluster

    When you create a hybrid cloud cluster, you need to configure the hardware settings, software settings, and basic settings of the compute nodes.

  2. Step 2: Scale out a hybrid cloud cluster

    After you create a hybrid cloud cluster, the cluster does not contain compute nodes. To use the cluster, you must scale it out.

    Before a scale-out, prepare a custom image based on your needs. The Deadline scheduler or a custom scheduler must be installed on the custom image. For more information, see Create a custom image from an instance.

Step 1: Create a hybrid cloud cluster

  1. Log on to the E-HPC console.

  2. In the upper-left corner of the top navigation bar, select a region.

  3. In the top navigation bar, click Cluster.

  4. On the Cluster page, click Create Hybrid Cluster.

  5. Configure hardware settings.

    1. In the Hardware Configurations step, configure the hardware settings.

      Parameter

      Description

      Availability Zone

      The zone of the compute nodes in the cluster.

      Pricing Model

      The billing method of the compute nodes in the cluster. The billing method does not apply to elastic IP addresses or NAS file systems.

      • Subscription: You can purchase or renew compute nodes by week, month, or year.

      • Pay-As-You-Go: Bills are generated on an hourly basis.

      • Preemptible Instance: Preemptible instances are more cost-effective than pay-as-you-go instances. The price of a preemptible instance fluctuates based on the supply and demand for its instance type. A preemptible instance is billed based on the duration for which you use it.

      For more information, see Billing method overview.

      Compute Node

      Compute nodes are used to run high-performance computing jobs. The overall performance of an E-HPC cluster depends on the compute node settings.

      Proxy Mode

      If you turn on Proxy Mode, the proxy nodes on the cloud manages cloud cluster resources in place of the on-premises cluster.

      Proxy Node

      The instance type of the proxy nodes on the cloud.

      Preinstalled Scheduler

      Schedulers are software that handles multiple jobs. A scheduler is deployed on an E-HPC cluster.

      • If you enable Proxy Mode, Preinstalled Scheduler must be disabled.

      • If you disable Proxy Mode, Preinstalled Scheduler must be enabled.

      Local Storage

      Specifies whether to mount an on-premises file system on the cluster.

      If an on-premises file system is mounted on the cluster, all cluster nodes can access data stored in the file system.

      Local Cluster Storage Mount Point

      The mount target of the on-premises file system.

      Mount Path

      The mount path of the on-premises file system.

      Local Path

      The path in which the on-premises file system is mounted on the nodes on the cloud.

    2. Click Advanced Configurations. In the Advanced Configurations section, specify the parameters based on your needs.

      Parameter

      Description

      Resource Group

      Resource Group

      The resource group to which the cluster nodes belong. You can use the resource group to manage multiple cluster nodes that belong to your account in a centralized manner.

      Network Settings

      VPC

      The VPC where the cluster resides. VPCs are isolated from each other. You can create and manage E-HPC clusters in a VPC.

      If you do not specify these parameters, the first VPC and vSwitch in the VPC and vSwitch drop-down lists are selected by default. Make sure that the number of available IP addresses is greater than that of cluster nodes.

      You can also click Create VPC and Create vSwitch (for subnet) to create a VPC and a vSwitch. For more information, see Work with VPCs and Work with vSwitches.

      vSwitch

      Create Security Group

      Select whether to create a security group.

      • If you turn on Create Security Group, you need to enter a new security group name in the Security Group Name field.

      • If you turn off Create Security Group, you need to select an existing security group in the Select Security Group drop-down list.

      Storage

      Configure by Directory

      Specifies whether to mount Apsara File Storage NAS file systems on the cluster.

      • If you turn on Proxy Mode, you must mount a NAS file system on the cluster.

      • If you turn off Proxy Mode, you can mount only an on-premises file system on the cluster.

      E-HPC stores all user data, scheduler data, and job sharing data on a NAS file system. All nodes in the cluster can access the data.

      • If you turn off Configure By Directory, only one file system is configured for the cluster.

      • If you turn on Configure By Directory, file systems are mounted for the directories of all nodes to improve the shared storage capability of the cluster.

      File System Type

      The type of the NAS file system. Valid values:

      • General Purpose

      • Extreme

      File System ID

      If you do not specify these parameters, the first file system and mount target in the File System ID and Mount Point drop-down lists are selected by default. Make sure that the file system has sufficient mount targets.

      You can also click Create a file system and Create mount point to create a file system and a mount target. For more information, see Create a file system and Manage mount targets.

      Mount Point

      Remote Directory

      The remote directory to which the file system is mounted.

  6. Click Next.

    Parameter

    Description

    Image Type

    An image provides the information that all cluster nodes require. Different image types apply to different operating systems. The operating systems of all the nodes in a cluster are the same.

    You can select Public Image, Custom Image, or Shared Image.

    Custom images have the following limits:

    • E-HPC allows you to modify only an image provided by Alibaba Cloud.

    • A custom image that is created based on an E-HPC cluster cannot be modified. Otherwise, compute nodes may not run as expected after the current cluster is created.

    • You cannot modify the yum source configurations of the operating system in a custom image. Otherwise, the cluster cannot be created or scaled out.

    • The mount directory of a custom image cannot be the /home or /opt directory.

    Image

    An image provides the information that all cluster nodes require. The information includes the operating system, pre-installed software, and deployed business or application data of a cluster.

    Only CentOS and Windows Server images are supported. The image types displayed on the Create Hybrid Cloud page of the console prevail.

    Scheduler

    Schedulers are software that handles multiple jobs. A scheduler is deployed on an E-HPC cluster. The following schedulers are supported based on whether Proxy Mode is enabled:

    • If Proxy Mode is enabled, SGE and PBS are supported.

    • If Proxy Mode is disabled, Deadline and custom schedulers are supported. You can also choose not to install a scheduler.

    Scheduler Node IP Address

    The IP address of the scheduling node.

    Scheduling Node Hostname

    The hostname of the scheduling node.

    Account System

    The domain account service of the scheduling node.

    The cluster and the cluster users are managed based on the domain account service. Valid values: nis and ldap.

    Local Cluster Domain Name

    The domain name of the on-premises cluster.

    Account Node IP Address

    The IP address of the account node.

    Account node host name

    The hostname of the account node.

  7. Click Next.

    Parameter

    Description

    Cluster Name

    The name of the cluster. The cluster name is displayed on the Cluster page.

    Logon Password

    The password of the cluster. This password is used when you remotely use SSH to access the logon node of the cluster. The username is root.

    Note

    If you turn on Proxy Mode, enter the root password of the on-premises cluster.

    Repeat Password

    Enter the password again.

  8. In the Configuration List section, check the settings that you configured. Read and select Alibaba Cloud International Website Product Terms of Service, and click OK.

    Navigate to the Cluster page. If the hybrid cloud cluster enters the Running state, it is created.

Step 2: Scale out a hybrid cloud cluster

  1. On the Cluster page, find the cluster that you want to scale out. Click Resize.

  2. In the Resize panel, complete the basic configurations.

    For more information, see Scale out a cluster. Set the following parameters:

    • Image Type: Select Custom Image.

    • Image: Select the custom image that you have created.

  3. Check the current settings. Read and select Alibaba Cloud International Website Product Terms of Service, and click Buy Now.

    After you scale out the cluster, you can query the status of each scale-out node. In the left-side navigation pane, choose Resource Management > Nodes. On the Nodes page, select the cluster from the Cluster drop-down list and Compute Node from the Node Type drop-down list. If the nodes are in the Running state, the cluster is scaled out.

References

CreateHybridCluster