A hybrid cloud cluster is deployed across hybrid environments. Its scheduling nodes and domain account nodes reside on a data center and its compute nodes reside on a data center and on the cloud. You can use on-premises high-performance computing clusters to scale out compute nodes on the cloud and schedule cloud resources and on-premises resources for high-performance computing. This topic describes how to create a hybrid cloud cluster in the Elastic High Performance Computing (E-HPC) console.
Prerequisites
A virtual private cloud (VPC) and a vSwitch are created. For more information, see Work with VPCs and Work with vSwitches.
A service-linked role for E-HPC is created. The first time you log on to the E-HPC console, you are prompted to create a service-linked role for E-HPC.
Your data center is connected to Alibaba Cloud by using a VPN gateway, a Cloud Enterprise Network (CEN) instance, or an Express Connect circuit. For more information, see Create a CEN instance, Create a VPN gateway, and What is a connection over an Express Connect circuit?
Background information
The management nodes of the on-premises high-performance computing clusters have the following environment requirements:
Operating system: Linux CentOS 7.2, 7.3, 7.4, 7.5, 7.6, or 8.0
Scheduler type: the Deadline scheduler or a custom scheduler
Account management service: a custom account management service
Procedure
The procedure for creating and using a hybrid cloud cluster contain the following steps:
Step 1: Create a hybrid cloud cluster
When you create a hybrid cloud cluster, you need to configure the hardware settings, software settings, and basic settings of the compute nodes.
Step 2: Scale out a hybrid cloud cluster
After you create a hybrid cloud cluster, the cluster does not contain compute nodes. To use the cluster, you must scale it out.
Before a scale-out, prepare a custom image based on your needs. The Deadline scheduler or a custom scheduler must be installed on the custom image. For more information, see Create a custom image from an instance.
Step 1: Create a hybrid cloud cluster
Log on to the E-HPC console.
In the top navigation bar, select a region.
In the left-side navigation pane, click Cluster.
On the Cluster page, click Create Hybrid Cluster.
Configure hardware settings.
In the Hardware Configurations step, configure the hardware settings.
Parameter
Description
Availability Zone
The zone of the compute nodes in the cluster.
Pricing Model
The billing method of the compute nodes in the cluster. The billing method does not apply to elastic IP addresses and NAS file systems.
Subscription: You can purchase or renew compute nodes by week, month, or year.
Pay-As-You-Go: You are charged for compute nodes by the hour.
Preemptible Instance: Preemptible instances are billed at lower prices compared with pay-as-you-go instances. The price of a preemptible instance fluctuates based on the supply and demand for its instance type. A preemptible instance is billed based on its actual duration of use.
For more information, see Billing method overview.
Compute Node
Compute nodes are used to run high-performance computing jobs. The overall performance of an E-HPC cluster depends on the compute node settings.
Preinstalled Scheduler
You must turn on the switch.
Schedulers are software that handle multiple jobs. A scheduler is deployed on an E-HPC cluster.
Local Storage
You must turn on the switch.
If you turn on the switch, an on-premises file system is mounted on the cluster. This way, all cluster nodes can access data stored in the file system.
Local Cluster Storage Mount Point
The mount target of the on-premises file system.
Mount Path
The mount path of the on-premises file system.
Local Path
The path in which the on-premises file system is mounted on the nodes on the cloud.
Click Advanced Configurations. In the Advanced Configurations section, specify parameters based on your needs.
Parameter
Description
Resource Group
The resource group where the cluster nodes reside. You can use the resource group to manage multiple cluster nodes that belong to your account in a centralized manner.
VPC
The VPC where the cluster resides. Different VPCs do not affect each other. You can create and manage E-HPC clusters in a VPC.
If you do not specify these parameters, the first VPC and vSwitch in the VPC and vSwitch drop-down lists are selected by default. Make sure that the number of available IP addresses is greater than that of cluster nodes.
You can also click Create VPC and Create vSwitch (for subnet) to create a VPC and a vSwitch. For more information, see Work with VPCs and Work with vSwitches.
vSwitch
Create Security Group
Select whether to create a security group.
If you turn on the switch, you need to enter a new security group name in the Security Group Name field.
If you turn off the switch, you need to select an existing security group in the Select Security Group drop-down list.
Storage
Hybrid cloud clusters do not support shared storage.
Click Next.
Parameter
Description
Image Type
An image provides the information that all cluster nodes require. Different image types apply to different operating systems. The operating systems of all the nodes in a cluster are the same.
You can select Public Image, Custom Image, or Shared Image.
Custom images have the following limits:
E-HPC allows you to modify only an image provided by Alibaba Cloud.
A custom image that is created based on an E-HPC cluster cannot be modified. Otherwise, compute nodes may not run as expected after the current cluster is created.
You cannot modify the yum source configurations of the operating system in a custom image. Otherwise, the cluster cannot be created or scaled out.
The mount directory of a custom image cannot be the /home directory or /opt directory.
Image
An image provides the information that all cluster nodes require. The information includes the operating system, pre-installed software, and deployed business or application data of a cluster.
Only CentOS and Windows Server images are supported. The image types displayed on the Create Hybrid Cloud page of the console prevail.
Scheduler
Schedulers are software that handle multiple jobs. A scheduler is deployed on an E-HPC cluster. Hybrid cloud clusters support only the Deadline scheduler and custom schedulers. PBS and Open Grid Scheduler are not supported.
Click Next.
Parameter
Description
Cluster Name
The name of the cluster. The cluster name is displayed on the Cluster page.
Logon Password
The password of the cluster. This password is used when you remotely use SSH to access the logon node of the cluster. The username is root.
Repeat Password
Enter the password again.
In the Configuration List section, check the settings that you configured. Read and select Alibaba Cloud International Website Product Terms of Service, and click OK.
Navigate to the Cluster page. If the hybrid cloud cluster enters the Running state, it is created.
Step 2: Scale out a hybrid cloud cluster
On the Cluster page, find the cluster that you want to scale out. Click Resize.
In the Resize panel, complete the basic configurations.
For more information, see Scale out a cluster. Set the following parameters:
Image Type: Select Custom Image.
Image: Select the custom image that you have created.
Check the current settings. Read and select Alibaba Cloud International Website Product Terms of Service, and click Buy Now.
After you scale out the cluster, you can query the statuses of scale-out nodes. In the left-side navigation pane, choose Resource Management > Nodes. On the Nodes page, select the cluster from the Cluster drop-down list and Compute Node from the Node Type drop-down list. If the nodes are in the Running state, the cluster is scaled out.