A hybrid cloud cluster is deployed across hybrid environments. Its scheduling nodes and domain account nodes reside on a data center and its compute nodes reside on a data center and on the cloud. You can use on-premises high-performance computing clusters to scale out compute nodes on the cloud and schedule cloud resources and on-premises resources for high-performance computing. This topic describes how to create a hybrid cloud cluster in the Elastic High Performance Computing (E-HPC) console.
A service-linked role for E-HPC is created. The first time you log on to the E-HPC console, you are prompted to create a service-linked role for E-HPC.
Your data center is connected to Alibaba Cloud by using a VPN gateway, a Cloud Enterprise Network (CEN) instance, or an Express Connect circuit. For more information, see Create a CEN instance, Create a VPN gateway, and What is a connection over an Express Connect circuit?
The management nodes of the on-premises high-performance computing clusters have the following environment requirements:
Operating system: Linux CentOS 7.2, 7.3, 7.4, 7.5, 7.6, or 8.0
Scheduler type: the Deadline scheduler or a custom scheduler
Account management service: a custom account management service
The procedure for creating and using a hybrid cloud cluster contain the following steps:
When you create a hybrid cloud cluster, you need to configure the hardware settings, software settings, and basic settings of the compute nodes.
After you create a hybrid cloud cluster, the cluster does not contain compute nodes. To use the cluster, you must scale it out.
Before a scale-out, prepare a custom image based on your needs. The Deadline scheduler or a custom scheduler must be installed on the custom image. For more information, see Create a custom image from an instance.
Step 1: Create a hybrid cloud cluster
Log on to the E-HPC console.
In the top navigation bar, select a region.
In the left-side navigation pane, click Cluster.
On the Cluster page, click Create Hybrid Cluster.
Configure hardware settings.
In the Hardware Configurations step, configure the hardware settings.
The zone of the compute nodes in the cluster.
The billing method of the compute nodes in the cluster. The billing method does not apply to elastic IP addresses and NAS file systems.
Subscription: You can purchase or renew compute nodes by week, month, or year.
Pay-As-You-Go: You are charged for compute nodes by the hour.
Preemptible Instance: Preemptible instances are billed at lower prices compared with pay-as-you-go instances. The price of a preemptible instance fluctuates based on the supply and demand for its instance type. A preemptible instance is billed based on its actual duration of use.
For more information, see Billing method overview.
Compute nodes are used to run high-performance computing jobs. The overall performance of an E-HPC cluster depends on the compute node settings.
You must turn on the switch.
Schedulers are software that handle multiple jobs. A scheduler is deployed on an E-HPC cluster.
You must turn on the switch.
If you turn on the switch, an on-premises file system is mounted on the cluster. This way, all cluster nodes can access data stored in the file system.
Local Cluster Storage Mount Point
The mount target of the on-premises file system.
The mount path of the on-premises file system.
The path in which the on-premises file system is mounted on the nodes on the cloud.
Click Advanced Configurations. In the Advanced Configurations section, specify parameters based on your needs.
The resource group where the cluster nodes reside. You can use the resource group to manage multiple cluster nodes that belong to your account in a centralized manner.
The VPC where the cluster resides. Different VPCs do not affect each other. You can create and manage E-HPC clusters in a VPC.
If you do not specify these parameters, the first VPC and vSwitch in the VPC and vSwitch drop-down lists are selected by default. Make sure that the number of available IP addresses is greater than that of cluster nodes.
Create Security Group
Select whether to create a security group.
If you turn on the switch, you need to enter a new security group name in the Security Group Name field.
If you turn off the switch, you need to select an existing security group in the Select Security Group drop-down list.
Hybrid cloud clusters do not support shared storage.
An image provides the information that all cluster nodes require. Different image types apply to different operating systems. The operating systems of all the nodes in a cluster are the same.
You can select Public Image, Custom Image, or Shared Image.
Custom images have the following limits:
E-HPC allows you to modify only an image provided by Alibaba Cloud.
A custom image that is created based on an E-HPC cluster cannot be modified. Otherwise, compute nodes may not run as expected after the current cluster is created.
You cannot modify the yum source configurations of the operating system in a custom image. Otherwise, the cluster cannot be created or scaled out.
The mount directory of a custom image cannot be the /home directory or /opt directory.
An image provides the information that all cluster nodes require. The information includes the operating system, pre-installed software, and deployed business or application data of a cluster.
Only CentOS and Windows Server images are supported. The image types displayed on the Create Hybrid Cloud page of the console prevail.
Schedulers are software that handle multiple jobs. A scheduler is deployed on an E-HPC cluster. Hybrid cloud clusters support only the Deadline scheduler and custom schedulers. PBS and Open Grid Scheduler are not supported.
The name of the cluster. The cluster name is displayed on the Cluster page.
The password of the cluster. This password is used when you remotely use SSH to access the logon node of the cluster. The username is root.
Enter the password again.
In the Configuration List section, check the settings that you configured. Read and select Alibaba Cloud International Website Product Terms of Service, and click OK.
Navigate to the Cluster page. If the hybrid cloud cluster enters the Running state, it is created.
Step 2: Scale out a hybrid cloud cluster
On the Cluster page, find the cluster that you want to scale out. Click Resize.
In the Resize panel, complete the basic configurations.
For more information, see Scale out a cluster. Set the following parameters:
Image Type: Select Custom Image.
Image: Select the custom image that you have created.
Check the current settings. Read and select Alibaba Cloud International Website Product Terms of Service, and click Buy Now.
After you scale out the cluster, you can query the statuses of scale-out nodes. In the left-side navigation pane, choose Resource Management > Nodes. On the Nodes page, select the cluster from the Cluster drop-down list and Compute Node from the Node Type drop-down list. If the nodes are in the Running state, the cluster is scaled out.