Create a hybrid cloud cluster in the E-HPC console - Elastic High Performance Computing

A hybrid cloud cluster is deployed across hybrid environments. Its scheduling nodes and domain account nodes reside on a data center, and its compute nodes reside on a data center and on the cloud. You can use on-premises high-performance computing clusters to scale out compute nodes on the cloud, and schedule cloud resources and on-premises resources for high-performance computing. This topic describes how to create a hybrid cloud cluster in the Elastic High Performance Computing (E-HPC) console.

Prerequisites

A virtual private cloud (VPC) and a vSwitch are created. For more information, see Create and manage a VPC and Create and manage a vSwitch.
A service-linked role for E-HPC is created. The first time you log on to the E-HPC console, you are prompted to create a service-linked role for E-HPC.
Your data center is connected to Alibaba Cloud by using a VPN gateway, a Cloud Enterprise Network (CEN) instance, or an Express Connect circuit. For more information, see Create a CEN instance, Create a VPN gateway, and What is a connection over an Express Connect circuit?

Background information

The management nodes of the on-premises high-performance computing cluster have the following environment requirements:

OS: CentOS 7.2, 7.3, 7.4, 7.5, 7.6, or 8.0
Scheduler type: Deadline, Portable Batch System (PBS) 18.1.1, Open Grid Scheduler (SGE), or a custom scheduler
Note
If your cluster uses PBS or SGE, its operating system must be CentOS 7.x.
Account management service: a custom account management service

Procedure

To create and use a hybrid cloud cluster, perform the following steps:

Step 1: Create a hybrid cloud cluster
When you create a hybrid cloud cluster, you need to configure the hardware settings, software settings, and basic settings of the compute nodes.
Step 2: Scale up the hybrid cloud cluster
After you create a hybrid cloud cluster, the cluster does not contain compute nodes. To use the cluster, you must scale it up.
Before you scale up the hybrid cloud cluster, prepare a custom image based on your requirements. The Deadline scheduler or a custom scheduler must be installed in the custom image. For more information, see Create a custom image from an instance.

Step 1: Create a hybrid cloud cluster

Log on to the E-HPC console.
In the top navigation bar, select a region.
In the top navigation bar, click Cluster.
On the Cluster page, click Create Hybrid Cluster.

Configure hardware settings.

Specify basic parameters.

Parameter	Description
Availability Zone	The zone in which the compute nodes in the cluster reside.
Pricing Model	The billing method of the compute nodes in the cluster. The billing method does not apply to elastic IP addresses (EIPs) or Apsara File Storage NAS (NAS) file systems. Subscription: You can purchase or renew compute nodes by week, month, or year. Pay-As-You-Go: Bills are generated on an hourly basis. Preemptible Instance: Preemptible instances are more cost-effective than pay-as-you-go instances. The price of a preemptible instance fluctuates based on the supply and demand for its instance type. A preemptible instance is billed based on the duration for which you use it. For more information, see ECS billing method overview.
Compute Node	Compute nodes are used to run high-performance computing jobs. The overall performance of an E-HPC cluster depends on the compute node settings.
Proxy Mode	After you enable this feature, a proxy node is created on the cloud to manage cluster resources in the cloud.
Proxy Node	If you turn on Proxy Mode, you must specify the instance type of a proxy node in the cloud.

Expand Advanced Configurations and configure advanced settings based on your business requirements.

Parameter		Description
Network Configuration	VPC and vSwitch	The VPC in which the cluster resides. VPCs are isolated from each other. You can create and manage E-HPC clusters in a VPC. By default, the first VPC and vSwitch in the VPC and vSwitch drop-down lists are selected. Make sure that the number of available IP addresses is greater than the number of cluster nodes. You can click Create VPC and Create vSwitch (for subnet) to create a VPC and a vSwitch. For more information, see Create and manage a VPC and Create and manage a vSwitch.
Network Configuration	Create Security Group and Security Group Name	Create Security Group is automatically enabled and cannot be disabled. Enter a name for the new security group.
Storage	Storage Path	Select the path to store files. Valid values: Do Not Mount File System, Use On-premises File System, and Use Cloud File System.
	Local Cluster Storage Mount Point	If you use an on-premises file system, the path of the on-premises storage is used.
	Mount Path	If you use an on-premises file system, the path to the cluster where you want to mount storage is used.
	Local Path	If you use an on-premises file system, the on-premises path where you want to mount storage to the cluster is used.
	Configure by Directory	When you use a cloud file storage system, you need to select whether to mount different file systems for each folder.
	Type	If you use a cloud file storage system, select the type of file system.
	File System ID and Mount Point	By default, the first file system and mount target in the File System ID and Mount Point drop-down lists are selected. Make sure that the file system has sufficient mount targets. You can also click Create a file system and Create mount point to create a file system and a mount target. For more information, see Create a file system and Manage mount targets.
	Mount Configurations	If you mount a General-purpose NAS file system, you can select a mount protocol. Valid values: Mount over NFSv3 and Mount over NFSv4.
	Remote Directory	The remote directory of the file system to be mounted.

Click Next.

Parameter	Description
Image Type and Image	The image type and image to deploy a compute node environment.
Scheduler	The type of the scheduler for the on-premises cluster.
Scheduler Node IP Address	The IP address of the scheduling node in the on-premises cluster.
Scheduling Node Hostname	The hostname of the scheduling node in the on-premises cluster.
Account System	The domain account service of the scheduling node in the on-premises cluster.
Local Cluster Domain Name	The domain name of the on-premises cluster.
Account Node IP Address	The IP address of the domain account node in the on-premises cluster.
Account Node Hostname	The hostname of the domain account node in the on-premises cluster.

Click Next.

Parameter	Description
Cluster Name	The name of the cluster. The cluster name is displayed on the Cluster page.
Logon Password	The password of the cluster. This password is required when you use SSH to remotely access the logon node of the cluster. The username is root. Note If you turn on Proxy Mode, enter the root password of the on-premises cluster.
Repeat Password	You must enter the password again.

In the Configuration List section, check the settings that you configured. Read and select Alibaba Cloud International Website Product Terms of Service, and click OK.
Go to the Cluster page. If the hybrid cloud cluster enters the Running state, it is created.

Step 2: Scale up the hybrid cloud cluster

On the Cluster page, find the cluster that you want to scale up. Click Resize.
In the Resize panel, complete the basic configurations.
For more information, see Manually scale out a cluster. Configure the following parameters:
- Image Type: Select Custom Image.
- Image: Select the custom image that you created.
Check the current settings. Read and select Alibaba Cloud International Website Product Terms of Service, and click Buy Now.
After you scale up the cluster, you can query the status of each new node. To do this, in the left-side navigation pane, choose Resource Management > Nodes. On the Nodes page, select the cluster from the Cluster drop-down list and Compute Node from the Node Type drop-down list. If the nodes are in the Running state, the cluster is scaled up.