Create a hybrid cloud cluster in the E-HPC console - Elastic High Performance Computing

A hybrid cloud cluster is deployed across hybrid environments. Its scheduling nodes and domain account nodes reside on a data center, and its compute nodes reside on a data center and on the cloud. You can use on-premises high-performance computing clusters to scale out compute nodes on the cloud, and schedule cloud resources and on-premises resources for high-performance computing. This topic describes how to create a hybrid cloud cluster in the Elastic High Performance Computing (E-HPC) console.

Prerequisites

A virtual private cloud (VPC) and a vSwitch are created. For more information, see Create and manage a VPC and Create and manage a vSwitch.
A service-linked role for E-HPC is created. The first time you log on to the E-HPC console, you are prompted to create a service-linked role for E-HPC.
Your data center is connected to Alibaba Cloud by using a VPN gateway, a Cloud Enterprise Network (CEN) instance, or an Express Connect circuit. For more information, see Create a CEN instance, Create a VPN gateway, and Connection over an Express Connect circuit

Background information

The management nodes of the on-premises high-performance computing cluster have the following environment requirements:

OS: CentOS 7.2, 7.3, 7.4, 7.5, 7.6, or 8.0
Scheduler type: Deadline, Portable Batch System (PBS) 18.1.1, Open Grid Scheduler (SGE), or a custom scheduler

Note
If your cluster uses PBS or SGE, its operating system must be CentOS 7.x.
Account management service: a custom account management service

Procedure

To create and use a hybrid cloud cluster, perform the following steps:

Step 1: Create a hybrid cloud cluster

When you create a hybrid cloud cluster, you need to configure the hardware settings, software settings, and basic settings of the compute nodes.
Step 2: Scale up the hybrid cloud cluster

After you create a hybrid cloud cluster, the cluster does not contain compute nodes. To use the cluster, you must scale it up.

Before you scale up the hybrid cloud cluster, prepare a custom image based on your requirements. The Deadline scheduler or a custom scheduler must be installed in the custom image. For more information, see Create a custom image from an instance.

Step 1: Create a hybrid cloud cluster

Log on to the E-HPC console.
In the top navigation bar, select a region.
In the left-side navigation pane, choose Cluster.
On the Cluster page, click Create Hybrid Cluster in the upper-right corner.

Configure hardware settings.

Specify basic parameters.

Parameter	Description
Availability Zone	The zone in which the compute nodes in the cluster reside.
Pricing Model	The billing method of the compute nodes in the cluster. The billing method does not apply to elastic IP addresses (EIPs) or File Storage NAS (NAS) file systems. Subscription: A prepaid model where you purchase resources for a specific period, such as weekly, monthly, or yearly. Pay-As-You-Go: A postpaid model where you are billed hourly for the resources you use. Preemptible Instance: A cost-effective option compared to Pay-As-You-Go instances. The price fluctuates with supply and demand. You are billed for the actual duration of use in a postpaid model. For more information, see ECS billing method overview.
Compute Node	Compute nodes are used to run high-performance computing jobs. The overall performance of an E-HPC cluster depends on the compute node settings.
Proxy Mode	After you enable this feature, a proxy node is created on the cloud to manage cluster resources in the cloud.
Proxy Node	If you enable Proxy Mode, specify the instance type for the cloud proxy node.

Expand Advanced Configurations and configure the settings as needed.

Parameter		Description
Network Configuration	VPC and vSwitch	The VPC in which the cluster resides. VPCs are isolated from each other. You can create and manage E-HPC clusters in a VPC. By default, the first VPC and vSwitch in the VPC and vSwitch drop-down lists are selected. Make sure that the number of available IP addresses is greater than the number of cluster nodes. You can click Create VPC and Create vSwitch (for subnet) to create a new VPC and vSwitch. For more information, see Create a VPC and Create a vSwitch.
Network Configuration	Create Security Group and Security Group Name	Only the Create Security Group option is supported. Enter a name for the new security group.
Storage	Storage Path	Select the path to store files. Valid values: Do Not Mount File System, Use On-premises File System, and Use Cloud File System.
	Local Cluster Storage Mount Point	If you use an on-premises file system, the path of the on-premises storage is used.
	Mount Path	If you use an on-premises file system, the path to the cluster where you want to mount storage is used.
	Local Path	If you use an on-premises file system, the on-premises path where you want to mount storage to the cluster is used.
	Configure by Directory	When you use a cloud file storage system, you need to select whether to mount different file systems for each folder.
	File System Type	If you use a cloud file storage system, select the type of file system.
	File System ID and Mount Point	By default, the first file system and mount target in the File System ID and Mount Point drop-down lists are selected. Make sure that the file system has sufficient mount targets. You can also click Create a file system and Create mount point to create a new file system and mount target. For more information, see Create a file system and Add a mount target.
	Mount Configurations	When you mount a General Purpose file system, you can select the mount protocol. Both NFSv3 and NFSv4 are supported.
	Remote Directory	The remote directory of the file system to be mounted.

Click Next and configure the software settings.

Parameter	Description
Image Type and Image	The image type and image to deploy a compute node environment.
Scheduler	The type of the scheduler for the on-premises cluster.
Scheduler Node IP Address	The IP address of the scheduling node in the on-premises cluster.
Scheduling Node Hostname	The hostname of the scheduling node in the on-premises cluster.
Account System	The domain account service of the scheduling node in the on-premises cluster.
Local Cluster Domain Name	The domain name of the on-premises cluster.
Account Node IP Address	The IP address of the domain account node in the on-premises cluster.
Account Node Hostname	The hostname of the domain account node in the on-premises cluster.

Click Next and configure basic information.

Parameter	Description
Name	The name of the cluster. The cluster name is displayed on the Cluster page.
Logon Password	The password of the cluster. This password is required when you use SSH to remotely access the logon node of the cluster. The username is root. Note If you turn on Proxy Mode, enter the root password of the on-premises cluster.
Repeat Password	You must enter the password again.

In the Configuration List pane on the right, review the cluster configuration. Read and select Alibaba Cloud International Website Product Terms of Service, and then click OK.

You can monitor the creation process on the Cluster page. The cluster is ready when its status changes to Running.

Step 2: Scale up the hybrid cloud cluster

On the Cluster page, find your cluster and click Expand Capacity.
In the Resize panel, configure the basic settings.

For more information, see Manually scale out a cluster. Configure the following parameters:
- Image Type: Select Custom image.
- Image: Select the custom image that you prepared.
Review your configuration, read and select Alibaba Cloud International Website Product Terms of Service, and then click Buy Now.

After you scale out the cluster, you can check the status of the new nodes. In the left-side navigation pane, choose Resource Management > Nodes and Queues. Select your cluster from the cluster list and select Compute Node from the Node Type list. The scale-out is complete when the new nodes are Running.