A hybrid cloud cluster is deployed across hybrid environments. Its scheduling nodes and domain account nodes reside on a data center and its compute nodes reside on a data center and on the cloud. You can use on-premises high-performance computing clusters to scale out compute nodes on the cloud, and schedule cloud resources and on-premises resources for high-performance computing. This topic describes how to create a hybrid cloud cluster in the Elastic High Performance Computing (E-HPC) console.
Prerequisites
A virtual private cloud (VPC) and a vSwitch are created. For more information, see Create and manage a VPC and Work with vSwitches.
A service-linked role for E-HPC is created. The first time you log on to the E-HPC console, you are prompted to create a service-linked role for E-HPC.
Your data center is connected to Alibaba Cloud by using a VPN gateway, a Cloud Enterprise Network (CEN) instance, or an Express Connect circuit. For more information, see Create a CEN instance, Create a VPN gateway, and What is a connection over an Express Connect circuit?
Background information
The management nodes of the on-premises high-performance computing cluster have the following environment requirements:
Operating system: CentOS 7.2, 7.3, 7.4, 7.5, 7.6, or 8.0
Scheduler type: Deadline, PBS 18.1.1, Open Grid Scheduler (SGE), or a custom scheduler
NoteIf your cluster uses PBS or SGE, its operating system must be CentOS 7.x.
Account management service: a custom account management service
Procedure
To create and use a hybrid cloud cluster, perform the following steps:
Step 1: Create a hybrid cloud cluster
When you create a hybrid cloud cluster, you need to configure the hardware settings, software settings, and basic settings of the compute nodes.
Step 2: Scale out a hybrid cloud cluster
After you create a hybrid cloud cluster, the cluster does not contain compute nodes. To use the cluster, you must scale it out.
Before a scale-out, prepare a custom image based on your needs. The Deadline scheduler or a custom scheduler must be installed on the custom image. For more information, see Create a custom image from an instance.
Step 1: Create a hybrid cloud cluster
Log on to the E-HPC console.
In the upper-left corner of the top navigation bar, select a region.
In the top navigation bar, click Cluster.
On the Cluster page, click Create Hybrid Cluster.
Configure hardware settings.
In the Hardware Configurations step, configure the hardware settings.
Parameter
Description
Availability Zone
The zone of the compute nodes in the cluster.
Pricing Model
The billing method of the compute nodes in the cluster. The billing method does not apply to elastic IP addresses or NAS file systems.
Subscription: You can purchase or renew compute nodes by week, month, or year.
Pay-As-You-Go: Bills are generated on an hourly basis.
Preemptible Instance: Preemptible instances are more cost-effective than pay-as-you-go instances. The price of a preemptible instance fluctuates based on the supply and demand for its instance type. A preemptible instance is billed based on the duration for which you use it.
For more information, see Billing method overview.
Compute Node
Compute nodes are used to run high-performance computing jobs. The overall performance of an E-HPC cluster depends on the compute node settings.
Proxy Mode
If you turn on Proxy Mode, the proxy nodes on the cloud manages cloud cluster resources in place of the on-premises cluster.
Proxy Node
The instance type of the proxy nodes on the cloud.
Preinstalled Scheduler
Schedulers are software that handles multiple jobs. A scheduler is deployed on an E-HPC cluster.
If you enable Proxy Mode, Preinstalled Scheduler must be disabled.
If you disable Proxy Mode, Preinstalled Scheduler must be enabled.
Local Storage
Specifies whether to mount an on-premises file system on the cluster.
If an on-premises file system is mounted on the cluster, all cluster nodes can access data stored in the file system.
Local Cluster Storage Mount Point
The mount target of the on-premises file system.
Mount Path
The mount path of the on-premises file system.
Local Path
The path in which the on-premises file system is mounted on the nodes on the cloud.
Click Advanced Configurations. In the Advanced Configurations section, specify the parameters based on your needs.
Parameter
Description
Resource Group
Resource Group
The resource group to which the cluster nodes belong. You can use the resource group to manage multiple cluster nodes that belong to your account in a centralized manner.
Network Settings
VPC
The VPC where the cluster resides. VPCs are isolated from each other. You can create and manage E-HPC clusters in a VPC.
If you do not specify these parameters, the first VPC and vSwitch in the VPC and vSwitch drop-down lists are selected by default. Make sure that the number of available IP addresses is greater than that of cluster nodes.
You can also click Create VPC and Create vSwitch (for subnet) to create a VPC and a vSwitch. For more information, see Work with VPCs and Work with vSwitches.
vSwitch
Create Security Group
Select whether to create a security group.
If you turn on Create Security Group, you need to enter a new security group name in the Security Group Name field.
If you turn off Create Security Group, you need to select an existing security group in the Select Security Group drop-down list.
Storage
Configure by Directory
Specifies whether to mount Apsara File Storage NAS file systems on the cluster.
If you turn on Proxy Mode, you must mount a NAS file system on the cluster.
If you turn off Proxy Mode, you can mount only an on-premises file system on the cluster.
E-HPC stores all user data, scheduler data, and job sharing data on a NAS file system. All nodes in the cluster can access the data.
If you turn off Configure By Directory, only one file system is configured for the cluster.
If you turn on Configure By Directory, file systems are mounted for the directories of all nodes to improve the shared storage capability of the cluster.
File System Type
The type of the NAS file system. Valid values:
General Purpose
Extreme
File System ID
If you do not specify these parameters, the first file system and mount target in the File System ID and Mount Point drop-down lists are selected by default. Make sure that the file system has sufficient mount targets.
You can also click Create a file system and Create mount point to create a file system and a mount target. For more information, see Create a file system and Manage mount targets.
Mount Point
Remote Directory
The remote directory to which the file system is mounted.
Click Next.
Parameter
Description
Image Type
An image provides the information that all cluster nodes require. Different image types apply to different operating systems. The operating systems of all the nodes in a cluster are the same.
You can select Public Image, Custom Image, or Shared Image.
Custom images have the following limits:
E-HPC allows you to modify only an image provided by Alibaba Cloud.
A custom image that is created based on an E-HPC cluster cannot be modified. Otherwise, compute nodes may not run as expected after the current cluster is created.
You cannot modify the yum source configurations of the operating system in a custom image. Otherwise, the cluster cannot be created or scaled out.
The mount directory of a custom image cannot be the
/homeor/optdirectory.
Image
An image provides the information that all cluster nodes require. The information includes the operating system, pre-installed software, and deployed business or application data of a cluster.
Only CentOS and Windows Server images are supported. The image types displayed on the Create Hybrid Cloud page of the console prevail.
Scheduler
Schedulers are software that handles multiple jobs. A scheduler is deployed on an E-HPC cluster. The following schedulers are supported based on whether Proxy Mode is enabled:
If Proxy Mode is enabled, SGE and PBS are supported.
If Proxy Mode is disabled, Deadline and custom schedulers are supported. You can also choose not to install a scheduler.
Scheduler Node IP Address
The IP address of the scheduling node.
Scheduling Node Hostname
The hostname of the scheduling node.
Account System
The domain account service of the scheduling node.
The cluster and the cluster users are managed based on the domain account service. Valid values: nis and ldap.
Local Cluster Domain Name
The domain name of the on-premises cluster.
Account Node IP Address
The IP address of the account node.
Account node host name
The hostname of the account node.
Click Next.
Parameter
Description
Cluster Name
The name of the cluster. The cluster name is displayed on the Cluster page.
Logon Password
The password of the cluster. This password is used when you remotely use SSH to access the logon node of the cluster. The username is root.
NoteIf you turn on Proxy Mode, enter the root password of the on-premises cluster.
Repeat Password
Enter the password again.
In the Configuration List section, check the settings that you configured. Read and select Alibaba Cloud International Website Product Terms of Service, and click OK.
Navigate to the Cluster page. If the hybrid cloud cluster enters the Running state, it is created.
Step 2: Scale out a hybrid cloud cluster
On the Cluster page, find the cluster that you want to scale out. Click Resize.
In the Resize panel, complete the basic configurations.
For more information, see Scale out a cluster. Set the following parameters:
Image Type: Select Custom Image.
Image: Select the custom image that you have created.
Check the current settings. Read and select Alibaba Cloud International Website Product Terms of Service, and click Buy Now.
After you scale out the cluster, you can query the status of each scale-out node. In the left-side navigation pane, choose Resource Management > Nodes. On the Nodes page, select the cluster from the Cluster drop-down list and Compute Node from the Node Type drop-down list. If the nodes are in the Running state, the cluster is scaled out.