edit-icon download-icon

Create a cluster

Last Updated: Apr 20, 2018

Before creating a cluster, you need to enter the cluster list page. To enter the page, folow these steps:

  1. Log on to Alibaba Cloud E-MapReduce Console Cluster Page.

  2. Complete RAM authorization. For operating steps, refer to Role Authorization.

  3. Select the region where the cluster will be created.

  4. Click Create a cluster.

Cluster Creation Process

Note: Clusters cannot be modified after creation, except for their name. Carefully confirm that the configuration is correct during creation.

To create a cluster, follow the below steps:

  1. Input the software configuration.

    1. Select the product version. An earlier version cluster cannot be upgraded to a later version.
    2. Select cluster type. Currently E-MapReduce provides the following cluster types:

      • Hadoop, Standard Hadoop cluster that contains most Hadoop-related components. Details about the components are provided in the selection list.
      • Kafka, Independent Kafka cluster that provides Message Service.
    3. Check the required components. The more components you select, the higher the requirements for your computer configuration will be, otherwise there may be insufficient resources to run these services.

    4. Security mode, Whether to enable the Kerberos authentication function of the cluster. Normally individual user clusters do not need this function. It is disabled by default.

    5. You can configure software in the cluster. For detailed instructions, refer to Software Configuration.

  2. Input the hardware configuration.

    1. Select the payment method. The default payment method is Pay-As-You-Go mode, however, subscription mode can be selected. Subscription mode will be cheaper than Pay-As-You-Go mode, but the latter is more flexible. If you select subscription mode, you must select a duration.

    2. Select the zone where the cluster is located. Different zones have different ECS series and disks.

    3. Select the network type (classic network/VPC). VPC requires additional provisions of the VPC and subnet (switcher). If these are not created, click Create VPC/Subnet (VSwitch) to go to the VPC console to create them. For detailed instruction of E-MapReduce VPC, refer to VPC.

      Note: Classic network is not interoperable with VPC, and the network type cannot be changed after purchase. In terms of ECS instance series, different zones have different instance series, either series I, II, or both.

    4. Select an existing security group. Generally, there is no security group when you create the cluster for the first time. Turn on the “New security group” switch, and input the new security group name.

    5. Select the cluster mode. If the default cluster is not in high availability mode, the Master node number is 1. If the high availability mode is opened, the Master node number will be 2.

    6. Select the Master node configuration. By default, the Master node is configured with 8 MB public network, which is subject to Pay-As-You-Go mode. This cost is not included in the cluster cost and will be subject to additional payment.

    7. Select the Core node configuration to adjust the node number. The cluster must be configured with at least 2 Core nodes.

  3. The cluster cost is displayed in the right corner of the page. For Pay-As-You-Go clusters, an hourly cost will be displayed, and for subscription clusters, the total cost will be displayed.

  4. Input the basic information.

    1. Input the cluster name, which must be between 1-64 characters and consist of only Chinese characters, letters, numbers, “-“, and “_”.

    2. The log save function is set to open by default, however, you need to select an OSS log directory for where the job logs are saved in the cluster. If you do not need to save job logs, you can close this function. We strongly recommend that you open the OSS log save function because it will be helpful in debugging jobs.

    3. Set the logon password of the Master node, which must be between 8-30 characters and can consist of uppercase letters, lowercase letters, and digits. After the cluster is enabled, implement SSH logon using the root account with the set password.

    4. (Optional) You can execute your custom script before Hadoop is enabled in the cluster. For detailed instructions, refer to Bootstrap Action.

  5. After all valid information has been inputted, the “Create” button will be highlighted. Click Create to create the cluster.

    Note:

    • The cluster will be created after a few minutes if it is a Pay-As-You-Go cluster. You will be redirected to the Cluster List page, where a cluster in “Creating Cluster” status can be viewed. After completion, the cluster will be switched to Idle status.

    • For a subscription cluster, it will not be created until the order is generated and paid.

Creation Failed

If the creation fails, the cluster list page will show “Cluster creation failed”. The reason for the failure can be seen when the cursor is placed on the red exclamation point.

Thank you! We've received your feedback.