This topic describes how to create a dedicated Kubernetes cluster for heterogeneous computing in the Container Service for Kubernetes (ACK) console.

Background information

ACK performs the following operations when a cluster is created:

  • Creates ECS instances, configures a public key to enable Secure Shell (SSH) logon from master nodes to other nodes, and then configures the ACK cluster through CloudInit.
  • Creates a security group that allows access to the VPC over Internet Control Message Protocol (ICMP).
  • If you do not specify an existing VPC, ACK creates a VPC and vSwitch and creates SNAT entries for the vSwitch.
  • Adds route entries to the VPC.
  • Creates a NAT gateway and EIPs.
  • Creates a Resource Access Management (RAM) user and an AccessKey pair. Grants the following permissions to the RAM user: permissions to query, create, and delete ECS instances, permissions to add and delete disks, and full permissions on SLB, CloudMonitor, VPC, Log Service, and Apsara File Storage NAS. The ACK cluster automatically creates SLB instances, disks, and VPC route entries based on your configuration.
  • Creates an internal-facing SLB instance and opens port 6443.
  • Creates an Internet-facing SLB instance and opens ports 6443, 8443, and 22. If you enable SSH logon when you create the cluster, port 22 is opened. Otherwise, port 22 is not exposed.

Limits

  • SLB instances that are created along with an ACK cluster support only the pay-as-you-go billing method.
  • ACK clusters support only VPCs.
  • By default, each account has specific quotas on cloud resources that can be created. You cannot create clusters if the quota is reached. Make sure that you have sufficient quotas before you create a cluster.

    To request a quota increase, submit a ticket.

    • By default, you can create up to 50 clusters across all regions with each account. Each cluster can contain up to 100 nodes. To increase the quota of clusters or nodes,submit a ticket.
      Notice By default, you can add up to 48 route entries to a VPC. This means that you can deploy up to 48 nodes in an ACK cluster that uses Flannel. An ACK cluster that uses Terway is not subject to this limit. To deploy more nodes in this case,submit a ticket to apply for an increase on the quota of route entries in the VPC that you want to use.
    • By default, you can create up to 100 security groups with each account.
    • By default, you can create up to 60 pay-as-you-go SLB instances with each account.
    • By default, you can create up to 20 EIPs with each account.
  • Limits on ECS instances:

    The pay-as-you-go and subscription billing methods are supported.

    Note After an ECS instance is created, you can change its billing method from pay-as-you-go to subscription in the ECS console. For more information, see Change the billing method of an instance from pay-as-you-go to subscription.

Procedure

  1. Log on to the ACK console.
  2. In the left-side navigation pane of the ACK console, click Clusters.
  3. In the upper-right corner of the Clusters page, click Cluster Templates.
  4. In the Select Cluster Template dialog box, find Dedicated Cluster for Heterogeneous Computing in the Other Clusters section and click Create.
  5. On the Dedicated Kubernetes tab, configure the cluster.
    1. Configure basic settings of the cluster.
      Parameter Description
      Cluster Name Enter a name for the cluster.
      Note The name must be 1 to 63 characters in length, and can contain digits, letters, and hyphens (-).
      Region Select the region where you want to deploy the cluster.
      Resource Group Move the pointer over All Resources at the top of the page and select the resource group to which the cluster belongs. The name of the selected resource group is displayed in the Resource Group field. Resource group
      Kubernetes Version Select a Kubernetes version.
      Container Runtime

      The containerd, Docker, and Sandboxed-Container runtimes are supported. For more information, see How do I choose between Docker and Sandboxed-Container?.

      VPC Select a virtual private cloud (VPC) to deploy the cluster. Shared VPCs and standard VPCs are supported.
      • Shared VPC: The owner of a VPC (resource owner) can share vSwitches in the VPC with other accounts in the same organization.
      • Standard VPC: The owner of a VPC (resource owner) cannot share vSwitches in the VPC with other accounts.
      Note ACK clusters support only VPCs. You can select a VPC from the drop-down list. If no VPC is available, click Create VPC to create one. For more information, see Work with VPCs.
      VSwitch Set vSwitches.

      You can select up to three vSwitches that are deployed in different zones. If no vSwitch is available, click Create VSwitch to create one. For more information, see Work with vSwitches.

      Network Plug-in Select and configure the network plug-in. Flannel and Terway are available. For more information, see Create a vSwitch.
      • Flannel: an open source Container Network Interface (CNI) plug-in, which is simple and stable. Flannel provides a few simple features and does not support standard Kubernetes network policies.
      • Terway: a network plug-in developed by Alibaba Cloud for ACK. Terway allows you to assign Alibaba Cloud elastic network interfaces (ENIs) to containers and use standard Kubernetes network policies to regulate how containers communicate with each other. Terway also supports bandwidth throttling on individual containers.
        Note
        • The number of pods that can be deployed on a node depends on the number of ENIs that are attached to the node and the maximum number of secondary IP addresses provided by these ENIs.
        • If you select a shared VPC for the cluster, you must select the Terway network plug-in.
      Pod CIDR Block If you select Flannel, you must set the Pod CIDR Block parameter.

      The CIDR block specified by Pod CIDR Block cannot overlap with that of the VPC or the CIDR blocks of existing ACK clusters in the VPC. After you create the cluster, you cannot modify the pod CIDR block. In addition, the Service CIDR block cannot overlap with the pod CIDR block. For more information about subnetting for ACK clusters, see Plan CIDR blocks for an ACK cluster.

      Terway Mode If you set Network Plug-in to Terway, the Terway Mode parameter is available.
      When you set Terway Mode, you can select or clear Assign One ENI to Each Pod.
      • If you select Assign One ENI to Each Pod, an ENI is assigned to each pod.
      • If you clear Assign One ENI to Each Pod, an ENI is shared among multiple pods. A secondary IP address of the ENI is assigned to each pod.
      Note This feature is available to only users in the whitelist. To apply to be added to the whitelist, submit a ticket.
      Pod VSwitch If you set Network Plug-in to Terway, the Pod VSwitch parameter is available. Pod VSwitch specifies vSwitches for pods. The ENIs that are assigned to pods must be in the same zone as the nodes that host the pods. For each vSwitch that is assigned to nodes, select a vSwitch for pods in the same zone. Pod vSwitches assign IP addresses to pods when the cluster is started. We recommend that you select vSwitches whose prefix length is no greater than 19 bits. This ensures a sufficient number of pods.
      Service CIDR Set the Service CIDR parameter. The CIDR block specified by Service CIDR cannot overlap with that of the VPC or the CIDR blocks of existing ACK clusters in the VPC. After you create the cluster, you cannot modify the Service CIDR block. In addition, the Service CIDR block cannot overlap with the pod CIDR block. For more information about subnetting for ACK clusters, see Plan CIDR blocks for an ACK cluster.
      IP Addresses per Node If you select Flannel, you must specify a value for the IP Addresses per Node parameter.
      Note IP Addresses per Node specifies the maximum number of IP addresses that can be assigned to each node. We recommend that you use the default value.
      Configure SNAT Specify whether to configure Source Network Address Translation (SNAT) rules for the VPC.
      • If the specified VPC has a Network Address Translation (NAT) gateway, ACK uses this NAT gateway.
      • Otherwise, the system automatically creates a NAT gateway. If you do not want the system to create a NAT gateway, clear Configure SNAT for VPC. In this case, you must manually create a NAT gateway and configure SNAT rules to enable Internet access for the VPC. Otherwise, ACK clusters in the VPC cannot access the Internet and the cluster cannot be created.
      Access to API Server
      By default, an internal-facing Server Load Balancer (SLB) instance is created for the cluster API server. You can modify the specification of the SLB instance. For more information, see Instance types and specifications.
      Notice If you delete the SLB instance, you cannot access the cluster API server.
      Select or clear Expose API Server with EIP. The ACK API server provides multiple HTTP-based RESTful APIs, which can be used to create, delete, modify, query, and monitor resources, such as pods and Services.
      • If you select this check box, an elastic IP address (EIP) is created and associated with an Internet-facing SLB instance. Port 6443 used by the API server is opened on master nodes. You can connect to and manage the ACK cluster by using kubeconfig over the Internet.
      • If you clear this check box, no EIP is created. You can connect to and manage the ACK cluster by using kubeconfig only within the VPC.
      SSH Logon

      To enable Secure Shell (SSH) logon, you must first select Expose API Server with EIP.

      • If you select Use SSH to Access the Cluster from the Internet, you can access the cluster through SSH.
      • If you clear Use SSH to Access the Cluster from the Internet, you cannot access the cluster through SSH or kubectl. If you want to access an Elastic Compute Service (ECS) instance in the cluster through SSH, you must manually bind an elastic IP address (EIP) to the ECS instance and configure security group rules to open SSH port 22. For more information, see Use SSH to connect to an ACK cluster.
      RDS Whitelist Set the Relational Database Service (RDS) whitelist. Add the IP addresses of the nodes in the cluster to the RDS whitelist.
      Security Group

      You can select Create Basic Security Group, Create Advanced Security Group, or Select Existing Security Group. For more information, see Overview.

    2. Configure advanced settings.
      Parameter Description
      Time Zone

      Select a time zone for the ACK cluster. By default, the time zone configured for your browser is selected.

      Kube-proxy Mode

      iptables and IPVS are supported.

      • iptables is a kube-proxy mode. It uses iptables rules to conduct Service discovery and load balancing. The performance of this mode is limited by the size of the cluster. This mode is suitable for clusters that run a small number of Services.
      • IPVS is a high-performance kube-proxy mode. It uses Linux IP Virtual Server (IPVS) to conduct Service discovery and load balancing. This mode is suitable for clusters that run a large number of Services. We recommend that you use this mode in scenarios where high-performance load balancing is required.
      Labels

      Add labels to the nodes in the cluster. Enter keys and values, and then click Add.

      Note
      • Key is required. Value is optional.
      • Keys cannot start with aliyun, http://, or https://. Keys are not case-sensitive and cannot exceed 64 characters in length.
      • Values are not case-sensitive. A value must not exceed 128 characters in length, and cannot start with http:// or https://.
      • The keys of labels that are added to the same resource must be unique. If you add a label with a used key, the label overwrites the one that uses the same key.
      • You can add up to 20 labels to each resource. If you add more than 20 labels to a resource, all labels become invalid. You must remove unused labels for the other labels to take effect.
      Custom Image

      You can select a custom image to replace the default image.

      Cluster Domain Set the domain name of the cluster.
      Note The default domain name is cluster.local. You can enter a custom domain name. A domain name consists of two parts. Each part must not exceed 63 characters in length, and can contain only letters and digits. You cannot leave these parts empty.
      Custom Certificate SANs

      You can specify custom subject alternative names (SANs) for the API server certificate. Separate multiple IP addresses or domain names with commas (,).

      Service Account Token Volume Projection

      Enable Service Account Token Volume Projection to enhance security when you use service accounts. For more information, see Enable service account token volume projection.

      Cluster CA If you select Custom Cluster CA, upload a Certificate Authority (CA) certificate for the cluster to ensure secure data transmission between the server and client.
      Deletion Protection Specify whether to enable deletion protection. If you select this check box, the cluster cannot be deleted in the console or by calling API operations. This protects the cluster from being accidentally deleted.
  6. Click Next:Master Configurations to configure master nodes.
    Parameter Description
    Billing Method Pay-As-You-Go and Subscription are supported.
    Duration If you select Subscription, you must specify the subscription duration.
    Auto Renewal If you select Subscription, you must specify whether to enable Auto Renewal.
    Master Node Quantity Specify the number of master nodes. You can create three or five master nodes.
    Instance Type Select an instance type for the master nodes. For more information, see Instance families.
    Note If no instance type is available, you can change vSwitches on the Cluster Configurations wizard page.
    System Disk By default, system disks are mounted to master nodes. ESSD Disk, SSD Disk, and Ultra Disk are supported.
    Note
    • You can select Enable Backup to back up disk data.
    • If you select an enhanced SSD as the system disk, you can set a performance level for the disk.

      You can select higher performance levels for enhanced SSDs with larger storage capacities. For example, you can select performance level 2 for an enhanced SSD with a storage capacity of more than 460 GiB. You can select performance level 3 for an enhanced SSD with a storage capacity of more than 1,260 GiB. For more information, see Capacity and PLs.

  7. Click Next:Worker Configurations to configure worker nodes.
    1. Set worker instances.
      • If you select Create Instance, you must set the parameters that are listed in the following table.
        Parameter Description
        Instance Type Choose Heterogeneous Computing GPU/FPGA/NPU > Compute Optimized Type with GPU. In the list of available instance types, select one or more required instance types. For more information, see Instance families.
        Note If no instance type is available, you can change vSwitches on the Cluster Configurations wizard page.
        Selected Types The selected instance types are displayed.
        Quantity Specify the number of worker nodes.
        System Disk
        Enhanced SSDs, standard SSDs, and ultra disks are supported.
        Note
        • You can select Enable Backup to back up disk data.
        • If you select an enhanced SSD as the system disk, you can set a custom performance level for the disk.

          You can select higher performance levels for enhanced SSDs with larger storage capacities. For example, you can select performance level 2 for an enhanced SSD with a storage capacity of more than 460 GiB. You can select performance level 3 for an enhanced SSD with a storage capacity of more than 1,260 GiB. For more information, see Capacity and PLs.

        Mount Data Disk

        ESSD Disk, SSD Disk, and Ultra Disk are supported. You can select Encrypt Disk and Enable Backup when you mount data disks.

        Operating System
        ACK supports the following node operating systems:
        • Alibaba Cloud Linux 2. This is the default operating system.
        • CentOS 7.x
          Note CentOS 8.x and later are not supported.
        Logon Type
        • Key pair logon
          • Key Pair: Select an SSH key pair from the drop-down list.
          • create a key pair: Create an SSH key pair if none is available. For more information about how to create an SSH key pair, see Create an SSH key pair. After the key pair is created, set it as the credential that is used to log on to the cluster.
        • Password logon
          • Password: Enter the password that is used to log on to the nodes.
          • Confirm Password: Enter the password again.
      • If you select Add Existing Instance, make sure that you have created ECS instances in the region where the cluster is deployed. Then, set Operating System, Logon Type, and Key Pair based on the preceding settings.
    2. Configure advanced settings.
      Parameter Description
      Node Protection Specify whether to enable node protection.
      Note By default, this check box is selected. The nodes in the cluster cannot be deleted in the console or by calling the API. This protects the nodes from being accidentally deleted.
      User Data For more information, see Overview of ECS instance user data.
      Custom Node Name Specify whether to enable Custom Node Name.
      A node name consists of a prefix, an IP substring, and a suffix.
      • Both the prefix and suffix can contain one or more parts that are separated with periods (.).These parts can contain lowercase letters, digits, and hyphens (-), and must start and end with a digit or lowercase letter.
      • The IP substring length specifies the number of digits to be truncated from the end of the node IP address. Valid values: 5 to 12.

      For example, if the node IP address is 192.168.0.55, the prefix is aliyun.com, the IP substring length is 5, and the suffix is test, the node name is aliyun.com00055test.

      Node Port Range Set the node port range. The default port range is 30000 to 32767.
      CPU Policy Set the CPU policy.
      • None: indicates that the default CPU affinity is used. This is the default policy.
      • Static: allows pods with specific resource characteristics on the node to be granted with enhanced CPU affinity and exclusivity.
      Taints Add taints to all worker nodes in the cluster.
  8. Click Next:Component Configurations to configure components.
    Parameter Description
    Ingress Specify whether to install Ingress controllers. By default, Install Ingress Controllers is selected. For more information, see Advanced Ingress configurations.
    Volume Plug-in Select a volume plug-in. FlexVolume and CSI are supported. ACK clusters can be automatically bound to Alibaba Cloud disks, Apsara File Storage NAS (NAS) file systems, and Object Storage Service (OSS) buckets that are mounted to pods. For more information, see Storage management-Flexvolume and Storage management-CSI.
    Monitoring Agents

    Specify whether to install the CloudMonitor agent. You can select Install CloudMonitor Agent on ECS Instance. After the CloudMonitor agent is installed, you can view monitoring information about ECS instances in the CloudMonitor console.

    Log Service

    Specify whether to enable Log Service. You can select an existing project or create a project.

    If you select Enable Log Service, the Log Service plug-in is automatically installed in the cluster. When you deploy an application, you can activate Log Service by performing a few steps. For more information, see Collect log files from containers by using Log Service.

    After you select Enable Log Service, you can specify whether to select Create Ingress Dashboard and Install node-problem-detector and Create Event Center.

    Workflow Engine Specify whether to enable Alibaba Cloud Genomics Compute Service (AGS).
    Note This feature is available to only users in the whitelist.
    • If you select this check box, the system automatically installs the AGS workflow plug-in when the system creates the cluster.
    • If you clear this check box, you must manually install the AGS workflow plug-in. For more information, see Introduction to AGS CLI.
  9. Click Next:Confirm Order.
  10. Select Terms of Service and click Create Cluster.
    Note It requires approximately 10 minutes for the system to create a managed Kubernetes cluster that contains multiple nodes.

What to do next

After the cluster is created, go to the Clusters page, find the created cluster, and then click the cluster name or click Details in the Actions column. In the left-side navigation pane of the details page, choose Nodes > Nodes. On the page that appears, select the worker node that is configured when you create the cluster, and choose More > Details in the Actions column. Then, you can view the GPU devices that are associated with the node.