You can add existing Elastic Compute Service (ECS) instances to a Container Service for Kubernetes (ACK) cluster in the ACK console. ECS instances can be added only as worker nodes to an ACK cluster. This topic describes how to manually add ECS instances to an ACK cluster. This topic also describes how to configure ACK to automatically add ECS instances to an ACK cluster.
Prerequisites
- An ACK cluster is created. For more information, see Create an ACK managed cluster.
A node pool is created in the cluster. For more information, see Procedure.
Limits
Limits on instance types
Instances of GPU-accelerated ECS Bare Metal Instance families ebmgn7 and ebmgn7e cannot automatically delete the Multi-Instance GPU (MIG) configuration. ACK automatically resets the MIG configuration retained on instances of the preceding instance families. The reset may be time-consuming. In this case, you may fail to add the instances to a cluster.
For more information about how to troubleshoot the issue, see What do I do if I fail to add an ecs.ebmgn7 instance?.
For more information about the ebmgn7e instance family, see ebmgn7e, GPU-accelerated compute-optimized ECS Bare Metal Instance family.
Limits on quotas
Make sure that you have a sufficient node quota in the cluster. To increase the node quota, apply for a quota increase in Quota Center. For more information about the quota limits related to ACK clusters, see Limits.
Limits on ECS instances
The ECS instances that you want to add must belong to the same Alibaba Cloud account as the cluster. The ECS instances that you want to add to your cluster must be deployed in the same region and virtual private cloud (VPC) as your cluster.
You cannot add ECS instances that belong to other clusters. If you want to add ECS instances that belong to other clusters, refer to Cause 1: The node already belongs to another cluster for solutions.
Limits on networks
If the cluster uses the Terway network plug-in, only ECS instances of specific instance types can be added. For more information about the limits on instance types, refer to Cause 2: The instance type of the node does not support the Terway network plug-in for solutions.
If the maximum number of pods supported by an instance type in inclusive ENI mode is less than 10, you cannot add ECS instances of this instance type to your cluster.
If the maximum number of pods supported by an instance type in exclusive ENI mode is less than 5, you cannot add ECS instances of this instance type to your cluster.
If the cluster uses the Flannel network plug-in, the number of custom route entries in the system route table of the VPC in which the cluster resides cannot exceed the quota limit. To increase the quota, submit an application in Quota Center. For more information about custom route entries, see Route table overview.
If IPv4/IPv6 dual stack is enabled for your cluster, you must assign an IPv6 address to the primary elastic network interface (ENI) of the ECS instance. For more information about how to assign an IPv6 address to the primary ENI of an ECS instance, see Step 2: Assign an IPv6 address.
Limits on security groups
The security group type of the ECS instances that you want to add must be the same as the security group type of the node pool. For more information about security group types, see Overview. If the security group type of the ECS instances that you want to add is different from the security group type of the node pool, see Cause 3: The security group type of the node is different from the security group type of the node pool for solutions.
The security group rules of the ECS instances that you want to add do not conflict with the security group rules of the cluster. If the security group rules of the ECS instances that you want to add conflict with the security group rules of the cluster, refer to Cause 4: The security group rules of the node conflict with the security group rules of the cluster to resolve this issue.
When you add an existing ECS instance to a node pool, the instance is also added to the security group of the node pool. Make sure that the number of security groups to which the instance belongs does not exceed the upper limit. The default upper limit is 5. For more information about this limit, see Limits on security groups. If the number of security groups to which the instance belongs exceeds the upper limit, refer to Cause 5: The number of security groups to which the node belongs exceeds the upper limit for solutions.
Considerations
Adding exiting nodes to a cluster does not interrupt nodes and applications in the cluster.
Operating systems and cloud disks
To avoid data loss, we recommend that you back up data before you add an existing node to a node pool.
If you add an existing node in auto mode, the operating system of the node is replaced by the operating system of the node pool after the node is added to the node pool. If you add an existing node in manual node, the node uses the original operating system after the node is added to the node pool. If you want to use the original operating system, add the node in manual mode.
After you add an existing node to a cluster, the original system disk of the node is released but the original data disk of the node is retained. However, the ID of the data disk is changed. This does not affect the data on the data disk.
After you add an existing node to a cluster, existing user snapshots of the original system disk are retained. Automatic snapshots are retained or deleted together with the system disk based on the setting of the Delete Automatic Snapshots While Releasing Disk attribute of the system disk. You can go to the details page of the disk in the ECS console and click Modify Attributes to view or modify the setting.
To ensure that you have a sufficient snapshot quota to periodically run automatic snapshot policies, we recommend that you delete user snapshots and automatic snapshots that are no longer required.
Limits on ENIs
If your cluster uses the Terway network plug-in, take note of the following items. If your cluster uses the Flannel network plug-in, ignore the following items.
If the node that you want to add resides in a new zone, you must update vSwitch configurations for Terway. If you do not update vSwitch configurations for Terway, the vSwitch to which the primary ENI of the node belongs is used to assign IP addresses to pods on the node after you add the node to your cluster. For more information, see Increase the number of pod vSwitches in a cluster that uses the Terway plug-in.
When you add an existing node to a node pool, the ENI that is already bound to the node is retained. The vSwitch to which the ENI belongs is used to assign IP addresses to the pods on the node. You must make sure that only the primary ENI is bound to the node. If the IP address of a pod on the node does not fall within the CIDR block of the vSwitch after the node is added to your cluster, you must remove the node from your cluster, unbind secondary ENIs from the node, and then add the node to your cluster again.
Billing
ECS instances that you add to a cluster are not released when you delete the cluster or node pools in the cluster. To avoid unnecessary costs, we recommend that you check the billing of the ECS instances in a timely manner.
Automatically add ECS instances
In auto mode, all ECS instances that are available within your Alibaba Cloud account are listed. You can select, configure, and add one or more ECS instances to a cluster in the ACK console. After you complete the configurations, the ECS instances are automatically added to the cluster.
If your cluster does not have a node pool, create a node pool before you add existing nodes to the cluster. For more information, see Procedure.
Log on to the ACK console. In the left-side navigation pane, click Clusters.
On the Clusters page, click the name of the cluster that you want to manage and choose in the left-side navigation pane.
On the Node Pools page, find the node pool that you want to manage, click More in the Actions column, and then click Add Existing Node.
On the Add Existing ECS Instance wizard page, select the auto mode to automatically add ECS instances to the cluster.
Set Mode to Auto and select the ECS instances that you want to add in the Select Existing ECS Instance section.
ImportantIf the ECS instance that you want to add is not displayed in the section, it indicates that the ECS instance cannot be added to the cluster. For more information about the reasons why the instance cannot be added to the cluster, see Limits. You can click Show Unavailable Instances to display instances that cannot be added and view the reasons. If the ECS instance that you want to add is not displayed after you select Show Unavailable Instances, check whether the instance is deployed in the same region and VPC as the cluster.
Click Next Step and set the parameters on the Specify Instance Information wizard page.
Parameter
Description
Cluster ID/Name
Information about the cluster to which you want to add the instances. This parameter is automatically set.
Data Disk
Specify whether to store the container and image data on a data disk.
If the ECS instances have data disks mounted and the file system of the last data disk is not initialized, the system automatically formats the data disk to ext4. Then, the system uses the disk to store the data in /var/lib/docker and /var/lib/kubelet.
ImportantData on a disk is lost after you format the disk. Before the system formats the data disk, we recommend that you back up the data on the data disk.
If no data disk is attached to the ECS instances, ACK does not attach a new data disk to the instances.
Retain Instance Name
By default, Retain Instance Name is turned on. If you do not want to retain the instance name, you can turn off Retain Instance Name. After you disable this feature, the nodes are renamed based on the node naming rules.
Instance Information
The IDs and names of the instances that you want to add.
Click Next Step. In the Confirm message, click Confirm.
Manually add ECS instances
Manually adding an ECS instance does not change the operating system of the instance. Apart from the limits described in the Limits section, you must also take note of the following items:
The operating system does not support swap.
If you store containers and images on data disks, the data disks support only the ext and XFS file systems.
In manual mode, you must obtain the installation command, log on to an ECS instance, and then run the command to add the ECS instance to an ACK cluster. You can add only one ECS instance at a time.
If your cluster does not have a node pool, create a node pool before you add existing nodes to the cluster. For more information, see Procedure.
Log on to the ACK console. In the left-side navigation pane, click Clusters.
On the Clusters page, click the name of the cluster that you want to manage and choose in the left-side navigation pane.
On the Node Pools page, find the node pool that you want to manage, click More in the Actions column, and then click Add Existing Node.
On the Select Existing ECS Instance wizard page, select the manual mode to add ECS instances to the cluster.
Set Mode to Manual and select the ECS instances that you want to add in the Select Existing ECS Instance section.
Click Next Step and set the parameters on the Specify Instance Information wizard page.
Parameter
Description
Cluster ID/Name
Information about the cluster to which you want to add the instances. This parameter is automatically set.
Data Disk
Specify whether to store the container and image data on a data disk.
If the ECS instances have data disks mounted and the file system of the last data disk is not initialized, the system automatically formats the data disk to ext4. Then, the system uses the disk to store the data in /var/lib/docker and /var/lib/kubelet.
ImportantData on a disk is lost after you format the disk. Before the system formats the data disk, we recommend that you back up the data on the data disk.
If no data disk is attached to the ECS instances, ACK does not attach a new data disk to the instances.
Retain Instance Name
By default, Retain Instance Name is turned on. If you do not want to retain the instance name, you can turn off Retain Instance Name. After you disable this feature, the nodes are renamed based on the node naming rules.
Instance Information
The IDs and names of the instances that you want to add.
Click Next Step to go to the Complete wizard page. On the Complete wizard page, copy the command and click Done.
Log on to the ECS console. In the left-side navigation pane, choose . Then, select the region where your cluster is deployed and select the ECS instance that you want to add.
Click Connect in the Actions column. In the Connection and Command dialog box, select a connection method and go to the connection page.
The following table describes the connection methods.
Connection method
Description
VNC Connection
For more information about how to connect to an ECS instance by using Virtual Network Computing (VNC), see Connect to an instance by using VNC or Connect to a Windows instance by using a password.
Send Remote Commands (Cloud Assistant)
This is the recommended method. This method allows you to run remote commands on an instance. You can perform operations such as viewing disk capacity, installing software, and starting or stopping services without logging on to the instance. This feature is implemented by using Cloud Assistant. For more information about how to install and activate the Cloud Assistant client, see Install Cloud Assistant Agent.
On the connection page, follow the instructions and paste the command that you copied in Step 6. Then, click Run to execute the script.
After the script is executed, the ECS instance is added to the cluster.
Verify the result
In the left-side navigation pane of the cluster details page, choose .
On the Node Pools page, find the node pool that you want to manage and click Details in the Actions column.
On the node pool details page, click the Nodes tab and view the node that you added.
Causes of failure to add a node to your cluster
Cause 1: The node already belongs to another cluster
You cannot add nodes that belong to other clusters. If the node that you want to add belongs to another cluster, you must first remove the node from the cluster. Then, you can add the node to your cluster. For more information about how to remove a node from a cluster, see Remove a node.
Cause 2: The instance type of the node does not support the Terway network plug-in
If the cluster uses the Terway network plug-in, the instance type of the node must meet the following requirements:
If the maximum number of pods supported by an instance type in inclusive ENI mode is less than 10, you cannot add ECS instances of this instance type to your cluster.
If the maximum number of pods supported by an instance type in exclusive ENI mode is less than 5, you cannot add ECS instances of this instance type to your cluster.
Note that the maximum number of pods supported by a node depends on the maximum number of ENIs supported by the instance type of the node.
Maximum number of pods supported by an instance type in inclusive ENI mode = (Maximum number of ENIs supported by the instance type - 1) × Number of private IP addresses provided by each shared ENI
Maximum number of pods supported by an instance type in exclusive ENI mode = Maximum number of ENIs supported by the instance type - 1
You can use one of the following methods to view the maximum number of pods supported by an instance type:
Method 1: When you create a node pool in the ACK console, you can view the maximum number of pods supported by an instance type in the Terway Mode (Supported Pods) column of the Instance Type section.
Method 2: Perform the following steps to manually calculate the maximum number of pods supported by an instance type:
Search the relevant documentation to obtain the number of ENIs supported by the instance type. For more information, see Overview of instance families.
Log on to the OpenAPI Explorer platform. Specify the instance type and click Call Now. In the output, the
EniQuantityfield indicates the maximum number of ENIs supported by the instance type and theEniPrivateIpAddressQuantityfield indicates the number of private IP addresses provided by each ENI.
If the instance type of the node does not meet the preceding requirements, upgrade the instance type of the node or create a new instance of an instance type that meets the requirements. For more information, see Change resource configurations.
Cause 3: The security group type of the node is different from the security group type of the node pool
When you add an existing ECS instance to a node pool, the instance is also added to the security group of the node pool. ECS instances cannot be added to a basic security group and an advanced security group at the same time. Therefore, the security group type of the ECS instance that you want to add must be the same as the security group type of the node pool. You can view the node pool of a node pool on the Overview tab of the node pool details page in the ACK console. You can also go to the ECS console to view the security group to which an ECS instance belongs. For more information about security groups, see Overview.
Note that you cannot change the security group of a node pool or the security group type of a node. If the security group type of the ECS instance that you want to add is different from the security group type of the node pool, you can perform the following operations to resolve this issue:
Change the security group of the ECS instance with a security group of the same security group type as the node pool. For more information, see Replace the security groups of ECS instances
Remove the ECS instance from the security group and then add the instance to the node pool or cluster.
Cause 4: The security group rules of the node conflict with the security group rules of the cluster
The security group rules of the node that you want to add to a cluster cannot conflict with the security group rules of the cluster. However, it is difficult to perform a precheck on whether the security group rules of the node conflict with the security group rules of the cluster. You can add the node to the security group of the node pool and use the security group rule check feature to check whether the security group rules of the node conflict with the security group rules of the node pool.
Nodes that already belong to the security group of the node pool are not added to the security group again.
Cause 5: The number of security groups to which the node belongs exceeds the upper limit
The number of security groups to which an ECS instance belongs cannot exceed an upper limit. If the number of security groups to which an ECS instance belongs exceeds the upper limit after you add the instance to the security group of a node pool, you can submit an application to increase the upper limit. For more information about the upper limit and how to increase the upper limit, see Security group limits.