Auto scaling of nodes - User Guide for Kubernetes Clusters| Alibaba Cloud Documentation Center

How auto scaling works

The auto scaling model of Kubernetes is different from the traditional scaling model that is based on the resource usage threshold. Developers must understand the differences between the two scaling models before they migrate workloads from traditional data centers or other orchestration systems to Kubernetes. For example, developers migrate workloads from Swarm clusters to ACK clusters.

The traditional scaling model is based on resource usage. For example, if a cluster contains three nodes and the CPU utilization or memory usage of the nodes exceeds the scaling threshold, new nodes are automatically added to the cluster. However, you must consider the following issues when you use the traditional scaling model:

Issue 1: How is a resource usage threshold specified and applied?

In a cluster, hot nodes may have high resource usage and other nodes may have low resource usage. If an average resource usage is specified as the threshold, auto scaling may not be triggered in a timely manner. If the lowest node resource usage is set as the scaling threshold, the newly added nodes may not be used. This may cause a waste of resources.

Issue 2: How is load balancing applied after instances are added?

In Kubernetes, a pod is used as the smallest unit that runs an application on each node of a cluster. When auto scaling is triggered for a cluster or a node in the cluster, pods with high resource usage are not replicated and the resource limits of these pods are not changed. As a result, the loads cannot be balanced to newly added nodes.

Issue 3: How is a scale-in activity triggered and implemented?

If scale-in activities are triggered based on resource usage, pods that request large amounts of resources but have low resource usage may be evicted. If the number of these pods is large within a Kubernetes cluster, resources may be exhausted and some pods may fail to be scheduled.

How does the auto scaling model of Kubernetes fix these issues? Kubernetes provides a two-layer scaling model that decouples pod scheduling from resource scaling.

Pods are scaled based on resource usage. When pods enter the Pending state due to insufficient resources, a scale-out activity is triggered. After new nodes are added to the cluster, the pending pods are automatically scheduled to the newly added nodes. This way, loads of the application are balanced. The following section describes the auto scaling model of Kubernetes in detail:

1. How are nodes selected during a scale-out activity?

cluster-autoscaler is used to trigger auto scaling by detecting pending pods. When pods enter the Pending state due to insufficient resources, cluster-autoscaler simulates pod scheduling to decide the scaling group that can provide new nodes to accept the pending pods. If a scaling group meets the requirement, nodes from this scaling group are added to the cluster.

A scaling group is treated as a node during the simulation. The instance type of the scaling group specifies the CPU, memory, and GPU resources of the node. The labels and taints of the scaling group are also applied to the node. The node is used to simulate the scheduling of the pending pods. If the pending pods can be scheduled to the node, cluster-autoscaler calculates the number of nodes that are required to be added from the scaling group.

2. How is a scale-in activity triggered?

Only nodes added by scale-out activities can be removed in scale-in activities. Static nodes cannot be managed by cluster-autoscaler. Each node is separately evaluated to determine whether the node needs to be removed. If the resource usage of a node drops below the scale-in threshold, a scale-in activity is triggered for the node. In this case, cluster-autoscaler simulates the eviction of all workloads on the node to determine whether the node can be completely drained. cluster-autoscaler does not drain the nodes that contain specific pods, such as non-DaemonSet pods in the kube-system namespace and pods that are controlled by PodDisruptionBudgets (PDBs). A node is drained before it is removed. After pods on the node are evicted to other nodes, the node can be removed.

3. How does cluster-autoscaler select among multiple scaling groups?

Each scaling group is regarded as an abstract node. cluster-autoscaler selects a scaling group for auto scaling based on a policy similar to the scheduling policy. Nodes are first filtered by the scheduling policy. Among the filtered nodes, the nodes that conform to policies, such as affinity settings, are selected. If no scheduling policy or affinity settings are configured, cluster-autoscaler selects a scaling group based on the least-waste policy. The least-waste policy selects the scaling group that has the fewest idle resources after simulation. If a scaling group of regular nodes and a scaling group of GPU-accelerated nodes both meet the requirements, the scaling group of regular nodes is selected by default.

4. How do I improve the success rate of auto scaling?

The result of auto scaling is dependent on the following factors:

Whether the scheduling policy is met
After you configure a scaling group, you must be aware of the pod scheduling policies that the scaling group supports. If you are unaware of the pod scheduling policies, you can simulate a scaling activity by using the node selectors of pending pods and the labels of the scaling group.
Whether resources are sufficient
After the scaling simulation is complete, a scaling group is selected. However, the scaling activity fails if the specified types of Elastic Compute Service (ECS) instances in the scaling group are out of stock. Therefore, you can configure multiple instance types and multiple zones for the scaling group to improve the success rate of auto scaling.

5. How do I accelerate auto scaling?

Method 1: Enable the swift mode to accelerate auto scaling. After a scaling group experiences a scale-out activity and a scale-in activity, the swift mode is enabled for this scaling group.
Method 2: Use custom images that are created from the base image of Alibaba Cloud Linux 2 (formerly known as Aliyun Linux 2). This ensures that the resources of Infrastructure as a Service (IaaS) are delivered 50% faster.

Considerations

For each account, the default CPU quota for pay-as-you-go instances in each region is 50 vCPUs. You can add at most 48 custom route entries to each route table of a virtual private cloud (VPC). To increase the quota, submit a ticket.
The stock of ECS instances may be insufficient for auto scaling if you specify only one ECS instance type for a scaling group. We recommend that you specify multiple ECS instance types with the same specification for a scaling group. This increases the success rate of auto scaling.
In swift mode, when a node is shut down and reclaimed, the node stops running and enters the NotReady state. When a scale-out activity is triggered, the state of the node changes to Ready.
If a node is shut down and reclaimed in swift mode, you are charged only for the disks. This rule does not apply to nodes that use local disks, such as the instance type of ecs.d1ne.2xlarge, for which you are also charged a computing fee. If the stock of nodes is sufficient, nodes can be launched within a short period of time.
If elastic IP addresses (EIPs) are bound to pods, we recommend that you do not delete the scaling group or the ECS nodes that are added from the scaling group in the ECS console. Otherwise, these EIPs cannot be automatically released.

Step 1: Configure auto scaling

Log on to the ACK console.
In the left-side navigation pane of the ACK console, click Clusters.
On the Clusters page, follow the instructions to navigate to the Configure Auto Scaling page.
You can navigate to the Configure Auto Scaling page in the following ways:
- Method 1: Find the cluster that you want to manage and choose More > Auto Scaling in the Actions column.
- Method 2:
  1. Find the cluster that you want to manage and click Details in the Actions column.
  2. In the left-side navigation pane of the details page, choose Nodes > Node Pools.
  3. In the upper-right corner of the Node Pools page, click Configure Auto Scaling.

Step 2: Perform authorization

You must perform authorization in the following scenarios:

The cluster has limited permissions on nodes

If ACK has limited permissions on nodes in the cluster, assign the AliyunCSManagedAutoScalerRole Resource Access Management (RAM) role to ACK.

Note You need to perform the authorization only once for each Alibaba Cloud account.

Activate Auto Scaling.
1. In the dialog box that appears, click the first hyperlink to log on to the Auto Scaling console.
2. Click Activate Auto Scaling to go to the Enable Service page.
3. Select the I agree with Auto Scaling Agreement of Service check box and click Enable Now.
4. On the Activated page, click Console to log on to the Auto Scaling console.
5. Click Go to Authorize to go to the Cloud Resource Access Authorization page. Then, authorize Auto Scaling to access other cloud resources.
6. Click Confirm Authorization Policy.
Assign a RAM role to ACK.
1. Click the second hyperlink in the dialog box.
  
  Note This step requires you to log on to the console with an Alibaba Cloud account.
2. On the Cloud Resource Access Authorization page, click Confirm Authorization Policy.

The cluster has unlimited permissions on nodes

Activate Auto Scaling.
1. In the dialog box that appears, click the first hyperlink to log on to the Auto Scaling console.
2. Click Activate Auto Scaling to go to the Enable Service page.
3. Select the I agree with Auto Scaling Agreement of Service check box and click Enable Now.
4. On the Activated page, click Console to log on to the Auto Scaling console.
5. Click Go to Authorize to go to the Cloud Resource Access Authorization page. Then, authorize Auto Scaling to access other cloud resources.
6. Click Confirm Authorization Policy.
If the authorization is successful, you are redirected to the Auto Scaling console. Close the page and modify the permissions of the worker RAM role.
Assigns one or more roles to a RAM user.
1. Click the second hyperlink in the dialog box to go to the RAM Roles page.
  
  Note This step requires you to log on to the console with an Alibaba Cloud account.
2. On the Permissions tab, click the name of the policy assigned to the RAM role. The details page of the policy appears.
3. Click Modify Policy Document. The Modify Policy Document panel appears on the right side of the page.
4. In the Policy Document section, add the following policy content to the Action field and click OK.
```
"ess:Describe*", 
"ess:CreateScalingRule", 
"ess:ModifyScalingGroup", 
"ess:RemoveInstances", 
"ess:ExecuteScalingRule", 
"ess:ModifyScalingRule", 
"ess:DeleteScalingRule", 
"ecs:DescribeInstanceTypes",
"ess:DetachInstances",
"vpc:DescribeVSwitches"
```
  Note Before you add the policy content, add a comma (,) to the end of the bottom line in the Action field.

The cluster requires to associate an auto-scaling node pool with an EIP

If you want to associate a scaling group with an EIP, perform the following steps to grant permissions:

Activate Auto Scaling.
1. In the dialog box that appears, click the first hyperlink to log on to the Auto Scaling console.
2. Click Activate Auto Scaling to go to the Enable Service page.
3. Select the I agree with Auto Scaling Agreement of Service check box and click Enable Now.
4. On the Activated page, click Console to log on to the Auto Scaling console.
5. Click Go to Authorize to go to the Cloud Resource Access Authorization page. Then, authorize Auto Scaling to access other cloud resources.
6. Click Confirm Authorization Policy.
If the authorization is successful, you are redirected to the Auto Scaling console. Close the page and modify the permissions of the worker RAM role.
Assigns one or more roles to a RAM user.
1. Click the second hyperlink in the dialog box to go to the RAM Roles page.
  
  Note This step requires you to log on to the console with an Alibaba Cloud account.
2. On the Permissions tab, click the name of the policy assigned to the RAM role. The details page of the policy appears.
3. Click Modify Policy Document. The Modify Policy Document panel appears on the right side of the page.
4. In the Policy Document section, add the following policy content to the Action field and click OK.
```
"ecs:AllocateEipAddress",
"ecs:AssociateEipAddress",
"ecs:DescribeEipAddresses",
"ecs:DescribeInstanceTypes",
"ecs:DescribeInvocationResults",
"ecs:DescribeInvocations",
"ecs:ReleaseEipAddress",
"ecs:RunCommand",
"ecs:UnassociateEipAddress",
"ess:CompleteLifecycleAction",
"ess:CreateScalingRule",
"ess:DeleteScalingRule",
"ess:Describe*",
"ess:DetachInstances",
"ess:ExecuteScalingRule",
"ess:ModifyScalingGroup",
"ess:ModifyScalingRule",
"ess:RemoveInstances",
"vpc:AllocateEipAddress",
"vpc:AssociateEipAddress",
"vpc:DescribeEipAddresses",
"vpc:DescribeVSwitches",
"vpc:ReleaseEipAddress",
"vpc:UnassociateEipAddress",
"vpc:TagResources"
```
  Note Before you add the policy content, add a comma (,) to the end of the bottom line in the Action field.
5. On the Roles page, click the name of the worker RAM role. On the details page of the RAM role, click the Trust Policy Management tab and click Edit Trust Policy. In the Edit Trust Policy panel, add oos.aliyuncs.com to the Service field, as shown in the following figure. Then, click OK.

Step 3: Configure auto scaling

On the Configure Auto Scaling page, set the following parameters and click Submit.


Parameter	Description
Cluster	The name of the cluster for which you want to enable auto scaling.
Allow Scale-in	Specify whether to allow the scale-in of nodes. If you turn off this option, scale-in configurations do not take effect. Proceed with caution.
Scale-in Threshold	For a scaling group that is managed by cluster-autoscaler, set the value to the ratio of the requested resources per node to the total resources per node. If the actual value is lower than the threshold, the node is removed from the cluster. Note In auto scaling, a scale-out activity is automatically triggered based on node scheduling. Therefore, you need only to set scale-in parameters.
GPU Scale-in Threshold	The scale-in threshold for GPU-accelerated nodes. If the actual value is lower than the threshold, one or more GPU-accelerated nodes are removed from the Kubernetes cluster.
Defer Scale-in For	The time to wait after the scale-in threshold is met and before the scale-in activity starts. Unit: minutes. The default value is 10 minutes.
Cooldown	Newly added nodes cannot be removed in scale-in activities during the cool down period.
Scan Interval	You can set this parameter to configure the interval at which the cluster is evaluated for scaling. Valid values: 15s, 30s, 60s. Default value: 30s.
Node Pool Scale-out Policy
least-waste	The default selection policy. If multiple node pools meet the requirement, this policy selects the node pool that will have the least idle resources after the scale-out activity is completed.
random	The random selection policy. If multiple node pools meet the requirement, this policy selects a random node pool for the scale-out activity.
priority	The priority-based selection policy. If multiple node pools meet the requirement, this policy selects the node pool with the highest priority for the scale-out activity. The priorities of node pools are configured in the `cluster-autoscaler-priority-expander` ConfigMap in the kube-system namespace. When a scale-out activity is triggered, the policy obtains the node pool priority from the ConfigMap by node pool ID and then selects the node pool with the highest priority for the scale-out activity. Note The priority must be an integer from 1 to 100. If you use the priority-based selection policy, you must configure the priorities for the node pools that have auto scaling enabled. You can specify only one priority value for a node pool ID. Node pools that meet the requirement but are not configured with priorities in the ConfigMap are not selected for the scale-out activity. Therefore, if you use the priority-based selection policy, you must configure priorities for all node pools that have auto scaling enabled in the `cluster-autoscaler-priority-expander` ConfigMap. The following code block is an example: `kind: ConfigMap metadata: name: cluster-autoscaler-priority-expander data: priorities: \|- 10: - asg-1largeid # Set the priority of asg-1largeid to 10. - asg-2largeid # Set the priority of asg-2largeid to 10. 50: - asg-3largeid # Set the priority of asg-3largeid to 50.` For example, asg-2largeid and asg-3largeid meet the requirement. In this case, asg-3largeid has a higher priority and is selected for the scale-out activity.

Select an instance type. Supported instance types are regular instances, GPU-accelerated instances, and preemptible instances. Then, click Create Node Pool.

In the Create Node Pool dialog box, set the node pool parameters.

For more information about the parameters, see Create an ACK managed cluster. The following table describes some of the parameters.


Parameter	Description
Region	The region where you want to deploy the scaling group. The scaling group and the Kubernetes cluster must be deployed in the same region. You cannot change the region after the scaling group is created.
VPC	The scaling group and the Kubernetes cluster must be deployed in the same VPC.
vSwitch	The vSwitches of the scaling group. You can specify vSwitches of different zones. The vSwitches allocate pod CIDR blocks to the scaling group.
Auto Scaling	Select the node type based on your requirements. You can select Regular Instance, GPU Instance, Shared GPU Instance, or Preemptible Instance. The selected node type must be the same as the node type that you select when you create the cluster.
Instance Type	The instance types in the scaling group.
Selected Types	The instance types that you have selected. You can select at most 10 instance types.
System Disk	The system disk of the scaling group.
Mount Data Disk	Specify whether to mount data disks to the scaling group. By default, no data disk is mounted.
Instances	The number of instances contained in the scaling group. Note Existing instances in the cluster are excluded. By default, the minimum number of instances is 0. If you specify one or more instances, the system adds the instances to the scaling group. When a scale-out activity is triggered, the instances in the scaling group are added to the cluster to which the scaling group is associated.
Operating System	When you enable auto scaling, you can select an image based on Alibaba Cloud Linux, CentOS, Windows, or Windows Core. Note If you select an image based on Windows or Windows Core, the system automatically adds the taint `{ effect: 'NoSchedule', key: 'os', value: 'windows' }` to nodes in the scaling group.
Key Pair	The key pair that is used to log on to the nodes in the scaling group. You can create key pairs in the ECS console. Note You can log on to the nodes only by using key pairs.
RDS Whitelist	The ApsaraDB RDS instances that can be accessed by the nodes in the scaling group after a scaling activity is triggered.
Node Label	Node labels are automatically added to nodes that are added to the cluster by scale-out activities.
Scaling Policy	Priority: scales the node pool based on the priorities of the vSwitches that you specify. If Auto Scaling fails to create ECS instances in the zone of the vSwitch with the highest priority, Auto Scaling attempts to create ECS instances in the zone of the vSwitch with a lower priority. Cost Optimization: creates ECS instances based on the ascending order of vCPU unit prices. The system preferably creates preemptible instances when multiple instance types are specified. If Auto Scaling fails to create preemptible instances for reasons such as preemptible instances are out of stock, Auto Scaling attempts to create pay-as-you-go ECS instances. If you set the scaling policy to cost optimization, you can configure the following parameters: Percentage of Pay-as-you-go Instances: Specify the percentage of pay-as-you-go instances in the node pool. Valid values: 0 to 100. Enable Supplemental Preemptible Instances: After you enable this feature, Auto Scaling automatically creates the same number of preemptible instances 5 minutes before the system reclaims the existing preemptible instances. The system sends a notification to Auto Scaling 5 minutes before it reclaims preemptible instances. Enable Supplemental Pay-as-you-go Instances: After you enable this feature, Auto Scaling attempts to create pay-as-you-go ECS instances to meet the scaling requirement if Auto Scaling fails to create preemptible instances for reasons such as the unit price is too high or preemptible instances are out of stock. Distribution Balancing: evenly distributes ECS instances across the zones of the vSwitches that are specified for the scaling group. If the distribution of ECS instances across zones is not balanced due to reasons such as ECS resources are out of stock, you can select this policy to evenly distribute the ECS instances across zones. Note This policy takes effect only when you have specified multiple VPC vSwitches.
Scaling Mode	You can select Standard or Swift. Standard: the standard mode. Auto scaling is implemented by creating and releasing ECS instances based on resource requests and usage. Swift: the swift mode. Auto scaling is implemented by creating, stopping, and starting ECS instances. This mode accelerates scaling activities. Note In Swift mode, if an instance fails to restart after it stops running, the instance cannot be automatically released. You must manually release the instance.
Taints	After you add taints to a node, ACK no longer schedules pods to the node.

Click Confirm Order to create the scaling group.

Expected result

On the Auto Scaling page, you can find the newly created scaling group below Regular Instance.
On the Clusters page, find the cluster that you want to manage and click the name of the cluster or click Details in the Actions column. The details page of the cluster appears.
In the left-side navigation pane of the details page, choose Workloads > Deployments.
On the Deployments page, select the kube-system namespace. You can find the cluster-autoscaler component. This indicates that the scaling group is created.

FAQ

Why does the auto scaling component fail to add nodes after a scale-out activity is triggered?
Check whether the following situations exist:
- The instance types in the scaling group cannot fulfill the resource request from pods. By default, system components are installed for each node. Therefore, the requested pod resources must be less than the resource capacity of the instance type.
- The RAM role does not have the permissions to manage the Kubernetes cluster. You must configure RAM roles for each Kubernetes cluster that is involved in the scale-out activity.
Why does the auto scaling component fail to remove nodes after a scale-in activity is triggered?
Check whether the following situations exist:
- The requested resource threshold of each pod is higher than the configured scale-in threshold.
- Pods that belong to the kube-system namespace are running on the node.
- A scheduling policy forces the pods to run on the current node. Therefore, the pods cannot be scheduled to other nodes.
- PodDisruptionBudget is set for the pods on the node and the minimum value of PodDisruptionBudget is reached.
For more information about FAQ, see open source component.
How does the system choose a scaling group for a scaling activity?
When pods cannot be scheduled to nodes, the auto scaling component simulates the scheduling of the pods based on the configurations of scaling groups. The configurations include labels, taints, and instance specifications. If a scaling group meets the requirements, this scaling group is selected for the scale-out activity. If more than one scaling group meet the requirements, the system selects the scaling group that has the fewest idle resources after simulation.