When you use a Container Service for Kubernetes (ACK) cluster, you may need to launch a large number of pods within a short period of time. If you choose to create Elastic Compute Service (ECS) instances for the pods, the creation process can be time-consuming. If you choose to reserve ECS instances, the instances are idle before pod creation and after pod termination, resulting in resource waste. With virtual nodes, you do not need to reserve or maintain node pools. You can directly schedule pods to elastic container instances that function as virtual nodes to ensure elasticity and reduce resource costs.
Why do you need virtual nodes?
What is a virtual node?
Nodes are the basic units that provide compute and storage resources to run workloads in ACK clusters. In most cases, an ACK cluster has at least one ECS node pool. After a pod is created, the kubelet schedules the pod to an ECS node in the node pool. This scheduling mode is suitable for applications that receive a stable volume of traffic. However, this scheduling mode cannot handle traffic spikes, even though ACK can scale out ECS instances. This is because the creation and startup of ECS instances is time-consuming. With the help of virtual nodes, you can directly schedule pods to elastic container instances. This simplifies node O&M, eliminates idle nodes, and reduces the resource cost.
A virtual node encapsulates compute resources by using the ack-virtual-node component. This way, you can deploy workloads without the need to worry about the infrastructure. ack-virtual-node will automatically schedule pods to elastic container instances. Elastic Container Instance is a serverless container service. Each elastic container instance is equivalent to a pod. To deploy applications on elastic container instances, you only need to provide a container image to deploy containers and pay for resources consumed by the containers.
Benefits
Virtual nodes provide the following benefits.
O&M-free: You do not need to manage and maintain infrastructure resources. In addition, virtual nodes are hosted resources. You do not need to perform regular node O&M operations for virtual nodes, such as system updates and patch installation.
Ultra-large capacity: You can scale out to 50,000 pods in a cluster without any plans in advance.
ImportantIf your pods are associated with large numbers of Services, we recommend that you keep less than 20,000 pods in the cluster.
Second-level scaling: You can create thousands of pods within a short period of time to handle traffic spikes.
Security isolation: You can deploy pods on elastic container instances. Instances on which pods are deployed are isolated from each other by using lightweight virtual sandboxes.
Cost reduction: Pods are created on demand and billed on a pay-as-you-go basis. The serverless architecture helps prevent resource waste and reduce O&M costs.
Scenarios
Virtual nodes are suitable for the following scenarios based on their characteristics and benefits.
Online services
For online services that need to frequently handle traffic spikes, such as online education and e-commerce, using virtual nodes can prevent system overloading caused by failures to scale out resources during peak hours and avoid resource waste during off-peak hours.
Data processing
If you use virtual nodes to handle large numbers of online concurrent tasks, such as Spark and Presto tasks, you no longer need to worry about the cost of underlying resources. You can deploy thousands of pods within a short period of time to handle big data services.
AI jobs
If you use virtual nodes, you do not need to reserve resources for long-term AI jobs that are thirsty for large amounts of compute resources, such as model training and model inference jobs. Resources can be deployed on demand and billed on a per-second basis to reduce costs. In addition, resources can be scaled out within seconds to handle unexpected jobs.
CI/CD testing
You can use virtual nodes to create and release container instances anytime in order to handle batch test tasks for CI/CD, such as CI packaging, stress tests, and simulation tests. Resources can be deployed on demand and billed on a per-second basis. This enables you to provision massive resources at a low cost.
Jobs and CronJobs
Jobs and CronJobs are automatically terminated after they are completed. The pods created by Jobs and CronJobs are also deleted. If you use virtual nodes, after a Job or CronJob is completed, resource billing automatically stops and the compute resources are released to avoid incurring unexpected costs.
Limits
Take note of the following limits before you use virtual nodes.
DaemonSets are not supported. You can replace DaemonSets with sidecar containers.
You cannot specify
HostPathorHostNetworkin podmanifests.Privileged containers are not supported. You can use a security context to add capabilities to a pod.
NoteThe privileged container feature is in internal preview. To use this feature, submit a ticket.
NodePort Services and the Session Affinity feature are not supported.
The China South Finance and Alibaba Gov Cloud regions are not supported.
Billing
The virtual node feature is free of charge. You are charged for the pods that are deployed on elastic container instances based on the billing rules of Elastic Container Instance. For more information, see Elastic Container Instance billing overview.
You are charged for Elastic Container Instance-based pods on a pay-as-you-go basis. The billing of an Elastic Container Instance-based pod starts when the pod enters the Pending state and stops when the pod enters the Succeeded or Failed state. For more information, see Lifecycle of a pod.
How to use virtual nodes
Quick start
You can refer to Schedule pods to elastic container instances that are deployed as virtual nodes to quickly learn how to schedule pods to virtual nodes.
Deploy ack-virtual-node
Install the ack-virtual-node component to enable the virtual node feature.
ACK managed cluster
In an ACK managed cluster, you need to deploy ack-virtual-node from the Add-ons page in the ACK console. By default, ack-virtual-node is managed by the cluster after it is deployed.
Log on to the ACK console. In the left-side navigation pane, click Clusters.
On the Clusters page, find the one you want to manage and click its name. In the left-side navigation pane, choose .
In the Core Components section of the Add-ons page, select ACK Virtual Node and click Install. Then, follow the on-screen instructions to complete the installation.
The default vSwitches and security group of the cluster are used for elastic container instances that are deployed by ack-virtual-node. If you want to change the vSwitches or security group, configure the corresponding eci-profile.
ACK dedicated cluster
In an ACK dedicated cluster, you need to deploy ack-virtual-node from the Marketplace page in the ACK console. After ack-virtual-node is installed, a Deployment named ack-virtual-node-controller is created in the kube-system namespace. The Deployment runs on worker nodes in the cluster.
Log on to the ACK console. In the left-side navigation pane, choose .
On the Marketplace page, click the App Catalog tab. Find and click ack-virtual-node. On the ack-virtual-node page, click Deploy.
In the Deploy panel, select a cluster and namespace, and then click Next.
Namespace is automatically set to kube-system. Release Name is automatically set to ack-virtual-node.
In the Parameters step, select the latest chart version, set virtual node parameters in the Parameters section, and then click OK.
Parameter
Required
Description
How to obtain the value
ALIYUN_CLUSTERID
Required
The ID of the cluster.
Go to the Cluster Information page. Click the Basic Information tab to view the cluster ID.
ALIYUN_RESOURCEGROUP_ID
Optional
The ID of the resource group.
If you do not specify this parameter, the default resource group is used. To specify a resource group, log on to the Resource Management console to obtain the ID of the resource group that you want to use.
ECI_REGION
Required
The region ID.
Go to the Cluster Information page. Click the Basic Information tab. In the Basic Information section, you can view the region where the cluster is deployed.
NoteFor more information about region names and the corresponding region IDs, see Regions and zones.
ECI_VPC
Optional
The ID of the VPC.
Go to the Cluster Information page. Click the Basic Information tab to view the cluster ID.
ECI_VSWITCH
Required
The IDs of vSwitches.
You can specify the vSwitches that are used to allocate IP addresses to pods. You can specify multiple IDs in the following format:
vsw-xxx1, vsw-xxx2. We recommend that you specify the vSwitches used by the node pools in the cluster.On the Node Pools page, click the ID of a node pool. Click the Overview tab. In the Node Configurations section, you can view the IDs of the vSwitches used by the nodes in the node pool.
NoteMake sure that the vSwitches you specify are deployed in the zones supported by Elastic Container Instance.
ECI_SECURITY_GROUP
Required
The ID of the security group.
Go to the Cluster Information page. Click the Basic Information tab to view the security group ID.
ECI_ACCESS_KEY
Required
The AccessKey ID of the Resource Access Management (RAM) user you use.
For more information, see Obtain an AccessKey pair.
You must attach the AliyunECIFullAccess policy to the RAM user in the RAM console. For more information, see Grant permissions to a RAM user.
ECI_SECRET_KEY
Required
The AccessKey ID of the RAM user you use.
For more information, see Obtain an AccessKey pair.
You must attach the AliyunECIFullAccess policy to the RAM user in the RAM console. For more information, see Grant permissions to a RAM user.
KUBERNETES_APISERVER_HOST
Required
The IP address of the API server of the cluster.
The IP address and port that the API server uses to provide services within the cluster. Go to the Cluster Information page. Click the Basic Information tab to view the internal endpoint of the API server of the cluster.
KUBERNETES_APISERVER_PORT
Required
The port of the API Server.
Check whether ack-virtual-node is deployed.
kubectl -n kube-system get deploy ack-virtual-node-controllerExpected output:
NAME READY UP-TO-DATE AVAILABLE AGE ack-virtual-node-controller 1/1 1 1 2m31s
Schedule pods to elastic container instances
ACK provides various scheduling solutions that can meet your requirements in scenarios where both elastic container instances and ECS instances are used to deploy pods. For more information, see Schedule a pod to a virtual node.
By default, Elastic Container Instance-based pods use the x86 CPU architecture and the Linux operating system. If you want to create an Elastic Container Instance-based pod that uses the ARM architecture or the Windows operating system, refer to Schedule workloads to ARM-based virtual nodes or (Invitational preview) Schedule workloads to Windows virtual nodes.
Configure Elastic Container Instance-based pods
You can add pod annotations to use some Elastic Container Instance features, such as specifying elastic container instance specifications, enabling image cache to accelerate pod creation, assigning IPv6 addresses to Elastic Container Instance-based pods, and expanding the ephemeral storage. For more information, see Pod annotations.
You can use the Elastic Container Instance Effect feature of eci-profile to dynamically add annotations to Elastic Container Instance-based pods in batches.
Manage virtual nodes
Upgrade ack-virtual-node
If you want to use the advanced features of virtual nodes, upgrade ack-virtual-node.
The upgrade requires approximately 1 minute. During the upgrade, you cannot create new pods. Existing pods are not affected.
ACK managed cluster
Log on to the ACK console. In the left-side navigation pane, click Clusters.
On the Clusters page, find the one you want to manage and click its name. In the left-side navigation pane, choose .
In the Core Components section of the Add-ons page, select ACK Virtual Node and click Upgrade. Then, follow the on-screen instructions to complete the upgrade.
ACK dedicated cluster
Log on to the ACK console. In the left-side navigation pane, click Clusters.
On the Clusters page, find the cluster that you want to manage and click its name. In the left-side navigation pane, choose .
On the Helm page, find ack-virtual-node and click Update in the Actions column. In the Update Release panel, select the latest version from the Version drop-down list.
Update the parameters in the YAML template based on your business requirements and click OK.
In addition, you can modify the
virtualNode.image.tagfield to upgrade the image version for the virtual node.
Modify the configurations of a virtual node
An eci-profile specifies the VPC and vSwitch used by an Elastic Container Instance-based pod and also specifies whether the pod uses the ARM architecture. You can update the parameters in the data section of the eci-profile.
Delete a virtual node
After you uninstall ack-virtual-node, virtual nodes and Elastic Container Instance-based pods are automatically deleted.
Make sure that your business is not affected after Elastic Container Instance-based pods are deleted.
Uninstall ack-virtual-node.
ACK managed cluster: Uninstall ack-virtual-node on the Add-ons page in the ACK console.
ACK dedicated cluster: Uninstall ack-virtual-node on the Helm page in the ACK console.