This topic describes virtual nodes and elastic container instances (ECIs). It also describes how to deploy a virtual node add-on in a cluster of Container Service for Kubernetes (ACK). You can schedule a pod to a virtual node by using ack-virtual-node.
Virtual nodes and ECI
ECI is a serverless compute service that is provided by Alibaba Cloud for containerized business. You can use ECI to quickly set up an operations and maintenance (O&M)-free and isolated runtime environment for your application containers. ECI allows you to focus on container-based applications without the need to purchase or manage Elastic Compute Service (ECS) instances. This saves you the hassle of infrastructure maintenance. You can create ECIs based on your business requirements. You are charged for resource usage on a per second basis.
Based on Virtual Kubelet, virtual nodes enable seamless integration between Kubernetes and ECI. This significantly improves the elasticity of Kubernetes clusters and eliminates the constraint of the computing capacity that is provided by cluster nodes. For more information about how Virtual Kubelet works and its architecture, see Virtual Kubelet.
- Online business with periodic traffic patterns, such as online education and e-commerce. Virtual nodes significantly reduce computing costs by optimizing resource pool maintenance.
- Virtual nodes effectively reduce costs in computing scenarios where Spark or Presto is used to process data.
- CI/CD Pipeline: Jenkins and Gitlab-Runner.
- Jobs: Jobs in Artificial Intelligence (AI) scenarios and CronJobs.

Deploy ack-virtual-node in an ACK cluster
- A managed Kubernetes cluster or dedicated Kubernetes cluster is created. For more information, see Create a managed kubernetes cluster.
- ECI is activated. You can log on to the ECI console to activate ECI.
- The region where the created cluster is deployed must be supported by ECI. To view the supported regions and zones, go to the ECI console.
Schedule a pod to a virtual node
- Set node selectors and tolerations for the pod.
Virtual nodes have specific taints. You must set node selectors and tolerations for a pod before you can schedule the pod to a virtual node. The following code block is a YAML template that is used to create a pod:
apiVersion: v1 kind: Pod metadata: name: nginx spec: containers: - image: nginx imagePullPolicy: Always name: nginx nodeSelector: type: virtual-kubelet tolerations: - key: virtual-kubelet.io/provider operator: Exists
- Attach labels to the pod .
The virtual-node-affinity-admission-controller webhook automatically schedules pods with specified labels to a virtual node. The eci=true label is used in the following example. Example:
Run the following commands:kubectl run nginx --image nginx -l eci=true
kubectl get pod -o wide|grep virtual-node-eci
The following output is returned:nginx-7fc9f746b6-r4xgx 0/1 ContainerCreating 0 20s 192.168. *.* virtual-node-eci-0 <none> <none>
- Add a label to the namespace where the pod is deployed.
The virtual-node-affinity-admission-controller webhook automatically schedules pods that are created in a namespace with specified labels to a virtual node. The virtual-node-affinity-injection=enabled label is used in the following example. Example:
Run the following commands:kubectl create ns vk
kubectl label namespace vk virtual-node-affinity-injection=enabled
kubectl -n vk run nginx --image nginx
kubectl -n vk get pod -o wide|grep virtual-node-eci
The following output is returned:nginx-6f489b847d-vgj4d 1/1 Running 0 1m 192.168. *.* virtual-node-eci-0 <none> <none>
Modify the configurations of the virtual node controller
The configurations of the virtual node controller determine how pods are scheduled to a virtual node and specify the pod runtime environment such as vSwitches and security group settings. You can modify the configurations of the controller based on your business requirements. Modified configurations apply to only pods that are scheduled after modifications and do not apply to existing pods that run on the node.
kubectl -n kube-system edit statefulset virtual-node-eci
- Upgrade the virtual node controller version.
To use the latest features of virtual nodes, you must upgrade the virtual node controller to the latest version. For example, to enable pods to access ClusterIP type Services, the virtual node controller version must be later than v1.0.0.2-aliyun.
- Modify security group settings.
You can modify the ECI_SECURITY_GROUP environment variable to change the security group that is associated with pods that are scheduled to the virtual node.
- Modify vSwitch settings.
You can modify the ECI_VSWITCH environment variable to change the vSwitch to which pods that are scheduled to the virtual node belong. We recommend that you configure multiple vSwitches that are deployed in different zones to ensure high availability. When ECI resources are insufficient in a zone, the controller can create pods in another zone.
- Modify kube-proxy settings.
By default, the ECI_KUBE_PROXY environment variable is set to true. This indicates that pods can access ClusterIP type Services. If the pods no longer need to access ClusterIP type Services, you can set the environment variable to false to disable kube-proxy. In some large-scale scenarios, a cluster may need to start a large number of pods. This significantly increases the number of concurrent connections between kube-proxy and the Kubernetes API server. In this case, you can disable kube-proxy to reduce the heavy loads on the API server. However, you can also configure PrivateZone to enable pods to access Services in the cluster.
- Create multiple virtual nodes.
We recommend that you deploy up to 3,000 pods on a virtual node. To create more pods, you can create more virtual nodes. You can modify the number of replicas in the StatefulSet configurations to create more virtual nodes. The number of replicas represents the number of virtual nodes in the cluster. Each virtual node is related to a virtual node controller that manages the pods on the virtual node. The controllers do not interfere with each other. Perform the following steps to modify the StatefulSet configurations:
Run the following command:kubectl -n kube-system scale statefulset virtual-node-eci --replicas=4
The following output is returned:statefulset.apps/virtual-node-eci scaled
Run the following command:kubectl get no
The following output is returned:NAME STATUS ROLES AGE VERSION cn-hangzhou.192.168.1.1 Ready <none> 63d v1.12.6-aliyun.1 cn-hangzhou.192.168.1.2 Ready <none> 63d v1.12.6-aliyun.1 virtual-node-eci-0 Ready agent 6m v1.11.2-aliyun-1.0.207 virtual-node-eci-1 Ready agent 1m v1.11.2-aliyun-1.0.207 virtual-node-eci-2 Ready agent 1m v1.11.2-aliyun-1.0.207 virtual-node-eci-3 Ready agent 1m v1.11.2-aliyun-1.0.207
Delete a virtual node
In most cases, you do not need to delete virtual nodes because they do not occupy the computing resources of the cluster. To delete a virtual node, we recommend that you manually evict or delete the pods that run on the node before you delete the virtual node controller and the node. If you delete the virtual node controller before pods are deleted, some ECIs may fail to be deleted.
kubectl drain virtual-node-eci-0 ...
kubectl -n kube-system delete statefulset virtual-node-eci
kubectl delete no virtual-node-eci-0 ...