This topic describes virtual nodes, elastic container instances (ECIs), and how to deploy ack-virtual-node in a cluster of Container Service for Kubernetes (ACK). You can schedule a pod to a virtual node by using ack-virtual-node.
Virtual nodes and ECI
Elastic Container Instance (ECI) is a container-oriented serverless computing service. It provides maintenance-free container runtimes that support strong isolation and quick startup. ECI allows you to focus on container-based applications without the need to purchase or manage Elastic Compute Service (ECS) instances. This saves you the hassle of infrastructure maintenance. You can create ECIs based on your requirements. You are charged based on the resource usage during container execution time.
Based on Virtual Kubelet, virtual nodes enable seamless integration between Kubernetes and ECI. This significantly improves the elasticity of Kubernetes clusters and eliminates the constraint of the computing power that is provided by cluster nodes. For more information about how Virtual Kubelet works and its architecture, see Virtual Kubelet.
- Online business with periodic traffic patterns, such as online education and e-commerce. Virtual nodes can significantly reduce computing costs by optimizing resource pool maintenance.
- Virtual nodes can effectively reduce costs in computing scenarios where Spark or Presto is used for data processing.
- CI/CD Pipeline: Jenkins and Gitlab-Runner.
- Jobs: Jobs in Artificial Intelligence (AI) scenarios and Cron Jobs.
Deploy ack-virtual-node in an ACK cluster
- A managed Kubernetes cluster or dedicated Kubernetes cluster is created. For more information, see Create a cluster of ACK Managed Edition.
- ECI is activated. You can log on to the ECI console to activate ECI.
- The region where the cluster is deployed must be supported by ECI. To view supported regions and zones, go to the ECI console.
- Log on to the ACK console.
- In the left-side navigation pane, choose .
- On the App Catalog page, click the Alibaba Cloud Apps tab, and find and click ack-virtual-node.In the upper-right corner of the App Catalog page, you can enter ack-virtual-node into the Name search bar and click the search icon. You can also enter a keyword to perform a fuzzy match.
- On the App Catalog - ack-virtual-node page, select a cluster in the Deploy section to deploy the application.Namespace is automatically set to kube-system. Release Name is automatically set to ack-virtual-node.
- On the App Catalog - ack-virtual-node page, click the Parameters tab and set the parameters. Then, click Create in the Deploy section.
Parameter Description How to obtain the value ALIYUN_CLUSTERID The ID of the cluster where you want to install ack-virtual-node. On the details page of the cluster, click the Basic Information tab. In the Basic Information section, you can obtain the ID of the cluster. ALIYUN_RESOURCEGROUP_ID The ID of the resource group. If you do not specify the parameter, the default resource group is used. To specify a resource group, log on to the Resource Management console to obtain the ID of the resource group. ECI_REGION The name of the region where the cluster is deployed. On the details page of the cluster, click the Basic Information tab. In the Basic Information section, you can check the region where the cluster is deployed, for example, cn-hangzhou represents the China (Hangzhou) region. ECI_VSWITCH VSwitches. On the Nodes page, click the ID of a node. On the instance details page, you can obtain the value of VSwtich in the Configuration Information section.Note
You must make sure that the zone of the VSwitch is supported by ECI.
VSwitches support multiple zones. You can specify multiple VSwitches in the following format:
ECI_VSWITCH: "vsw-xxxxxxx1, vsw-xxxxxxx2, vsw-xxxxxxx3".
ECI_SECURITY_GROUP The ID of the security group. On the Nodes page, click the ID of a node. In the left-side navigation pane, click Security Groups. Click the Security Groups tab to obtain the ID of the security group. ECI_ACCESS_KEY The AccessKey ID of your Alibaba Cloud account. For more information, see How can I obtain an AccessKey pair?. ECI_SECRET_KEY The AccessKey secret of your Alibaba Cloud account. For more information, see How can I obtain an AccessKey pair?.
- After ack-virtual-node is installed, you can view information about the newly added
virtual-node-eci node by performing the following steps:
- In the left-side navigation pane, click Clusters.
- On the Clusters page, click the name of a cluster or click Applications in the Actions column.
- In the left-side navigation pane, click Nodes. On the Nodes page, you can find the newly added virtual-node-eci node.
- Run the following commands to query the deployment status of virtual-node-controller and virtual-node-admission-controller: For more information, see Use kubectl on Cloud Shell to manage ACK clusters.Run the following command:
kubectl -n kube-system get statefulset virtual-node-eciThe following output is returned:
NAME READY AGE virtual-node-eci 1/1 1mRun the following command:
kubectl -n kube-system get deploy ack-virtual-node-affinity-admission-controllerThe following output is returned:
NAME READY UP-TO-DATE AVAILABLE AGE ack-virtual-node-affinity-admission-controller 1/1 1 1 1mRun the following command:
kubectl -n kube-system get pod|grep virtual-node-eciThe following output is returned:
virtual-node-eci-0 1/1 Running 0 1mRun the following command:
kubectl get no|grep virtual-node-eciThe following output is returned:
virtual-node-eci-0 Ready agent 1m v1.11.2-aliyun-1.0.207
Schedule a pod to a virtual node
- Set nodeSelector and tolerations.
Virtual nodes have specific taints. You must set nodeSelector and tolerations for a pod before you can schedule the pod to a virtual node. Example:
apiVersion: v1 kind: Pod metadata: name: nginx spec: containers: - image: nginx imagePullPolicy: Always name: nginx nodeSelector: type: virtual-kubelet tolerations: - key: virtual-kubelet.io/provider operator: Exists
- Attach labels to the pod.
The virtual-node-affinity-admission-controller webhook automatically schedules pods with specified labels to a virtual node. The following example uses the eci=true label. Example:Run the following command:
kubectl run nginx --image nginx -l eci=true
kubectl get pod -o wide|grep virtual-node-eciThe following output is returned:
nginx-7fc9f746b6-r4xgx 0/1 ContainerCreating 0 20s 192.168. *.* virtual-node-eci-0 <none> <none>
- Attach a label to the namespace where the pod is deployed.
The virtual-node-affinity-admission-controller webhook automatically schedules pods in a namespace with specified labels to a virtual node. The following example uses the virtual-node-affinity-injection=enabled label. Example:Run the following command:
kubectl create ns vk
kubectl label namespace vk virtual-node-affinity-injection=enabled
kubectl -n vk run nginx --image nginx
kubectl -n vk get pod -o wide|grep virtual-node-eciThe following output is returned:
nginx-6f489b847d-vgj4d 1/1 Running 0 1m 192.168. *.* virtual-node-eci-0 <none> <none>
Modify the configurations of the virtual node controller
The configurations of the virtual node controller determine how pods are scheduled to a virtual node and specify the pod runtime environment such as VSwitches and security group settings. You can modify the configurations of the controller based on your requirements. Modified configurations apply to only pods that are scheduled after modifications and do not apply to existing pods that run on the node.
kubectl -n kube-system edit statefulset virtual-node-eci
- Upgrade the virtual node controller version.
To use the latest features of virtual nodes, you must upgrade the virtual node controller to the latest version. For example, to enable pods to access ClusterIP type Services, the virtual node controller version must be later than v184.108.40.206-aliyun.
- Modify security group settings.
You can modify the ECI_SECURITY_GROUP environment variable to change the security group that is associated with pods that are scheduled to the virtual node.
- Modify VSwitch settings.
You can modify the ECI_VSWITCH environment variable to change the VSwitch to which the pods that are scheduled to the virtual node belong. We recommend that you configure multiple VSwitches that are deployed in different zones to ensure high availability. When ECI resources are insufficient in a zone, the controller can create pods in another zone.
- Modify kube-proxy settings.
By default, the ECI_KUBE_PROXY environment variable is set to true. This indicates that pods can access ClusterIP type Services. If the pods no longer need to access ClusterIP type Services, you can set the environment variable to false to disable kube-proxy. In some large-scale scenarios, a cluster may need to start a large number of pods. This significantly increases the number of concurrent connections between kube-proxy and the Kubernetes API server. In this case, you can disable kube-proxy to reduce the heavy loads on the API server. However, you can also configure PrivateZone to enable pods to access Services in the cluster.
- Create multiple virtual nodes.
We recommend that you deploy a maximum of 3,000 pods on a virtual node. To create more pods, you can create more virtual nodes. You can modify the number of replicas in the StatefulSet configurations to create more virtual nodes. The number of replicas represents the number of virtual nodes in the cluster. Each virtual node is related to a virtual node controller that manages the pods on the virtual node. The controllers do not interfere with each other. Perform the following steps to modify the StatefulSet configurations:Run the following command:
kubectl -n kube-system scale statefulset virtual-node-eci --replicas=4The following output is returned:
statefulset.apps/virtual-node-eci scaledRun the following command:
kubectl get noThe following output is returned:
NAME STATUS ROLES AGE VERSION cn-hangzhou.192.168.1.1 Ready <none> 63d v1.12.6-aliyun.1 cn-hangzhou.192.168.1.2 Ready <none> 63d v1.12.6-aliyun.1 virtual-node-eci-0 Ready agent 6m v1.11.2-aliyun-1.0.207 virtual-node-eci-1 Ready agent 1m v1.11.2-aliyun-1.0.207 virtual-node-eci-2 Ready agent 1m v1.11.2-aliyun-1.0.207 virtual-node-eci-3 Ready agent 1m v1.11.2-aliyun-1.0.207
Delete a virtual node
In most cases, you do not need to delete virtual nodes because they do not occupy computing resources. If you want to delete a virtual node, we recommend that you manually evict or delete the pods that run on the node before you delete the virtual node controller and the node. If you delete the virtual node controller before pods are deleted, some pods may fail to be deleted.
kubectl drain virtual-node-eci-0 ...
kubectl -n kube-system delete statefulset virtual-node-eci
kubectl delete no virtual-node-eci-0 ...