Schedule pods to elastic container instances that are deployed as virtual nodes in an ACK cluster - Container Service for Kubernetes

When you use a Container Service for Kubernetes (ACK) cluster, you may need to launch a large number of pods within a short period of time. If you choose to create Elastic Compute Service (ECS) instances for the pods, the creation process is time-consuming. If you choose to reserve ECS instances, the instances are idle before pod creation and after pod termination, resulting in resource waste. To address this issue, we recommend that you schedule the pods to elastic container instances that are deployed as virtual nodes. This way, you do not need to purchase or manage ECS instances. This topic describes how to schedule pods to elastic container instances that are deployed as virtual nodes in an ACK managed cluster or ACK dedicated cluster.

How elastic container instances run in ACK clusters

Elastic Container Instance is a serverless compute service that is provided by Alibaba Cloud for containerization. You can use elastic container instances to efficiently set up an O&M-free and isolated runtime environment for your containers. Elastic container instances allow you to focus on containerized applications without the need to purchase or manage Elastic Compute Service (ECS) instances. This eliminates the need for infrastructure maintenance. You can create elastic container instances on demand. You are charged for resource usage on a per second basis.

In most cases, an ACK cluster has at least one node pool. After you create a pod, ACK schedules the pod to an ECS instance in the node pool. This scheduling mode is suitable for applications that receive a stable volume of traffic. However, this scheduling mode cannot handle traffic spikes, even though ACK can scale out ECS instances. This is because the creation and startup of ECS instances is time-consuming. In this case, we recommend that you schedule pods to elastic container instances that are deployed as virtual nodes. This eliminates the need to add new nodes, prevents resource waste, and reduces computing costs.

Scenarios

You can use elastic container instances to handle traffic spikes and reduce computing costs. Elastic container instances are suitable for the following scenarios:

Online business that requires elastic scaling to handle traffic fluctuations, such as online education and e-commerce. You can use elastic container instances to efficiently handle traffic spikes. In addition, elastic container instances can optimize resource pool maintenance. This helps you reduce computing costs.
Computing tasks that are run on demand. You can use elastic container instances to run such tasks. This way, you do not need to retain nodes when no tasks are run and you are charged only for computing resources that are actually used by computing tasks. The following types of computing tasks are included:
- Data processing tasks run by using Spark or Presto.
- Continuous integration and continuous delivery (CI/CD) pipelines that are built by using Jenkins and GitLab Runner.
- Jobs in AI computing scenarios and CronJobs.

Prerequisites

An ACK managed cluster or ACK dedicated cluster is created and the Kubernetes version of the cluster is 1.16 or later. For more information about how to create an ACK cluster, see Create an ACK managed cluster or Create an ACK dedicated cluster. For more information about how to update an ACK cluster, see Update an ACK cluster.
Elastic Container Instance is activated and the region where the cluster is deployed is supported by Elastic Container Instance.
You can log on to the Elastic Container Instance console to activate the service and view the regions and zones supported by the service.
A kubectl client is connected to the cluster. For more information, see Use kubectl on Cloud Shell to manage ACK clusters.

Step 1: Deploy ack-virtual-node in the cluster

In an ACK managed cluster, ack-virtual-node is installed and managed in a different way than in an ACK dedicated cluster.

In an ACK managed cluster, you need to deploy ack-virtual-node from the Add-ons page of the ACK console. By default, ack-virtual-node is managed by the cluster after it is deployed.
In an ACK dedicated cluster, you need to deploy ack-virtual-node from the Marketplace page of the ACK console. After ack-virtual-node is installed, a Deployment named ack-virtual-node-controller is created in the kube-system namespace. The Deployment runs on worker nodes in the cluster.

ACK managed cluster

Log on to the ACK console. In the left-side navigation pane, click Clusters.
On the Clusters page, find the cluster that you want to manage and click its name. In the left-side navigation pane, choose Operations > Add-ons.
In the Core Components section of the Add-ons page, select ack-virtual-node and click Install. Then, follow the instructions on the page to complete the installation.
The default vSwitch and security group of the cluster are used for elastic container instances that are deployed by ack-virtual-node. If you want to modify these settings, see Related operations.

ACK dedicated cluster

Log on to the ACK console. In the left-side navigation pane, choose Marketplace > Marketplace.
On the Marketplace page, click the App Catalog tab. Find and click ack-virtual-node. On the ack-virtual-node page, click Deploy.
In the Deploy panel, select a cluster and namespace, and click Next.
Namespace is automatically set to kube-system. Release Name is automatically set to ack-virtual-node.

On the Parameters wizard page, select the latest chart version, set virtual node parameters in the Parameters section, and then click OK.

Parameter	Required	Description	How to obtain the value
ALIYUN_CLUSTERID	Yes	The cluster ID.	Go to the cluster details page. Click the Basic Information tab to view the cluster ID.
ALIYUN_RESOURCEGROUP_ID	No	The resource group ID.	If you do not specify this parameter, the default resource group is used. To specify a resource group, log on to the Resource Management console to obtain the ID of the resource group that you want to use.
ECI_REGION	Yes	The region ID.	Go to the cluster details page. Click the Basic Information tab. In the Basic Information section, you can view the region where the cluster is deployed. Note For more information about region names and the corresponding region IDs, see Regions and zones.
ECI_VPC	No	The VPC ID.	Go to the cluster details page. Click the Cluster Resources tab to view the ID of the VPC where the cluster is deployed.
ECI_VSWITCH	Yes	The vSwitch IDs.	You can specify the vSwitches that are used to allocate IP addresses to pods. You can specify multiple IDs in the following format: `vsw-xxx1, vsw-xxx2`. We recommend that you specify the vSwitches used by the node pools in the cluster. On the Node Pools page, click the ID of a node pool. Click the Overview tab. In the Node Configurations section, you can view the IDs of the vSwitches used by the nodes in the node pool. Note Make sure that the vSwitches you specify are deployed in the zones supported by Elastic Container Instance.
ECI_SECURITY_GROUP	Yes	The security group ID.	Go to the cluster details page. Click the Cluster Resources tab to view the ID of the security group used by the cluster.
ECI_ACCESS_KEY	Yes	The AccessKey ID of the Resource Access Management (RAM) user you use.	For more information, see Obtain an AccessKey pair. You must attach the AliyunECIFullAccess policy to the RAM user in the RAM console. For more information, see Grant permissions to a RAM user.
ECI_SECRET_KEY	Yes	The AccessKey ID of the RAM user you use.	For more information, see Obtain an AccessKey pair. You must attach the AliyunECIFullAccess policy to the RAM user in the RAM console. For more information, see Grant permissions to a RAM user.
KUBERNETES_APISERVER_HOST	No	The IP address of the Kubernetes API server of the cluster.	If you leave the parameters empty, the internal endpoint of the Kubernetes API server of the cluster is used by default. Go to the cluster details page. Click the Basic Information tab to view the internal endpoint of the Kubernetes API server of the cluster. To use a custom endpoint, modify the parameters based on your business requirements.
KUBERNETES_APISERVER_PORT	No	The port of the Kubernetes API server of the cluster.

Run the following command to check whether ack-virtual-node is deployed:

kubectl -n kube-system get deploy ack-virtual-node-controller

Expected output:

NAME                          READY   UP-TO-DATE   AVAILABLE   AGE
ack-virtual-node-controller   1/1     1            1           2m31s

Step 2: Schedule pods to elastic container instances

After ack-virtual-node is deployed in your cluster, you can schedule pods to elastic container instances that are deployed as virtual nodes in your cluster. You can use one of the following methods to schedule pods to elastic container instances in an ACK managed cluster or ACK dedicated cluster.

If you use an ACK Pro cluster, you can configure the cluster to schedule pods only to elastic container instances, prioritize ECS instances for pod scheduling, or configure the cluster to schedule pods only to ECS instances. For more information, see Configure Elastic Container Instance-based scaling.

Schedule pods to elastic container instances by adding pod labels

To schedule pods to elastic container instances, you can perform the following steps to add the alibabacloud.com/eci=true label to pod configurations:

Run the following command to add the alibabacloud.com/eci=true label to pod configurations:
```
kubectl run nginx --image nginx -l alibabacloud.com/eci=true
```

Run the following command to check whether a pod is scheduled to an elastic container instance that is deployed as a virtual node in the cluster:

kubectl get pod -o wide|grep virtual-kubelet

Expected output:

nginx-7fc9f746b6-r4xgx     1/1     Running   0          34s   192.XX.XX.108   virtual-kubelet-cn-hangzhou-k        <none>           <none>

Schedule pods to elastic container instances by adding namespace labels

To schedule pods to elastic container instances, you can perform the following steps to add the alibabacloud.com/eci=true label to the namespaces where you want to deploy the pods.

Run the following command to create a namespace named vk:
```
kubectl create ns vk
```
Run the following command to add the alibabacloud.com/eci=true label to the vk namespace.
```
kubectl label namespace vk alibabacloud.com/eci=true
```
Run the following command to create a pod in the vk namespace and schedule the pod to a virtual node:
```
kubectl -n vk run nginx --image nginx
```

Run the following command to check whether a pod is scheduled to an elastic container instance that is deployed as a virtual node in the cluster:

kubectl -n vk get pod -o wide|grep virtual-kubelet

Expected output:

nginx-6f489b847d-vgj4d      1/1     Running             0          1m   192.XX.XX.108   virtual-kubelet-cn-hangzhou-k        <none>           <none>

Related operations

If you want to use the advanced features of virtual nodes, update ack-virtual-node or modify the virtual node configurations. If you no longer need to use virtual nodes, you can uninstall ack-virtual-node.

Update ack-virtual-node

The update requires approximately 1 minute. During the update, you cannot create new pods. Existing pods are not affected.

ACK managed cluster

Log on to the ACK console. In the left-side navigation pane, click Clusters.
On the Clusters page, find the cluster that you want to manage and click its name. In the left-side navigation pane, choose Operations > Add-ons.
In the Core Components section of the Add-ons page, select ack-virtual-node and click Update. Then, follow the instructions on the page to complete the installation.

ACK dedicated cluster

Log on to the ACK console. In the left-side navigation pane, click Clusters.
On the Clusters page, find the cluster that you want to manage and click its name. In the left-side navigation pane, choose Applications > Helm.
On the Helm page, find ack-virtual-node and click Update in the Actions column. In the Update Release panel, select the latest version from the Version drop-down list.
Modify the required parameters based on the parameter description in Step 1: Deploy ack-virtual-node in the cluster. If optional parameters are included, modify the parameters based on the parameter description in Step 1: Deploy ack-virtual-node in the cluster. Then, click OK.
In addition, you can modify the virtualNode.image.tag field to update the image version for the virtual node.

Modify the configurations of virtual nodes in an ACK cluster

For more information, see Configure an eci-profile.

Delete virtual nodes in ACK clusters

Uninstall ack-virtual-node.
- After you delete all Elastic Container Instance-based pods in an ACK managed cluster, uninstall ack-virtual-node on the Add-ons page.
- After you delete all Elastic Container Instance-based pods in an ACK dedicated cluster, delete ack-virtual-node on the Helm page.
Run the kubectl delete node <node name> to delete the virtual nodes that you do not need.
Note
After you uninstall ack-virtual-node, existing elastic container instances in your cluster are not deleted.