Schedule pods to elastic container instances on virtual nodes - Container Service for Kubernetes

When you need to create many Pods quickly, scaling out ECS instances may not be fast enough, and reserving extra instances wastes resources. ACK virtual nodes let you schedule Pods to Elastic Container Instance (ECI) without provisioning or managing ECS instances. This topic describes how to schedule Pods to run on ECI in an ACK managed cluster.

Usage notes

For kube-scheduler v6.11.0 or later, if you create more than 1,000 ECI pods concurrently by adding labels, upgrade to the latest version and enable the SchedulerBatchMoveEvents=true feature gate.

Use cases

Running Pods on ECI is ideal for handling traffic spikes and reducing compute costs. Typical use cases include:

Online services with fluctuating traffic patterns
ECI pods help you respond to traffic spikes faster, reduce maintenance on fixed resource pools, and lower compute costs.
Short-lived or batch computing tasks
When you run computing tasks as ECI pods, you do not need to maintain dedicated Nodes and pay only for the compute resources used during task execution. This approach reduces costs for workloads such as Spark and AI jobs.

Prerequisites

Your ACK cluster must be running Kubernetes 1.16 or later and be in a region that supports ECI.

Note

To check if your cluster's region is supported, see Regions and zones supported by ECI. Then, log on to the Elastic Container Instance console to activate the ECI service.

Procedure

Step 1: Deploy the ack-virtual-node component

Note

The following steps use an ACK managed cluster as an example. For an ACK dedicated cluster, you must deploy the ack-virtual-node component from the Marketplace page. For more information, see Deploy the ack-virtual-node component.

Log on to the ACK console. In the left navigation pane, click Clusters.
On the Clusters page, click the name of your cluster. In the left navigation pane, click Components and Add-ons.
On the Add-ons page, click the Core Components tab, find ACK Virtual Node, and then click Install. Follow the on-screen instructions to complete the installation.
During the installation, the default vSwitch and security group of the cluster are used as the initial ECI configuration parameters. To modify the parameters, you can update them by configuring an eci-profile.

Step 2: Schedule a pod to ECI

Note

The following examples show how to schedule a Pod to a virtual node to run in ECI by adding a label to the Pod or Namespace. For more scheduling methods, see Schedule a Pod to a virtual node.

Pod

To run an individual Pod in ECI, add the label alibabacloud.com/eci: "true" to the Pod. This schedules the Pod to an x86-architecture virtual node to run as an ECI pod.

Create a Deployment from the following YAML file.

kubectl create -f eci-pod.yaml

The following code shows an example of the eci-pod.yaml file:

Note

In the following YAML, the k8s.aliyun.com/eci-use-specs annotation specifies the compute specification for the ECI pod. For more information about configuring ECI pod specifications, see Specify the compute specification of an ECI pod.

apiVersion: apps/v1
kind: Deployment
metadata:
  name: test
  labels:
    app: test
spec:
  replicas: 2
  selector:
    matchLabels:
      app: nginx
  template:
    metadata:
      name: test
      labels:
        app: nginx
        alibabacloud.com/eci: "true"             # Add this label to schedule the Pod to ECI. 
      annotations:
        k8s.aliyun.com/eci-use-specs: "2-4Gi"   # Specify 2 vCPUs and 4 GiB of memory for the ECI pod.
    spec:
      containers:
      - name: nginx
        image: anolis-registry.cn-zhangjiakou.cr.aliyuncs.com/openanolis/nginx:1.14.1-8.6
        ports:
        - containerPort: 80

Verify that the Pod is running on a virtual node.

kubectl get pod -o wide -l app=nginx

The expected output shows that the Pod has been scheduled to a virtual node, as indicated in the NODE column.

NAME                    READY   STATUS    RESTARTS   AGE   IP            NODE                            NOMINATED NODE   READINESS GATES
test-86f7fbc94f-g5m22   1/1     Running   0          38s   10.16.XX.XX   virtual-kubelet-cn-hangzhou-j   <none>           <none>
test-86f7fbc94f-r4wcn   1/1     Running   0          38s   10.16.XX.XX   virtual-kubelet-cn-hangzhou-j   <none>           <none>

View the details of the ECI pod.
```
kubectl get pod <pod-name> -o yaml
```
In the YAML output, find the corresponding ECI instance ID in the k8s.aliyun.com/eci-instance-id annotation and confirm the compute specification of the ECI pod used for billing in the k8s.aliyun.com/eci-instance-spec annotation.

Namespace

If you want to run a specific category of Pods in ECI, you can create a Namespace and add the label alibabacloud.com/eci: "true" to it. All Pods created in this Namespace are then scheduled to an x86-architecture virtual node to run as ECI pods.

Create a Namespace named vk and add the required label to it.

kubectl create ns vk
kubectl label namespace vk alibabacloud.com/eci=true

Use the following YAML to create a Deployment in the vk namespace.

kubectl create -f eci-namespace.yaml

The following code shows an example of the eci-namespace.yaml file:

Note

apiVersion: apps/v1
kind: Deployment
metadata:
  name: test
  namespace: vk       # Specify the labeled Namespace to schedule Pods to ECI.
  labels:
    app: test
spec:
  replicas: 2
  selector:
    matchLabels:
      app: nginx
  template:
    metadata:
      name: test
      labels:
        app: nginx
      annotations:
        k8s.aliyun.com/eci-use-specs: "2-4Gi"   # Specify 2 vCPUs and 4 GiB of memory for the ECI pod.
    spec:
      containers:
      - name: nginx
        image: anolis-registry.cn-zhangjiakou.cr.aliyuncs.com/openanolis/nginx:1.14.1-8.6
        ports:
        - containerPort: 80

Verify that the Pod is running on a virtual node.

kubectl get pod -o wide -l app=nginx -n vk

The expected output shows that the Pod has been scheduled to a virtual node, as indicated in the NODE column.

NAME                   READY   STATUS    RESTARTS   AGE   IP            NODE                            NOMINATED NODE   READINESS GATES
test-8f54bcfb5-86pvc   1/1     Running   0          14s   10.16.XX.XX   virtual-kubelet-cn-hangzhou-j   <none>           <none>
test-8f54bcfb5-skvkg   1/1     Running   0          14s   10.16.XX.XX   virtual-kubelet-cn-hangzhou-j   <none>           <none>

View the details of the ECI pod.
```
kubectl get pod <pod-name> -o yaml -n vk
```
In the YAML output, find the corresponding ECI instance ID in the k8s.aliyun.com/eci-instance-id annotation and confirm the compute specification of the ECI pod used for billing in the k8s.aliyun.com/eci-instance-spec annotation.