Container Service for Kubernetes:Enable serverless use of cloud CPU/GPU resources for K8s clusters

How it works

Installing the ack-virtual-node add-on on your ACK One registered cluster creates a virtual node backed by ECI. Pods scheduled to this virtual node run as ECI instances: they start quickly, their lifecycle matches that of the pod, and you pay only while they're running.

The ack-co-scheduler add-on introduces a ResourcePolicy custom resource (CR) that enables multilevel scheduling. The scheduler prioritizes your on-premises nodes. When on-premises capacity is exhausted, it automatically overflows to ECI—without any manual intervention.

Use cases

Prerequisites

Before you begin, ensure that you have:

• An on-premises Kubernetes cluster connected to Alibaba Cloud using an ACK One registered cluster. For more information, see Create an ACK One registered cluster.

• Elastic Container Instance (ECI) activated.Elastic Container Instance (ECI) is activated.

• kubectl configured to connect to the registered cluster. For more information, see Obtain the kubeconfig of a cluster and use kubectl to connect to the cluster.

Step 1: Install add-ons

Install the following add-ons on your registered cluster:

Choose one of the following installation methods.

Step 2: Verify the virtual node

After installing ack-virtual-node, run the following command to confirm that the virtual node appears in your cluster.

kubectl get node

Expected output:

NAME                               STATUS   ROLES    AGE    VERSION
iz8vb1xtnuu0ne6b58hvx0z            Ready    master   4d3h   v1.20.9   # An on-premises node. In this example, the master node also runs application workloads.
virtual-kubelet-cn-zhangjiakou-a   Ready    agent    99s    v1.20.9   # The virtual node created by ack-virtual-node, backed by ECI.

The virtual-kubelet node connects your cluster to ECI. Pods scheduled to this node run as ECI instances.

Step 3: Run pods on ECI

Use one of the following methods to schedule pods to ECI.

kubectl label namespace <namespace-name> alibabacloud.com/eci=true

Step 4: Use multilevel resource scheduling

The multilevel resource scheduling feature of ACK One registered clusters prioritizes your on-premises nodes and automatically overflows to ECI when on-premises capacity is exhausted.

The ack-co-scheduler add-on implements this through a ResourcePolicy CR. ResourcePolicy is a namespaced resource.

`ResourcePolicy` parameters

The following example demonstrates multilevel resource scheduling. The setup uses a 4-replica Nginx deployment on a data center cluster with a single 6-CPU node (which can run a maximum of 2 Nginx pods after system resource reservations).

1. Create a ResourcePolicy that prioritizes on-premises nodes and falls back to ECI.

   apiVersion: scheduling.alibabacloud.com/v1alpha1
   kind: ResourcePolicy
   metadata:
     name: cost-balance-policy
   spec:
     selector:
       app: nginx       # Selects pods with this label.
       key1: value1
     strategy: prefer
     units:
     - resource: idc    # Use on-premises nodes first.
       max: 3
     - resource: eci    # Overflow to ECI when on-premises capacity is exhausted.
       nodeSelector:
         key2: value2

   kubectl apply -f <your-policy-file>.yaml

2. Create a Deployment with 2 replicas, each requesting 2 CPUs.

   apiVersion: apps/v1
   kind: Deployment
   metadata:
     name: nginx
     labels:
       app: nginx
   spec:
     replicas: 2
     selector:
       matchLabels:
         app: nginx
     template:
       metadata:
         name: nginx
         annotations:
           addannotion: "true"
         labels:
           app: nginx    # Must match the selector in the ResourcePolicy above.
       spec:
         schedulerName: ack-co-scheduler
         containers:
         - name: nginx
           image: acr-multiple-clusters-registry.cn-hangzhou.cr.aliyuncs.com/ack-multiple-clusters/nginx
           resources:
             requests:
               cpu: 2
             limits:
               cpu: 2

   kubectl apply -f <your-deployment-file>.yaml

3. Scale the Deployment to 4 replicas.

   kubectl scale deployment nginx --replicas 4

   The on-premises node can run at most 2 Nginx pods. The remaining 2 replicas automatically overflow to ECI.

4. Verify pod placement.

   kubectl get pod -o wide

   Expected output:

   NAME                     READY   STATUS    RESTARTS   AGE     IP              NODE
   nginx-79cd98b4b5-97s47   1/1     Running   0          84s     10.100.XX.XX    iz8vb1xtnuu0ne6b58h****
   nginx-79cd98b4b5-gxd8z   1/1     Running   0          84s     10.100.XX.XX    iz8vb1xtnuu0ne6b58h****
   nginx-79cd98b4b5-k55rb   1/1     Running   0          58s     10.100.XX.XX    virtual-kubelet-cn-zhangjiakou-a
   nginx-79cd98b4b5-m9jxm   1/1     Running   0          58s     10.100.XX.XX    virtual-kubelet-cn-zhangjiakou-a

   Two pods are running on the on-premises node. The other two are running on the virtual node, backed by serverless ECI.