Run ACS GPU and CPU Workloads in ACK One Registered Clusters - Container Compute Service

How it works

ACS uses a layered architecture that separates Kubernetes control from compute. The Kubernetes control layer manages workloads (Deployments, Services, StatefulSets, CronJobs), while the compute layer schedules and allocates resources to pods.

In ACK One registered clusters, the ack-virtual-node component provisions virtual nodes that ACS backs. Pods scheduled to these virtual nodes run as ACS pods—isolated, serverless containers that scale on demand without requiring you to plan node capacity. ACS pods communicate with pods on physical nodes in the same cluster.

Prerequisites

Before you begin, make sure you have:

An ACK One registered cluster connected to a data center or another cloud provider's Kubernetes cluster. Kubernetes 1.24 or later is recommended. See Create a registered cluster.
The ack-virtual-node component installed at version 2.13.0 or later. See Grant RAM permissions to ack-virtual-node and Install ack-virtual-node.

Choose a scheduling method

ACS supports two methods for scheduling pods to virtual nodes. Choose based on your use case:

Method	How it works	Best for
Label scheduling (recommended)	Add `alibabacloud.com/acs: "true"` to pod labels	Migrating multiple workloads; no nodeSelector or toleration required
NodeSelector	Set `nodeSelector: type: virtual-kubelet` and add a toleration	Explicit control over which pods land on virtual nodes

If you do not specify a compute class, elastic container instances are prioritized for pod scheduling by default.

Use CPU computing power

Label scheduling

All pods with alibabacloud.com/acs: "true" in their labels are automatically scheduled to virtual nodes—no nodeSelector or toleration needed.

Create a file named nginx.yaml with the following content:

apiVersion: apps/v1
kind: Deployment
metadata:
  name: nginx
  labels:
    app: nginx
spec:
  replicas: 2
  selector:
    matchLabels:
      app: nginx
  template:
    metadata:
      labels:
        app: nginx
        alibabacloud.com/acs: "true"                   # Schedule to ACS virtual nodes.
        alibabacloud.com/compute-class: general-purpose # Compute class. Default: general-purpose.
        alibabacloud.com/compute-qos: default           # QoS class. Default: default.
    spec:
      containers:
      - name: nginx
        image: mirrors-ssl.aliyuncs.com/nginx:stable-alpine
        ports:
          - containerPort: 80
            protocol: TCP
        resources:
          limits:
            cpu: 2
          requests:
            cpu: 2

Deploy the application:
```
kubectl apply -f nginx.yaml
```

Verify the pods are running on virtual nodes:

kubectl get pods -o wide

Expected output:

NAME                     READY   STATUS    RESTARTS   AGE     IP               NODE                            NOMINATED NODE   READINESS GATES
nginx-54bcbc9b66-****   1/1     Running   0          3m29s   192.168.XX.XXX   virtual-kubelet-cn-shanghai-l   <none>           <none>
nginx-54bcbc9b66-****   1/1     Running   0          3m29s   192.168.XX.XXX   virtual-kubelet-cn-shanghai-l   <none>           <none>

Confirm that the pods are ACS pods by checking the annotations:

kubectl describe pod nginx-54bcbc9b66-****

The output includes the alibabacloud.com/instance-id: acs-uf6008giwgjxlvn***** annotation, which confirms the pod is running as an ACS pod.

Annotations:  ProviderCreate: done
              alibabacloud.com/instance-id: acs-uf6008giwgjxlvn*****
              alibabacloud.com/pod-ephemeral-storage: 30Gi
              alibabacloud.com/pod-use-spec: 2-2Gi
              kubernetes.io/pod-stream-port: 10250
              network.alibabacloud.com/enable-dns-cache: false
              topology.kubernetes.io/region: cn-shanghai

NodeSelector

With this method, you explicitly target virtual nodes using a nodeSelector and toleration. First, check the labels on your virtual node to confirm the type: virtual-kubelet label is present.

Query the labels on a virtual node. Replace virtual-kubelet-cn-shanghai-l with your actual virtual node name:

kubectl get node virtual-kubelet-cn-shanghai-l -oyaml

Expected output:

apiVersion: v1
kind: Node
metadata:
  labels:
   beta.kubernetes.io/arch: amd64
    beta.kubernetes.io/os: linux
    kubernetes.io/arch: amd64
    kubernetes.io/hostname: virtual-kubelet-cn-shanghai-l
    kubernetes.io/os: linux
    kubernetes.io/role: agent
    service.alibabacloud.com/exclude-node: "true"
    topology.diskplugin.csi.alibabacloud.com/zone: cn-shanghai-l
    topology.kubernetes.io/region: cn-shanghai
    topology.kubernetes.io/zone: cn-shanghai-l
    type: virtual-kubelet # Use this label as the nodeSelector value to target virtual nodes.
  name: virtual-kubelet-cn-shanghai-l
spec:
  taints:
  - effect: NoSchedule
    key: virtual-kubelet.io/provider
    value: alibabacloud

Create a file named nginx.yaml with the following content:

apiVersion: apps/v1
kind: Deployment
metadata:
  name: nginx
  labels:
    app: nginx
spec:
  replicas: 2
  selector:
    matchLabels:
      app: nginx
  template:
    metadata:
      name: nginx
      labels:
        app: nginx
        alibabacloud.com/compute-class: general-purpose # Compute class. Default: general-purpose.
        alibabacloud.com/compute-qos: default           # QoS class. Default: default.
    spec:
      nodeSelector:
        type: virtual-kubelet          # Targets virtual nodes.
      tolerations:
      - key: "virtual-kubelet.io/provider" # Tolerates the virtual node taint.
        operator: "Exists"
        effect: "NoSchedule"
      containers:
      - name: nginx
        image: mirrors-ssl.aliyuncs.com/nginx:stable-alpine
        ports:
          - containerPort: 80
            protocol: TCP
        resources:
          limits:
            cpu: 2
          requests:
            cpu: 2

Deploy the application:
```
kubectl apply -f nginx.yaml
```

Verify the pods are running on virtual nodes:

kubectl get pods -o wide

Expected output:

NAME                     READY   STATUS    RESTARTS   AGE     IP               NODE                            NOMINATED NODE   READINESS GATES
nginx-54bcbc9b66-****   1/1     Running   0          3m29s   192.168.XX.XXX   virtual-kubelet-cn-shanghai-l   <none>           <none>
nginx-54bcbc9b66-****   1/1     Running   0          3m29s   192.168.XX.XXX   virtual-kubelet-cn-shanghai-l   <none>           <none>

For more scheduling options, including affinity, anti-affinity, and ResourcePolicies, see Node affinity scheduling and ACS pod overview.

Use GPU computing power

GPU computing power in ACK One registered clusters is in invitational preview. To request access, submit a ticket.

The procedure is similar to CPU, with two differences: set alibabacloud.com/compute-class: gpu and add the alibabacloud.com/gpu-model-series label to specify the GPU model (for example, T4). You also need to ensure that the scheduling components meet the version requirements.

For a full list of supported GPU models, see GPU models. For compute class and quality of service (QoS) class mappings, see Mappings between compute classes and computing power QoS classes.

Label scheduling

Create a file named gpu-workload.yaml with the following content:

apiVersion: apps/v1
kind: Deployment
metadata:
  name: dep-node-selector-demo
  labels:
    app: node-selector-demo
spec:
  replicas: 1
  selector:
    matchLabels:
      app: node-selector-demo
  template:
    metadata:
      labels:
        app: node-selector-demo
        alibabacloud.com/acs: "true"                        # Schedule to ACS virtual nodes.
        alibabacloud.com/compute-class: gpu                 # Set to gpu for GPU compute power.
        alibabacloud.com/compute-qos: default
        alibabacloud.com/gpu-model-series: example-model   # Specify the GPU model, such as T4.
    spec:
      containers:
      - name: node-selector-demo
        image: registry-cn-hangzhou.ack.aliyuncs.com/acs/stress:v1.0.4
        command:
        - "sleep"
        - "1000h"
        resources:
          limits:
            cpu: 1
            memory: 1Gi
            nvidia.com/gpu: "1"
          requests:
            cpu: 1
            memory: 1Gi
            nvidia.com/gpu: "1"

Deploy the workload:
```
kubectl apply -f gpu-workload.yaml
```

Verify the GPU workload is running:

kubectl get pod node-selector-demo-9cdf7bbf9-s**** -oyaml

Expected output:

    phase: Running

    resources:
      limits:
        #other resources
        nvidia.com/gpu: "1"
      requests:
        #other resources
        nvidia.com/gpu: "1"

NodeSelector

Create a file named gpu-workload.yaml with the following content:

apiVersion: apps/v1
kind: Deployment
metadata:
  name: dep-node-selector-demo
  labels:
    app: node-selector-demo
spec:
  replicas: 1
  selector:
    matchLabels:
      app: node-selector-demo
  template:
    metadata:
      labels:
        app: node-selector-demo
        alibabacloud.com/compute-class: gpu                # Set to gpu for GPU compute power.
        alibabacloud.com/compute-qos: default
        alibabacloud.com/gpu-model-series: example-model  # Specify the GPU model, such as T4.
    spec:
      nodeSelector:
        type: virtual-kubelet                              # Targets virtual nodes.
      tolerations:
      - key: "virtual-kubelet.io/provider"                # Tolerates the virtual node taint.
        operator: "Exists"
        effect: "NoSchedule"
      containers:
      - name: node-selector-demo
        image: registry-cn-hangzhou.ack.aliyuncs.com/acs/stress:v1.0.4
        command:
        - "sleep"
        - "1000h"
        resources:
          limits:
            cpu: 1
            memory: 1Gi
            nvidia.com/gpu: "1"
          requests:
            cpu: 1
            memory: 1Gi
            nvidia.com/gpu: "1"

Deploy the workload:
```
kubectl apply -f gpu-workload.yaml
```

Verify the GPU workload is running:

kubectl get pod node-selector-demo-9cdf7bbf9-s**** -oyaml

Expected output:

    phase: Running

    resources:
      limits:
        #other resources
        nvidia.com/gpu: "1"
      requests:
        #other resources
        nvidia.com/gpu: "1"

Container Compute Service:Schedule pods to ACS using virtual nodes

How it works

Prerequisites

Choose a scheduling method

Use CPU computing power

Label scheduling

NodeSelector

Use GPU computing power

Label scheduling

NodeSelector

What's next