Container Service for Kubernetes:Create ARM-based virtual nodes and schedule workloads to ARM-based nodes

Prerequisites

• A cluster that runs Kubernetes 1.20 or later is created. The operating system of the cluster is Alibaba Cloud Linux 3. For more information, see Create a cluster and Upgrade clusters.

  Note

  • On the Add-ons page, only the Core Components, Logs and Monitoring, Storage, and Networking components can be used in ARM-based node pools.

  • Components displayed on the Marketplace page of the ACK console cannot be deployed in ARM-based node pools.

• The kube-scheduler component is installed. For more information, see Manage components.

Usage notes

If both ARM-based nodes and x86-based nodes exist in your cluster, we recommend that you add the kubernetes.io/arch=arm64:NoSchedule taint to ARM-based nodes. This avoids accidentally scheduling applications or components that do not support the ARM architecture to ARM-based nodes. When you configure nodeSelector or nodeAffinity to schedule applications to ARM-based nodes in an ACK cluster that runs a Kubernetes version earlier than 1.24, you need to add a toleration to tolerate the kubernetes.io/arch=arm64:NoSchedule taint. However, if the cluster runs Kubernetes 1.24 or later, the scheduler automatically tolerates the kubernetes.io/arch=arm64:NoSchedule taint. In this case, you do not need to add a toleration to tolerate the taint.

Billing

For more information about the ECS instance types that use the ARM architecture and the pricing of these types, see the following topics:

• Instance family

• Pricing of ECS instance types

• Instance Types Available for Each Region

Create an ARM-based cluster or node pool

When you create an ACK cluster, you can add ARM-based nodes to obtain a cluster that contains only ARM-based nodes. You can also create an ARM-based node pool in an existing cluster to obtain a node pool that contains only ARM-based nodes.

Schedule ARM workloads to ARM-based nodes

If your cluster contains ARM-based nodes and other nodes and all workloads use the ARM architecture, you must schedule the workloads only to ARM-based nodes. If pods are scheduled to other nodes, the pods cannot launch. By default, all ARM-based nodes have the kubernetes.io/arch=arm64 label. You can use the nodeSelector or nodeAffinity setting to schedule workloads to the nodes.

nodeSelector:
  kubernetes.io/arch: arm64 # Specify the label that is used to select an ARM-based node.

apiVersion: apps/v1
kind: Deployment
metadata:
  name: only-arm
spec:
  selector:
    matchLabels:
      app: nginx
  replicas: 1
  template:
    metadata:
      labels:
        app: nginx
    spec:
      nodeSelector:
        kubernetes.io/arch: arm64 # Specify the label that is used to select an ARM-based node. 
      containers:
      - name: nginx
        image: nginx

affinity:
  nodeAffinity:
    requiredDuringSchedulingIgnoredDuringExecution:
      nodeSelectorTerms:
      - matchExpressions:
        - key: kubernetes.io/arch
          operator: In
          values:
          - arm64

apiVersion: apps/v1
kind: Deployment
metadata:
  name: only-arm
spec:
  selector:
    matchLabels:
      app: nginx
  replicas: 1
  template:
    metadata:
      labels:
        app: nginx
    spec:
      affinity:
        nodeAffinity:
          requiredDuringSchedulingIgnoredDuringExecution:
            nodeSelectorTerms:
            - matchExpressions:
              - key: kubernetes.io/arch
                operator: In
                values:
                - arm64
      containers:
      - name: nginx
        image: nginx

Schedule multi-arch workloads to ARM-based nodes

By default, ACK schedules all workloads to x86-based nodes. If x86 nodes are insufficient, pods become pending. If your application image is a multi-arch image, such as an image that supports the x86 and ARM architectures, you must configure cross-architecture node scheduling.

For example, you can configure node affinity to preferably schedule workloads to ARM-based or x86-based nodes. Then, schedule workloads to other types of nodes when ARM-based or x86-based nodes are insufficient.

      affinity:
        nodeAffinity:
          preferredDuringSchedulingIgnoredDuringExecution:
          - weight: 1
            preference:
              matchExpressions:
              - key: kubernetes.io/arch
                operator: In
                values:
                - arm64

apiVersion: apps/v1
kind: Deployment
metadata:
  name: arm-prefer
spec:
  replicas: 1
  selector:
    matchLabels:
      app: my-app
  template:
    metadata:
      labels:
        app: my-app
    spec:
# Preferentially schedule the workload to an ARM-based node. 
      affinity:
        nodeAffinity:
          preferredDuringSchedulingIgnoredDuringExecution:
          - weight: 1
            preference:
              matchExpressions:
              - key: kubernetes.io/arch
                operator: In
                values:
                - arm64
      containers:
      - name: my-container
        image: nginx

apiVersion: apps/v1
kind: Deployment
metadata:
  name: amd-prefer
spec:
  replicas: 1
  selector:
    matchLabels:
      app: my-app
  template:
    metadata:
      labels:
        app: my-app
    spec:
      tolerations:
# Preferably schedule the workload to an x86-based node. 
      affinity:
        nodeAffinity:
          preferredDuringSchedulingIgnoredDuringExecution:
          - weight: 1
            preference:
              matchExpressions:
              - key: kubernetes.io/arch
                operator: In
                values:
                - amd64
      containers:
      - name: my-container
        image: nginx

FAQ

Can I use ARM-based preemptible instances?

Yes, you can use ARM-based preemptible instances. For more information, see Use preemptible instances.

What are the limits on using ARM-based nodes in ACK clusters?

Reference

• You can create ARM-based virtual nodes and preferably schedule workloads to ARM-based virtual nodes. For more information, see Schedule workloads to ARM-based virtual nodes.

• You can use Container Registry Enterprise Edition to build multi-arch container images. For more information, see Build a multi-arch container image.

• If you want to run big data jobs without performing O&M on cluster resources, see Run Spark jobs on ARM-based virtual nodes.