When you schedule a Pod to run on a virtual node as an Elastic Container Instance (ECI), both ACK clusters and ACK Serverless clusters schedule the Pod to an x86-based virtual node by default. If your application uses the ARM architecture, you must add a nodeSelector to specify an ARM-based virtual node.
Prerequisites
The cluster meets the following requirements:
-
The region where the cluster is deployed supports ARM-based instances.
ECI supports the g8y, c8y, and r8y ECS instance type families on the ARM architecture. For more information about the supported regions, see ECS instance types available for each region.
The cluster runs Kubernetes 1.20 or later.
-
The version of the ACK Virtual Node component in the cluster supports ARM-based nodes.
-
The version of the kube-scheduler component in the cluster supports ARM-based nodes.
Precautions
You are billed for the actual ARM-based ECS instance type provisioned, not for the vCPU and memory resources you specify.
After creating an ECI Pod, run kubectl describe pod <pod name> and check the k8s.aliyun.com/eci-instance-spec annotation to confirm the actual ECS instance type.
Add an ARM-based virtual node
Follow these steps to modify the eci-profile and add an ARM-based virtual node to your cluster.
-
Modify the eci-profile.
kubectl edit configmap eci-profile -n kube-systemIn the
datasection of the eci-profile, setenableLinuxArm64Nodetotrue. Ensure that at least one of the configured zones supports ARM-based instance types.data: enableClusterIp: "true" enableHybridMode: "false" enableLinuxArm64Node: "true" # Enable ARM-based nodes. enableLogController: "false" enablePVCController: "false" enablePrivateZone: "false" enableReuseSSLKey: "false" featureGates: MetricsVpcNet=true,ProtectionFinalizers=false,WaitForFirstConsumer=false resourceGroupId: "" securityGroupId: sg-2zeg1fci0oq1hljo6h0a selectors: "" slsMachineGroup: "" vSwitchIds: vsw-2zewa5mb19wr45g63****,vsw-2zevanrscmoiaxryx****,vsw-2ze94pjtfuj9vaymf**** # The configured zones must include a zone that supports ARM-based instance types. vpcId: vpc-2zeghwzptn5zii0w7**** -
View the ARM-based virtual node that is generated in the cluster.
After you set
enableLinuxArm64Nodetotrue, run thekubectl get nodecommand to view node information. An ARM-based virtual node is automatically added to the cluster.kubectl get node NAME STATUS ROLES AGE VERSION virtual-kubelet-cn-beijing-h Ready agent 169m v1.24.6-aliyun.1 virtual-kubelet-cn-beijing-i Ready agent 169m v1.24.6-aliyun.1 virtual-kubelet-cn-beijing-i-linux-arm64 Ready agent 75s v1.24.6-aliyun.1 virtual-kubelet-cn-beijing-k Ready agent 169m v1.24.6-aliyun.1
Schedule pods to an ARM-based virtual node
Configure the nodeSelector field
ARM-based virtual nodes in ACK clusters and ACK Serverless clusters have the kubernetes.io/arch: arm64 label. You can use this label in a nodeSelector to schedule an ECI Pod to an ARM-based virtual node.
nodeSelector:
kubernetes.io/arch: arm64
Sample YAML file of a pod:
-
No specifications are specified for the pod.
If you do not specify an ECS instance type, the system defaults to ecs.c8y.large, an ARM-based instance type with 2 vCPUs and 4 GiB of memory.
apiVersion: apps/v1 kind: Deployment metadata: name: test labels: app: test spec: replicas: 1 selector: matchLabels: app: nginx template: metadata: name: nginx-test labels: app: nginx alibabacloud.com/eci: "true" # Schedule the Pod to a virtual node. spec: containers: - name: nginx image: arm64v8/centos:7.9.2009 # Use an image based on the ARM architecture. command: ["sleep"] args: ["999999"] nodeSelector: kubernetes.io/arch: arm64 # Specify an ARM-based node. -
vCPU and memory specifications are specified.
If you specify vCPU and memory resources, the system automatically selects a matching ARM-based ECS instance type from the instance types supported by ECI.
NoteYou can specify vCPU and memory by defining the
limitsorrequestsfields for the container (usinglimitsis recommended), or by adding thek8s.aliyun.com/eci-use-specsannotation.apiVersion: apps/v1 kind: Deployment metadata: name: test labels: app: test spec: replicas: 1 selector: matchLabels: app: nginx template: metadata: name: nginx-test labels: app: nginx alibabacloud.com/eci: "true" # Schedule the Pod to a virtual node. spec: containers: - name: nginx image: arm64v8/centos:7.9.2009 # Use an image based on the ARM architecture. command: ["sleep"] args: ["999999"] resources: limits: cpu: "1000m" # Specify 1 vCPU. memory: "4096Mi" # Specify 4 GiB of memory for the nginx container. nodeSelector: kubernetes.io/arch: arm64 # Specify an ARM-based node. -
Specify an ARM-based ECS instance type
If you have special requirements for the specifications, you can use the
k8s.aliyun.com/eci-use-specsannotation to specify specific Arm specifications. For information about the Arm specifications that ECI supports, see Specify Arm specifications to create a pod.apiVersion: apps/v1 kind: Deployment metadata: name: test labels: app: test spec: replicas: 1 selector: matchLabels: app: nginx template: metadata: name: nginx-test labels: app: nginx alibabacloud.com/eci: "true" annotations: k8s.aliyun.com/eci-use-specs: "ecs.c8y.large,ecs.g8y.large" # Specify the supported ARM-based ECS instance types. You can specify up to five types. spec: containers: - name: nginx image: arm64v8/centos:7.9.2009 # Use an image based on the ARM architecture. command: ["sleep"] args: ["999999"] nodeSelector: kubernetes.io/arch: arm64 # Schedule the Pod to an ARM-based node.
Configure tolerations and nodeaffinity to schedule multi-architecture images
If you use a multi-architecture image and need to schedule Pods across both x86 and ARM architectures, you can use tolerations and nodeAffinity to set a scheduling preference for either ARM-based or x86-based virtual nodes.
Before you use nodeAffinity to configure node affinity based on the following YAML file, make sure that the virtual node-based pod scheduling feature is enabled for the cluster. For more information, see Enable the virtual node-based pod scheduling policy for an ACK cluster.
Sample YAML file of a pod:
-
Prefer ARM-based nodes
apiVersion: apps/v1 kind: Deployment metadata: name: test labels: app: test spec: replicas: 1 selector: matchLabels: app: nginx template: metadata: name: nginx-test labels: app: nginx alibabacloud.com/eci: "true" # Schedule the Pod to a virtual node. spec: tolerations: # Tolerate the taint on ARM-based nodes. - key: kubernetes.io/arch operator: Equal value: arm64 effect: NoSchedule affinity: nodeAffinity: preferredDuringSchedulingIgnoredDuringExecution: - weight: 1 preference: matchExpressions: - key: kubernetes.io/arch operator: In values: - arm64 # Prefer ARM-based nodes. containers: - name: nginx image: nginx # Use a multi-architecture image. command: ["sleep"] args: ["999999"] -
Preferentially schedule the pod to an x86-based virtual node
apiVersion: apps/v1 kind: Deployment metadata: name: test labels: app: test spec: replicas: 1 selector: matchLabels: app: nginx template: metadata: name: nginx-test labels: app: nginx alibabacloud.com/eci: "true" # Schedule the Pod to a virtual node. spec: tolerations: # Tolerate the taint on ARM-based nodes. - key: kubernetes.io/arch operator: Equal value: arm64 effect: NoSchedule affinity: nodeAffinity: preferredDuringSchedulingIgnoredDuringExecution: - weight: 1 preference: matchExpressions: - key: kubernetes.io/arch operator: In values: - amd64 # Prefer x86-based nodes. containers: - name: nginx image: nginx # Use a multi-architecture image. command: ["sleep"] args: ["999999"]
References
ACK clusters: Schedule workloads to ARM-based nodes
ACK Serverless clusters: Schedule workloads to ARM-based virtual nodes