By default, the ACK cluster and schedule all workloads to x86-based virtual nodes. If your cluster includes ARM-based virtual nodes alongside other types, such as x86-based nodes, you can configure Kubernetes native scheduling to ensure that workloads compatible only with the ARM architecture are scheduled to ARM-based virtual nodes, or that multi-arch images are preferably scheduled to ARM-based virtual nodes.
Prerequisites
-
Cluster:
You have created an ACK cluster with a version of 1.20 or later. For more information, see Create an ACK managed cluster and Manually upgrade a cluster.
NoteARM instances are available in limited regions and zones. Ensure your cluster is in an open region. For more information about the regions and zones where ARM instances are available, see Overview of regions available for ECS instance types.
-
Components: The ack-virtual-node component is installed, and the component version is 2.9.0 or later. For more information, see ACK Virtual Node.
Considerations
If your cluster version is earlier than 1.24, when using nodeSelector
or nodeAffinity
to schedule applications to ARM nodes, you must also declare the taint toleration kubernetes.io/arch=arm64:NoSchedule
in tolerations
. If your cluster version is 1.24 or later, the scheduler can automatically detect the taint kubernetes.io/arch=arm64:NoSchedule
on ARM nodes, and you do not need to declare tolerations
additionally.
Billing
Pods deployed on elastic container instances that adopt the ARM architecture are billed based on the ECS instance type used to create the elastic container instances. The pods are not billed based on the usage of vCPUs and memory.
After an Elastic Container Instance-based pod is created, you can run the kubectl describe pod
command to view the YAML content of the pod. The k8s.aliyun.com/eci-instance-spec
parameter indicates the ECS instance type used by the pod. The pod is billed based on the ECS instance type.
For more information about the ECS instance types that use the ARM architecture, refer to the following topics:
Step 1: Add ARM-based virtual nodes
Before deploying ARM workloads in the cluster, you need to create ARM-based virtual nodes. You can create virtual nodes that support the ARM architecture by configuring the ECI Profile. You can edit the eci-profile
configuration file in the following two ways. For more information about the ECI Profile, see Configure eci-profile.
Console
Log on to the ACK console. In the left-side navigation pane, click Clusters.
On the Clusters page, find the cluster that you want to manage and click its name. In the left-side navigation pane, choose .
-
Select the Namespace as kube-system. Locate eci-profile and click Edit. Modify the key value
enableLinuxArm64Node
totrue
, then click Confirm.NoteIf all vSwitches of the cluster are in zones that do not support ARM instances, you need to create a vSwitch in a specified zone. After the vSwitch is created, add the ID of the vSwitch to
vSwitchIds
. For more information about creating a vSwitch in a specified zone, see Create and manage a vSwitch.Once the modification is complete, please wait approximately 30 seconds before viewing the newly created virtual node named
virtual-kubelet-<zoneId>-linux-arm64
on the Node page.
kubectl
Prerequisites
Obtain the cluster KubeConfig and connect to the cluster using the kubectl tool.
Procedure
Execute the following command to edit the ConfigMap.
kubectl edit configmap eci-profile -n kube-system
-
Add or modify the parameter
enableLinuxArm64Node
totrue
. -
Set
vSwitchIds
. Ensure that at least one vSwitch in thevSwitchIds
used by the current cluster is in a zone that supports ARM instances.NoteIf all vSwitches of the cluster are in zones```html that do not support ARM instances, you need to create a vSwitch in a specified zone. After the vSwitch is created, add the ID of the vSwitch to
vSwitchIds
. For more information about creating a vSwitch in a specified zone, see Create and manage a vSwitch.Once the modification is finished, please wait approximately 30 seconds before viewing the newly created virtual node named
virtual-kubelet-<zoneId>-linux-arm64
on the Node page.
Step 2: Schedule workloads to ARM virtual nodes
Schedule ARM-based workloads to ARM virtual nodes
If your cluster contains both ARM nodes and other nodes and your application supports only the ARM architecture, you can specify the application to run on ARM nodes to prevent the application pod from being scheduled to non-ARM nodes, which may cause the pod to fail to start. All ARM nodes have the label kubernetes.io/arch=arm64
by default. You can use either nodeSelector or nodeAffinity to specify the application to be deployed on ARM nodes.
nodeSelector
You can add the following constraint to the pod to schedule the pod to ARM-based virtual nodes using nodeSelector. nodeSelector specifies that this workload is scheduled only to nodes with the arm64
label. ARM-based virtual nodes in the ACK cluster have this label.
nodeSelector:
kubernetes.io/arch: arm64 # Specify ARM nodes.
You can use the following example code to deploy a stateless application to ARM virtual nodes.
nodeAffinity
Prerequisites
Enable the cluster virtual node scheduling policy, and ensure that the cluster version and component version meet the requirements.
Example operation
You can add the following constraint to the pod to specify the application to be deployed on ARM nodes using node affinity. This constraint specifies that the pod can only be scheduled to nodes with the label kubernetes.io/arm=arm64
.
When this constraint is added to the pod spec, the scheduler automatically tolerates the taint kubernetes.io/arch=arm64:NoSchedule
on the node.
affinity:
nodeAffinity:
requiredDuringSchedulingIgnoredDuringExecution:
nodeSelectorTerms:
- matchExpressions:
- key: kubernetes.io/arch
operator: In
values:
- arm64
You can use the following example code to deploy a stateless application to ARM virtual nodes.
Schedule multi-arch images to ARM virtual nodes
Prerequisites
Enable the cluster virtual node scheduling policy, and ensure that the cluster version and component version meet the requirements.
Example operation
ACK cluster by default schedules all workloads to x86-based virtual nodes and waits for x86 node resources when x86 node resources are insufficient. If your application image is a multi-arch image, such as an image that supports both x86 and ARM architectures, you need to configure cross-architecture node scheduling.
For example, you can configure node affinity to preferably schedule workloads to ARM-based virtual nodes or x86-based virtual nodes. When the requested virtual nodes are insufficient, the scheduler will attempt to schedule workloads to other types of virtual nodes.
affinity:
nodeAffinity:
preferredDuringSchedulingIgnoredDuringExecution:
- weight: 1
preference:
matchExpressions:
- key: kubernetes.io/arch
operator: In
values:
- arm64
Preferentially schedule to ARM architecture
The following example preferably schedules workloads to ARM-based virtual nodes.
Preferentially schedule to x86 architecture
The following example preferably schedules workloads to x86-based virtual nodes.
FAQ
Why is the pod scheduled to x86-based ECS nodes when nodeAffinity is configured to preferentially schedule the pod to ARM-based nodes?
The cluster scheduler by default preferentially schedules to ECS nodes. When ECS node resources are insufficient, it schedules to virtual nodes. Without modifying the weight of the scheduler scoring plug-in, if there are sufficient resources on x86 ECS nodes in the cluster, even if nodeAffinity is configured to preferentially schedule to ARM-based nodes, the pod may still be scheduled to x86-based ECS nodes. Therefore, the nodeAffinity configuration in this topic can only ensure the priority of virtual nodes of different architectures (ARM/x86) but cannot ensure the priority between virtual nodes and ECS nodes.
Can ARM-based preemptible (Spot) instances be used?
ARM-based preemptible instances are available. For more information, see Use preemptible instances.
How do I configure a network that supports ARM-based virtual nodes after I create a cluster in the corresponding region?
After you create an ACK cluster in the corresponding zone, configure the vSwitchIds
field in the eci-profile to select a virtual switch in a zone that supports ARM instances, ensuring that ARM-based virtual nodes are created.
What are the limitations of using ARM-based nodes in an ACK cluster?
Currently, the ARM architecture does not support components from the application marketplace. The component center supports only the following modules:
-
Core components
-
Logging and monitoring
-
Storage
-
Network
References
You can build multi-architecture container images using Container Registry Enterprise Edition. For more information, see Build multi-arch container images.
For instructions on creating and managing regular ARM ECS nodes, see Schedule to ARM nodes.
If your tasks involve big data and you prefer not to manage the underlying cluster resources, you can run Spark jobs on ARM virtual nodes.