How to configure a cGPU scheduling policy for GPU sharing - Container Service for Kubernetes

ACK Pro clusters support GPU sharing. Configure the cGPU policy parameter to choose among six compute-allocation strategies. This topic explains how to set the shared GPU scheduling policy.

Prerequisites

An ACK Pro cluster is created, running Kubernetes 1.18.8 or later. Manually upgrade a cluster if needed.
cGPU version 1.0.6 or later is installed. Upgrade the cGPU version of a node if needed.

Important notes

If the cGPU isolation module is already installed on a node, you must restart the node after installing the cGPU component to apply the policy. See Restart an instance.
Note
Log on to the node and run cat /proc/cgpu_km/version. If a version number is returned, the cGPU isolation module is installed.
If the cGPU isolation module is not installed or has been uninstalled, the policy takes effect immediately after you install the cGPU component.
All GPU-sharing nodes in a cluster must use the same policy.

Check whether cGPU is installed

The configuration procedure depends on whether cGPU is already installed.

Log on to the ACK console. In the left navigation pane, click Clusters.
On the Clusters page, click the name of your cluster. In the left navigation pane, click Applications > Helm.
On the Helm page, check whether ack-ai-installer is listed.
If ack-ai-installer is listed, cGPU is installed.

Configure the scheduling policy

cGPU not installed

Log on to the ACK console. In the left navigation pane, click Clusters.
On the Clusters page, click the name of your cluster. In the left navigation pane, click Applications > Cloud-native AI Suite.
On the Cloud-native AI Suite page, click Deploy.
In the Scheduling section, select Scheduling Policy Extension (Batch Task Scheduling, GPU Sharing, Topology-aware GPU Scheduling), and then click Advanced.

On the Parameters page, modify the policy field, and then click OK.

cgpu:
  enabled: true
  image: acs/cgpu-installer
  tag: v1.5.3-8d3fc1b-aliyun
  financeCloudImageTag: v1.0.7-fin
  imagePullPolicy: IfNotPresent
  policy: 1
  nodeSelector: {}
  checkRegions: true

Valid values are listed below. See cGPU usage examples.

Value	Description
0	Fair-share scheduling. Each container gets a fixed time slice proportional to `1/max_inst`.
1	Preemptive scheduling. Each container uses as many time slices as possible. Time slice per container: `1/Number of current containers`.
2	Weight-based preemptive scheduling. Enabled automatically when ALIYUN_COM_GPU_SCHD_WEIGHT exceeds 1.
3	Fixed percentage scheduling. Assigns a fixed percentage of computing power.
4	Soft scheduling. Provides weaker isolation than preemptive scheduling.
5	Built-in scheduling. Uses the GPU driver's native scheduling method.

Click Deploy Cloud-native AI Suite.

cGPU already installed

Edit the DaemonSet that installs the cGPU isolation module.
```
kubectl edit daemonset cgpu-installer -nkube-system
```

Modify and save the DaemonSet.

Check the image version of the DaemonSet.
The image version must be v1.0.6 or later. Example:
```
 image: registry-vpc.cn-hongkong.aliyuncs.com/acs/cgpu-installer:<image_version>
```

Modify the value.

In containers.env, set the value of POLICY to the desired policy number.

# Other fields are omitted for brevity.
spec:
  containers:
  - env:
    - name: POLICY
      value: "1"
# Other fields are omitted for brevity.

Valid values:

Value	Description
0	Fair-share scheduling. Each container gets a fixed time slice proportional to `1/max_inst`.
1	Preemptive scheduling. Each container uses as many time slices as possible. Time slice per container: `1/Number of current containers`.
2	Weight-based preemptive scheduling. Enabled automatically when ALIYUN_COM_GPU_SCHD_WEIGHT exceeds 1.
3	Fixed percentage scheduling. Assigns a fixed percentage of computing power.
4	Soft scheduling. Provides weaker isolation than preemptive scheduling.
5	Built-in scheduling. Uses the GPU driver's native scheduling method.

Restart the GPU-sharing nodes. See Restart an instance.