All Products
Search
Document Center

Container Service for Kubernetes:Manage the GPU sharing component

Last Updated:Nov 02, 2026

Container Service for Kubernetes (ACK) provides GPU sharing and scheduling capabilities for model inference scenarios that share a single GPU. It also enforces GPU memory isolation in the kernel mode of the NVIDIA driver. Upgrade the component when a node's GPU driver or operating system is incompatible with the installed cGPU version to keep GPU scheduling and isolation on your nodes.

Prerequisites

Limitations

  • The cGPU memory isolation feature is supported only on Alibaba Cloud Elastic Compute Service (ECS) nodes. Do not add the ack.node.gpu.schedule=cgpu or ack.node.gpu.schedule=core_mem label to nodes that are not Alibaba Cloud ECS nodes.

  • Do not set the CPU policy to static for nodes that use GPU sharing.

  • If you need to specify a custom path for the KubeConfig file, run the export KUBECONFIG=<kubeconfig> command. The kubectl inspect cgpu command does not support the --kubeconfig parameter.

  • Because cGPU isolation does not support Unified Virtual Memory (UVM), you cannot call cudaMallocManaged() to allocate GPU memory. Instead, call cudaMalloc(). For more information, see the NVIDIA documentation.

  • The GPU sharing DaemonSet pods do not have the highest priority on a node, so other higher-priority pods may preempt their resources and cause eviction. To prevent this, modify the DaemonSet you are using, such as gpushare-device-plugin-ds for shared GPU memory, and add priorityClassName: system-node-critical to ensure high priority.

  • For performance reasons, a single GPU card supports a maximum of 20 pods when using cGPU. If this limit is exceeded, subsequent pods scheduled to that card will fail to run and return the error message Error occurs when creating cGPU instance: unknown.

    To prevent the scheduler from continuing to schedule cGPU pods to nodes where a single card has already reached the 20-pod limit, run the following command to add a label to the node:

    kubectl label nodes <NODE> ack.node.gpu.max-pod=20

    The value 20 is fixed and cannot be changed to other values.

  • The GPU sharing component can be installed in any region. However, the GPU memory isolation feature is available only in the following regions. Make sure your cluster is in one of these regions.

    Regions

    Region

    Region ID

    China (Beijing)

    cn-beijing

    China (Shanghai)

    cn-shanghai

    China (Hangzhou)

    cn-hangzhou

    China (Zhangjiakou)

    cn-zhangjiakou

    China (Wulanchabu)

    cn-wulanchabu

    China (Shenzhen)

    cn-shenzhen

    China (Chengdu)

    cn-chengdu

    China (Heyuan)

    cn-heyuan

    China (Hong Kong)

    cn-hongkong

    Japan (Tokyo)

    ap-northeast-1

    Indonesia (Jakarta)

    ap-southeast-5

    Singapore

    ap-southeast-1

    US (Virginia)

    us-east-1

    US (Silicon Valley)

    us-west-1

    Germany (Frankfurt)

    eu-central-1

  • Version compatibility.

    Item

    Supported versions

    Kubernetes version

    • ack-ai-installer versions earlier than 1.12.0 support clusters running Kubernetes 1.18.8 or later.

    • ack-ai-installer 1.12.0 or later supports only clusters running Kubernetes 1.20 or later.

    NVIDIA driver version

    418.87.01 or later

    Container runtime version

    • Docker: 19.03.5 or later

    • containerd: 1.4.3 or later

    Operating system

    Alibaba Cloud Linux 3.x (container-optimized edition requires ack-ai-installer v1.12.6 or later), Alibaba Cloud Linux 2.x, CentOS 7.6, CentOS 7.7, CentOS 7.9, Ubuntu 22.04

    Supported GPUs

    P-series, T-series, V-series, A-series, and H-series

Install the GPU sharing component

Step 1: Install the GPU sharing component

Cloud-native AI Suite not deployed

  1. Log on to the ACK console. In the left navigation pane, click Clusters.

  2. On the Clusters page, click the name of your cluster. In the left navigation pane, click Applications > Cloud-native AI Suite.

  3. On the Cloud-native AI Suite page, click Deploy.

  4. On the Deploy Cloud-native AI Suite page, select Scheduling Policy Extension (Batch Task Scheduling, GPU Sharing, Topology-aware GPU Scheduling).

  5. (Optional) To customize the cGPU policy, click Advanced to the right of Scheduling Policy Extension (Batch Task Scheduling, GPU Sharing, Topology-aware GPU Scheduling). In the Parameters dialog box, modify the policy field for cGPU and click OK.

    If you have no special requirements for cGPU compute power sharing, the recommended setting is the default policy: 5 (native scheduling). For more information about cGPU policies, see Install and use the cGPU service.

  6. At the bottom of the Cloud-native AI Suite page, click Deploy Cloud-native AI Suite.

    After installation, the ack-ai-installer component appears in the component list on the Components page.

Cloud-native AI Suite deployed

  1. Log on to the ACK console. In the left navigation pane, click Clusters.

  2. On the Clusters page, click the name of your cluster. In the left navigation pane, click Applications > Cloud-native AI Suite.

  3. Find the ack-ai-installer component and click Deploy in the Actions column.

  4. (Optional) In the Parameters dialog box that appears, modify the policy field for cGPU.

    If you have no special requirements for cGPU compute power sharing, the recommended setting is the default policy: 5 (native scheduling). For more information about cGPU policies, see Install and use the cGPU service.

  5. After you finish modifying the parameters, click OK.

    After the component is installed, the Status of ack-ai-installer changes to Deployed.

Step 2: Enable GPU sharing, scheduling, and memory isolation

  1. On the Clusters page, click the name of your cluster. In the left navigation pane, click Nodes > Node Pools.

  2. On the Node Pools page, click Create Node Pool. Configure the parameters as described in Create and manage a node pool.

  3. On the Create Node Pool page, configure the parameters for the node pool and then click Confirm. The following table describes the key parameters.

    Parameter

    Description

    Expected number of nodes

    Enter the initial number of nodes for the node pool. Use 0 if no nodes are needed initially.

    Node Labels

    Set the label value based on your business requirements. For more information about node labels, see Enable scheduling features.

    This example uses the label value cgpu. This value enables GPU sharing on the node, where each pod only needs to request GPU memory resources. Multiple pods on a single GPU card share compute power and have isolated memory.

    Click Node Labels next to Node Labels. Set the Key to ack.node.gpu.schedule and the Value to cgpu.

    Important
    • For notes on using the cGPU isolation feature, see cGPU FAQ.

    • After adding the GPU sharing label, do not change the GPU scheduling label with the kubectl label nodes command or the label management feature on the Nodes page. Changing the label this way may cause potential issues. For more information, see Enable scheduling features. The recommended method is described in Enable scheduling features.

Step 3: Add GPU nodes

If you already created GPU nodes when you added the node pool, you can skip this step.

After creating the node pool, add GPU nodes to it. When doing so, make sure the instance type architecture is set to GPU cloud server. For more information, see Add existing nodes to a node pool or Create and manage a node pool.

Step 4: Install and use the GPU query tool

  1. Download the kubectl-inspect-cgpu executable to a directory in your PATH, such as /usr/local/bin/.

    • If you are using Linux:

      wget http://aliacs-k8s-cn-beijing.oss-cn-beijing.aliyuncs.com/gpushare/kubectl-inspect-cgpu-linux -O /usr/local/bin/kubectl-inspect-cgpu
    • If you are using macOS:

      wget http://aliacs-k8s-cn-beijing.oss-cn-beijing.aliyuncs.com/gpushare/kubectl-inspect-cgpu-darwin -O /usr/local/bin/kubectl-inspect-cgpu
  2. Make the file executable:

    chmod +x /usr/local/bin/kubectl-inspect-cgpu
  3. Check the GPU usage in the cluster:

    kubectl inspect cgpu

    Expected output:

    NAME                       IPADDRESS      GPU0(Allocated/Total)  GPU Memory(GiB)
    cn-shanghai.192.168.6.104  192.168.6.104  0/15                   0/15
    ----------------------------------------------------------------------
    Allocated/Total GPU Memory In Cluster:
    0/15 (0%)

Upgrade the GPU sharing component

Step 1: Determine the upgrade method

The upgrade method depends on how the ack-ai-installer component was installed. There are two installation methods:

  • Installed through the Cloud-native AI Suite (Recommended): The component was installed from the Components page.

  • Installed from the App Catalog (no longer available): The ack-ai-installer component was installed from the App Catalog page in the Marketplace. This method is closed, but existing components installed this way can still be upgraded through it.

    Important

    If you uninstall a component that was installed using this method, you must activate the Cloud-native AI Suite and complete the installation to reinstall it.

Determine the installation method

  1. Log on to the ACK console. In the left navigation pane, click Clusters.

  2. On the Clusters page, click the name of your cluster. In the left navigation pane, click Applications > Cloud-native AI Suite.

  3. On the Components page, check if the ack-ai-installer component is deployed.

    If the component is deployed, it was installed through the Components. Otherwise, it was installed from the App Catalog.

Step 2: Upgrade the component

Upgrade through Cloud-native AI Suite

  1. Log on to the ACK console. In the left navigation pane, click Clusters.

  2. On the Clusters page, click the name of your cluster. In the left navigation pane, click Applications > Cloud-native AI Suite.

  3. In the Components section, locate the ack-ai-installer component and click Upgrade in the Actions column.

Upgrade through App Catalog

  1. Log on to the ACK console. In the left navigation pane, click Clusters.

  2. On the Clusters page, click the name of your cluster. In the left navigation pane, click Applications > Helm.

  3. On the Helm page, locate the ack-ai-installer release. In the Actions column, click Update. Follow the prompts to select the latest Chart version and update the component.

    Important

    If you need to customize the Chart configuration, confirm the component update after making your changes.

    After the update, check the Helm page to confirm that the Chart version of the ack-ai-installer release is the latest version.

Step 3: Upgrade existing nodes

Upgrading the ack-ai-installer component does not automatically upgrade the cGPU version on existing nodes. Use the following information to determine whether the cGPU isolation feature is enabled on your nodes.

  • If your cluster contains GPU nodes with the cGPU isolation feature enabled, you must also upgrade the cGPU version on those nodes. For more information, see Upgrade the cGPU version of a node.

  • If your cluster does not have any nodes with the cGPU isolation feature enabled, skip this step.

    Note
    • If a node has the label ack.node.gpu.schedule=cgpu or ack.node.gpu.schedule=core_mem, the cGPU isolation feature is enabled.

    • Upgrading the cGPU version on existing nodes requires stopping all business pods on those nodes. Perform this operation during off-peak hours to minimize business disruption.