Container Service for Kubernetes:Vulnerability CVE-2025-23266

Prerequisites

• Confirm the Alibaba Cloud account or RAM user has the CloudOps Orchestration Service (OOS) permissions. For details, see AliyunOOSFullAccess.

• Verify whether the cGPU add-on is installed in the cluster:

  • If not installed, proceed to Step 1: Drain the node.

  • If installed, ensure the version is 1.1.0 or later.

  How to check if cGPU is installed and how to upgrade it

  1. Log on to the ACK console. In the navigation pane on the left, click Clusters.

  2. On the Clusters page, find the cluster you want and click its name. In the left-side navigation pane, choose Applications > Helm.

  3. On the Helm page, check the add-on version.

     • No ack-cgpu found: The cGPU add-on is not installed.

     • Both ack-ai-installer and ack-cgpu exist: Submit a ticket for assistance.

     • ack-ai-installer exists: If its version is earlier than 1.7.5, upgrade it.

     • ack-cgpu exists: If its version is earlier than 1.5.1, click Update to the right of the add-on and follow the on-screen instructions to upgrade it.

  4. After add-on upgrades, update existing cGPU nodes in the cluster.

Step 1: Drain the node

Step 2: Run the fix script on the node

1. Log on to the ACK console. In the navigation pane on the left, click Clusters.

2. On the Clusters page, click the name of the one you want to change. In the navigation pane on the left, choose Nodes > Nodes.

3. On the Nodes page, select the target nodes, and click Batch Operations at the bottom of the page. In the Batch Operations dialog box, select Run Shell Scripts, then click OK.

4. On the Basic Information step, configure the parameters according to the following table, then click Next Step: Parameter Settings.

   Parameter	Value
   Template Category	Public Template
   Template	ACS-ECS-BulkyRunCommand
   Execution Mode	Suspend upon Failure

5. On the Parameter Settings step, click Run Shell Script. Enter the following CVE fix script in the CommandContent field and click Next Step: OK.

   #!/bin/bash
   set -e
   # Set the region where the node resides.
   export REGION=$(curl 100.100.100.200/latest/meta-data/region-id 2>/dev/null)
   
   if [[ $REGION == "" ]];then
       echo "Error: failed to get region"
       exit 1
   fi
   
   NV_TOOLKIT_VERSION=1.17.8
   
   PACKAGE=upgrade_nvidia-container-toolkit-${NV_TOOLKIT_VERSION}.tar.gz
   
   cd /tmp
   
   curl -o ${PACKAGE}  http://aliacs-k8s-${REGION}.oss-${REGION}-internal.aliyuncs.com/public/pkg/nvidia-container-runtime/${PACKAGE} 
   
   tar -xf ${PACKAGE} 
   
   cd pkg/nvidia-container-runtime/upgrade/common
   
   bash upgrade-nvidia-container-toolkit.sh

6. On the OK step, verify the information and click Create.

7. After the task runs, in the navigation pane on the left of the CloudOps Orchestration Service console, choose Automated Tasks > Task Execution Management. Find and click the ID of the executed task, and view the Outputs in the Execution Steps and Results section.

   • If the script returns the following output, the node is not vulnerable to CVE-2025-23266. No changes were applied and no further action is required.

     2025-03-22/xxxx  INFO  No need to upgrade current nvidia-container-toolkit(1.17.8)

   • If the script returns the following output, the node was affected by the NVIDIA Container Toolkit vulnerability and has been fixed.

     2025-03-22/xxxxx  INFO  succeed to upgrade nvidia container toolkit

Step 3: Set the node to schedulable

kubectl uncordon <NODE_NAME>

(Optional) Step 4: Verify the GPU-accelerated nodes

After completing the preceding steps, deploy a GPU application using a sample YAML template from one of the following topics to verify that the node works as expected:

• Exclusive GPU: Use the default GPU scheduling mode

• Shared GPU: Use GPU sharing to share GPUs