This topic describes the impacts of the security vulnerability CVE-2019-5736 in runc and how to remove it. This vulnerability has been fixed for the Kubernetes cluster versions 1.11 and 1.12.

Background

The security vulnerability may occur with Docker, containerd, or any other containers that use runc. This vulnerability gives attackers the ability to use a specific container image or run the exec command to obtain the file handle used by the running host runc. Attackers can overwrite the host runc binary file, then obtain root permission to access the host, and execute commands as with root permission.

For more information, see CVE-2019-5736.

Affected clusters

  • Alibaba Cloud Container Service clusters affected by the vulnerability:
    • All Docker Swarm clusters from versions earlier than Docker v18.09.02.
    • All Kubernetes clusters except for Serverless Kubernetes clusters.
  • Self-built Docker and Kubernetes clusters affected by the vulnerability:
    • All clusters that use Docker versions earlier than v18.09.2.
    • All clusters that use runc v1.0-rc6 or earlier.

Resolution

To fix the security CVE-2019-5736 vulnarability for Kubernetes clusters earlier than V1.11 and V1.12, use one of the following two methods:
  • Upgrade the Docker version of all existing clusters to v18.09.2 or later. Using this method will interrupt your cluster containers and services.
  • Only upgrade runc. This method is applicable to clusters running Docker v17.06. We recommend that you upgrade the runc binary file of each cluster node individually to avoid a service interruption caused by upgrading the Docker engine. To upgrade a runc binary file, complete the following steps:
    1. Run the following command to locate docker-runc:
      Note Usually, docker-runc is located in /usr/bin/docker-runc.
      which docker-runc
    2. Run the following command to back up the original runc:
      mv /usr/bin/docker-runc /usr/bin/docker-runc.orig.$(date -Iseconds)
    3. Run the following command to download the fixed runc:
      curl -o /usr/bin/docker-runc -sSL https://acs-public-mirror.oss-cn-hangzhou.aliyuncs.com/runc/docker-runc-17.06-amd64
    4. Run the following command to set permission availability for docker-runc:
      chmod +x /usr/bin/docker-runc
    5. Run the following command to test whether runc works normally:
      docker-runc -v
      # runc version 1.0.0-rc3
      # commit: fc48a25bde6fb041aae0977111ad8141ff396438
      # spec: 1.0.0-rc5
      docker run -it --rm ubuntu echo OK
    6. To upgrade the runc binary file of a Kubernetes cluster GPU node, you must also install nvidia-runtime by completing the following steps:
      1. Run the following command to locate nvidia-container-runtime:
        Note Usually, nvidia-container-runtime is located in /usr/bin/nvidia-container-runtime.
        which nvidia-container-runtime
      2. Run the following command to back up the original nvidia-container-runtime:
        mv /usr/bin/nvidia-container-runtime /usr/bin/nvidia-container-runtime.orig.$(date -Iseconds)
      3. Run the following command to download the fixed nvidia-container-runtime:
        curl -o /usr/bin/nvidia-container-runtime -sSL https://acs-public-mirror.oss-cn-hangzhou.aliyuncs.com/runc/nvidia-container-runtime-17.06-amd64
      4. Run the following command to set permission availability for nvidia-container-runtime:
        chmod +x /usr/bin/nvidia-container-runtime
      5. Run the following command to test whether nvidia-container-runtime works normally:
        nvidia-container-runtime -v
        #  runc version 1.0.0-rc3
        #  commit: fc48a25bde6fb041aae0977111ad8141ff396438-dirty
        #  spec: 1.0.0-rc5
        
        docker run -it --rm -e NVIDIA_VISIBLE_DEVICES=all ubuntu nvidia-smi -L
        #  GPU 0: Tesla P100-PCIE-16GB (UUID: GPU-122e199c-9aa6-5063-0fd2-da009017e6dc)
        Note This test is performed on a node of the GPU P100 model. Test outputs vary by GPU models.