This topic describes GPU scheduling for Container Service for Kubernetes (ACK) clusters with GPU-accelerated nodes by setting up a GPU-accelerated environment to run TensorFlow.

Set up a GPU-accelerated environment to run TensorFlow

In most cases, data scientists use Jupyter to set up an environment for TensorFlow. The following example shows how to deploy a Jupyter application.

  1. Log on to the ACK console.
  2. In the left-side navigation pane of the ACK console, click Clusters.
  3. On the Clusters page, find the cluster that you want to manage and click the name of the cluster or click Details in the Actions column. The details page of the cluster appears.
  4. In the left-side navigation pane of the details page, choose Workloads > Deployments.
  5. On the Deployments page, click Create from YAML in the upper-right corner.
  6. Select the required cluster and namespace. Select a sample template, or set Sample Template to Custom and customize the template in the Template field. Then, click Create to create the application.
    Create a Deployment

    In this example, the template for the Jupyter application contains a Deployment and a Service.

    # Define the tensorflow deployment
    apiVersion: apps/v1
    kind: Deployment
      name: tf-notebook
        app: tf-notebook
      replicas: 1
      selector: # define how the deployment finds the pods it mangages
          app: tf-notebook
      template: # define the pods specifications
            app: tf-notebook
          - name: tf-notebook
            image: tensorflow/tensorflow:1.4.1-gpu-py3
       1          #Specify the number of NVIDIA GPUs that are scheduled.
            - containerPort: 8888
              hostPort: 8888
              - name: PASSWORD                #The password that is used to access the Jupyter application. You can modify the password as needed.
                value: mypassword
    # Define the tensorflow service
    apiVersion: v1
    kind: Service
      name: tf-notebook
      - port: 80
        targetPort: 8888
        name: jupyter
        app: tf-notebook
      type: LoadBalancer                           #A Server Load Balancer (SLB) instance is used to route internal traffic and perform load balancing.

    If you use a GPU scheduling solution provided by Kubernetes earlier than V1.9.3, you must define the volumes where the NVIDIA drivers are located.

        - hostPath:
            path: /usr/lib/nvidia-375/bin
            name: bin
        - hostPath:
            path: /usr/lib/nvidia-375
            name: lib

    This solution requires you to manually modify the template if you want to schedule the GPUs to other clusters. However, in Kubernetes 1.9.3 and later, you do not need to set hostPath. The NVIDIA plug-in automatically identifies the links of the libraries and executable files that are required by NVIDIA drivers.

  7. In the left-side navigation pane of the details page, choose Network > Services. Select the required cluster and namespace, find the tf-notebook Service, and then check its external endpoint.
    View the Service
  8. To connect to the Jupyter application, enter http://EXTERNAL-IP into the address bar of your browser and enter the password specified in the template.
  9. You can run the following program to verify that the Jupyter application is allowed to use GPU resources. The program lists all devices that can be used by TensorFlow:
    from tensorflow.python.client import device_lib
    def get_available_devices():
        local_device_protos = device_lib.list_local_devices()
        return [ for x in local_device_protos]
    Check the result