Use Docker in DSW - Platform For AI - Alibaba Cloud Documentation Center

To start sub-containers with a graphical user interface, see Manage Sub-containers with DockerBoard.

Limitations

Secondary containers are available only for DSW instances created from a Lingjun resource group or a version 1.0 general-purpose resource group.
The Docker Daemon configuration (/etc/docker/daemon.json) for secondary containers currently cannot be modified in DSW instances, so you cannot customize options such as data-root and registry-mirrors.
Restarting an instance deletes all runtime data in a secondary container, including built images, even if the cloud disk feature is enabled. To prevent data loss, regularly save your important environments and data.

When starting a secondary container, you cannot use parameters that grant elevated privileges or pose security risks. The following table lists common restricted parameters:

Parameter	Description
--privileged	Grants the container nearly all host machine permissions.
--ipc=host	Shares the host's IPC namespace with the container.
--security-opt	Allows bypassing or modifying container security policies.
--cap-add	Adds Linux kernel capabilities to the container, expanding its permission scope.

Note

If you see the error Error response from daemon: authorization denied by plugin authZ: Permission denied when running docker run or docker create, you used a restricted parameter.

Create and configure a DSW instance

When creating a DSW instance, configure the following key parameters.

Resource Type: Select Resource Quota.
Resource Quota: Select the Lingjun resource quota that you created. For more information, see Create a new resource group and purchase Lingjun computing resources.
Enable Multi-Container Isolation (DinD): Turn on this switch.

Click OK to create the instance. For the next steps, see Working with secondary containers.

Work with secondary containers

Start a secondary container

Run the following command to start and enter a secondary container:

docker run -it dsw-registry.cn-hangzhou.cr.aliyuncs.com/pai/pytorch:1.8PAI-gpu-py36-cu101-ubuntu18.04 /bin/bash

Mounting devices:

Mount GPUs: Use --gpus=all to mount all GPU devices in the DSW instance.
Mount a PPU: Unlike GPUs, you must manually mount the PPU device to the secondary container by using the --device parameter:
```
docker run -it --network=host \
  --device=/dev/alixpu_ppu0 \
  --device=/dev/alixpu \
  --device=/dev/alixpu_ctl \
  your-ppu-enabled-image
```
Note
The secondary container's image must have the required PPU components pre-installed to use the PPU.

Mounting data:

Use Docker Bind Mount to mount data into a secondary container. The data is first mounted to the DSW instance by using a Dataset Mounting or Storage Path Mounting (Volume Mount and tmpfs Mount are not currently supported). For example, if a DSW instance has data mounted at /mnt/data0 and /mnt/data1, use the -v or --mount parameter to bind mount these paths into the secondary container:

# In this example, <path_in_dsw> can be /mnt/data0, /mnt/data1, or their subpaths.
# Other paths cannot currently be mounted into the secondary container.

# Use the -v parameter
docker run -v <path_in_dsw>:<path_in_secondary_container>[:options] {image}

# Use the --mount parameter
docker run  --mount type=bind,source=<path_in_dsw>,target=<path_in_secondary_container>[,readonly] {image}

Access a secondary container

Use the docker exec command to start a new process in a running container.

For example, to start a bash shell in interactive mode (-i) with a pseudo-terminal (-t), run:

docker exec -it your-container /bin/bash

Note

Currently, you cannot use parameters that pose a security risk, such as --privileged, to enter a container.

You can also use the docker attach command to attach to the terminal of the main process in a running container. If you started the container with standard input open (-i) and a pseudo-TTY allocated (-t), you can view and interact with the container's standard I/O streams (stdin/stdout/stderr):

docker attach {container_id}

Note

To detach from the container without stopping it, use the default escape sequence: Ctrl+p then Ctrl+q.

Build an image

In a DSW instance, you can build an image from a Dockerfile:

docker build -t test-build .

Note

Due to network restrictions, you cannot pull images directly from Docker Hub. Use an image from Alibaba Cloud Container Registry (ACR) as the base image in your FROM instruction.

After the build completes, run the following command to view the new image:

root@dsw-330808-8445dd65f4-mhnrt:/mnt/workspace/test# docker image ls
REPOSITORY          TAG         IMAGE ID       CREATED              SIZE
test-build          latest      fad7fdb5e81a   About a minute ago   15.4GB

Pull images

To pull a public image from ACR, run the following command:

docker pull dsw-registry.cn-hangzhou.cr.aliyuncs.com/pai/pytorch:1.8PAI-gpu-py36-cu101-ubuntu18.04

To pull an image from a private repository, first log in with the docker login command:

docker login --username={username} registry.cn-hangzhou.aliyuncs.com

FAQ

Q: Why can't I see a secondary container's processes when I run the ps -aux command in a DSW instance?

The main container and secondary containers in DSW do not share the same PID namespace, so processes inside a secondary container are not visible from the main container.

Q: Why does pulling an image from Docker Hub fail?

Pulling images directly from Docker Hub fails due to network restrictions. You will see an error similar to the following:

root@dsw-381955:/mnt/workspace# docker run --network=host -it ubuntu /bin/bash
Unable to find image 'ubuntu:latest' locally
docker: Error response from daemon: Get "https://registry-1.docker.io/v2/": context deadline exceeded.
See 'docker run --help'.

Instead, pull images from the Artifact Center in Alibaba Cloud Container Registry (ACR). Alternatively, see Cross-domain pull of overseas models or container images.

Q: What should I do if I see the error "Error response from daemon: authorization denied by plugin authZ: Permission denied"?

This error occurs when you use an unsupported Docker command or parameter. For assistance, submit a ticket to technical support.

Q: What should I do if I see the error "Cannot connect to the Docker daemon at unix:///var/run/docker.sock. Is the docker daemon running?"

Check whether the /var/docker/proxy/docker-proxy.sock file exists in your DSW instance. If it does, run the following command and try again:

export DOCKER_HOST=unix:///var/docker/proxy/docker-proxy.sock

After setting the environment variable, run the Docker command again.