Multi-platform Docker image building tutorial-Alibaba Cloud Developer Community

Yunqi information: [Click to view more industry information] here you can find the first-hand cloud information from different industries. What are you waiting for? Come on!

Adrian Mouat is honored as Docker Captain. He is the chief scientist of Container Solutions. Currently, he is developing Trow, which is a container registry for security management Kubernetes image streams in clusters.

Currently, Docker images have become the standard tool for testing and deploying new third-party software. Adrian is the main developer of the open-source Trow registry, and Docker images are the main way for people to install the tool. If he does not provide images, others will eventually release their own images, which will lead to repeated "wheel creation" and maintenance problems.

By default, the Docker image we create runs on linux/amd64. It is applicable to most development machines and cloud providers, but ignores users on other platforms. This group is very large-think about Raspberry Pi-based home laboratories, companies that produce Internet of Things devices, organizations running on IBM prototypes, and clouds that use low-power arm64 chips.

In general, users of these platforms usually build their own images or look for other solutions.

How do you build images for these platforms? The most obvious method is to build an image on the target platform. This applies to many scenarios. But if your target is s390x, I hope you have IBM mainframe that can be used. More common platforms, such as Raspberry Pi and IoT devices, usually have limited power, slow speed, or fail to build images.

What should we do? There are two options: 1. Target platform simulation, 2. Cross compilation. Interestingly, I found that there is a way to combine the two options with the best effect.

1. Simulation

let's start with the first option-simulation. There is a very good project called QEMU, which can simulate many platforms. With the recent preview of buildx, it is easier to use QEMU in Docker.

https://www.qemu.org/

https://github.com/docker/buildx

QEMU integration relies on a Linux kernel feature, which has a slightly mysterious name binfmt_misc handler. When Linux encounters an unrecognized executable file format (for example, a file format for different architectures), it will use the handler to check whether any "user space application" is configured to process the format (for example, simulator or VM). If yes, it will pass the executable file to the application.

To achieve this, we need to register the platform that we focus on in the kernel. If you are using Docker Desktop, you do not need to do this for most common platforms. If you are using Linux, you can register a handler in the same way as docker Desktop by running the latest Docker/binfmt Image. For example:

docker run --privileged --rm docker/binfmt:a7996909642ee92942dcd6cff44b9b95f08dad64

after completing this operation, you may need to restart Docker. If you want to have more control over the platform you want to register or use more sophisticated platforms (such as PowerPC), check the qus project.

https://github.com/dbhi/qus

Buildx has two different uses, but the simplest method may be to enable experimental features on the Docker CLI (if you have not already done so), edit ~/.docker/config. A json file that contains the following content:

{
    ...
     "experimental": “enabled”
}

you should be able to run docker buildx ls now and get output similar to the following:

$ docker buildx ls
NAME/NODE     DRIVER/ENDPOINT             STATUS   PLATFORMS
default       docker                               
  default     default                     running  linux/amd64, linux/arm64, linux/riscv64, linux/ppc64le, linux/s390x, linux/386, linux/arm/v7, linux/arm/v6

let's build an image for another platform, starting with the Dockerfile:

FROM debian:buster
CMD uname -m

if the build is normal, run the following command:

$ docker buildx build -t local-build .
…
$ docker run --rm local-build
x86_64

however, if we explicitly specify the platform for the build, run the following command:

$ docker buildx build --platform linux/arm/v7 -t arm-build .
…
$ docker run --rm arm-build
armv7l

successful! We have successfully built and run the armv7 image on the x86_64 notebook and have done very little. This technology is very effective, but for more complex constructions, you may find that it runs too slowly or encounters bugs in QEMU. In these cases, it is necessary to study whether your image can be cross-compiled.

2. Cross-compilation

some compilers can generate binary code for foreign platforms, the most famous of which are Go and Rust. Through the Trow registry project, we found that cross-compilation is the fastest and most reliable way to create images for other platforms. For example, here is the Dockerfile of the Trow armv7 image.

https://github.com/ContainerSolutions/trow/blob/master/docker/Dockerfile.armv7

the most important line is:

RUN cargo build --target armv7-unknown-linux-gnueabihf -Z unstable-options --out-dir ./out

it clearly tells Rust on which platform we want binary files to run. Then, we can use multi-level build to copy the binary file to the basic image of the target architecture (scratch can also be used for static compilation). However, for the Trow registry, I want to set more things in the final image, so the final phase actually starts:

FROM --platform=linux/arm/v7 debian:stable-slim

therefore, I actually mixed simulation and cross-compilation-cross-compilation is used to create binary files, and simulation is used to run and configure the final image.

3. List

in the above suggestions on simulation, you may have noticed that we use the -- platform parameter to set the build platform, but we specify the image as debian:buster in the FROM line. This seems meaningless-the platform certainly depends on the basic image and how it is built, not users' subsequent decisions.

This is because Docker uses a thing called a list of lists. For a given image, these lists contain pointers to images of different architectures. Because the official debian image has a defined list, when I pull this image from the notebook, I will automatically obtain the amd64 image, when I pull it from the raspberry pie, I will get the armv7 image.

To satisfy users, we can create a list for our own images. If we return to the previous example, we first need to rebuild and push the image to an image library:

$ docker buildx build --platform linux/arm/v7 -t amouat/arch-test:armv7 .
…
$ docker push amouat/arch-test:armv7
…
$ docker buildx build -t amouat/arch-test:amd64 .
…
$ docker push amouat/arch-test:amd64

next, create a list pointing to the two separate images and push them:

$ docker manifest create amouat/arch-test:blog amouat/arch-test:amd64 amouat/arch-test:armv7
Created manifest list docker.io/amouat/arch-test:blog
$ docker manifest push amouat/arch-test:blog
sha256:039dd768fc0758fbe82e3296d40b45f71fd69768f21bb9e0da02d0fb28c67648

now, Docker pulls and runs the image suitable for the current platform:

$ docker run amouat/arch-test:blog
Unable to find image 'amouat/arch-test:blog' locally
blog: Pulling from amouat/arch-test
Digest: sha256:039dd768fc0758fbe82e3296d40b45f71fd69768f21bb9e0da02d0fb28c67648
Status: Downloaded newer image for amouat/arch-test:blog
x86_64

readers with Raspberry Pi can try to run this image and make sure it works on that platform! Review: Not all users of Docker images run AMD64. buildx and QEMU support these users with only a small amount of work.

[Yunqi online class] product and technical experts share it every day! Course Address: https://yqh.aliyun.com/live

join the community immediately, face to face with experts, and keep abreast of the latest developments in the course! [Yunqi online classroom community] https://c.tb.cn/F3.Z8gvnK

original Published on: 2020-04-14 author: Adrian Mouat this article comes from: "The public account of the public account of the WeChat of the InfoQ". For more information, please pay attention to "the public account of the WeChat"

Selected, One-Stop Store for Enterprise Applications
Support various scenarios to meet companies' needs at different stages of development

Start Building Today with a Free Trial to 50+ Products

Learn and experience the power of Alibaba Cloud.

Sign Up Now