Docker 1.12 Routing and Container Application Distribution for Deployment on Alibaba Cloud

Abstract: Docker 1.12 now has built-in orchestration capabilities and mechanisms for service routing support (routing mesh) as well as multi-container application distribution deployment (Docker Stack and Distributed Application Bundle).

Description
Docker 1.12 now has built-in orchestration capabilities and mechanisms for service routing support (routing mesh) as well as multi-container application distribution deployment (Docker Stack and Distributed Application Bundle).
The mechanisms include:
● Distributed Application Bundle (DAB): DAB is a file format used for application distribution. It may comprise of multiple container service definitions.
● Stacks (Application Stacks): The application stack instance can be created from a DAB file. A single stack may contain multiple service instances.
Users familiar with Docker Compose must be versed with these concepts.
A Docker Compose template, initially designed as a standalone development tool, may comprise of multiple service descriptions and can be leveraged to create a complete application. However, it cannot perform deployment or management for distributed applications. To address this problem, DAB offers a standardized distribution format. Additionally, Docker provides tools that can be used to convert existing Docker Compose templates to the DAB format and deploy them as Docker Stack instances.
 
This blog post will focus on deploying a WordPress DAB application to a Swarm-mode Docker cluster on Alibaba Cloud. The application contains two services: WordPress and MySQL. The Alibaba Cloud Server Load Balancer is used for WordPress task container routing and load balancing.

Create Cluster
The first step is to create a Swarm-mode Docker cluster. There are two ways to go about it - using a ROS resource orchestration template to create a Swarm cluster with a single click or using the Docker Machine to manually create a complete environment. To fully understand the process, the following represents a detailed breakdown of the cluster creation using the Docker Machine method.
 
In this environment, a Swarm cluster containing three ECS instances is created, one of which is a manager. Also, a Server Load Balancer instance is created to carry out service routing and load balancing. These three ECS instances in the cluster will serve as the Server Load Balancer backend servers and the application is deployed, post which service routes are configured on this cluster.

First, a Server Load Balancer instance is created:
 
Once the Server Load Balancer instance is created, docker-machine is used to create three ECS nodes to compose the Docker cluster runtime environment.
The public environment variables are created using the following command:
export DEBUG=true
export ECS_ACCESS_KEY_ID=<your Access Key ID>
export ECS_ACCESS_KEY_SECRET=<your Access Key Secret>
export ECS_REGION=cn-beijing
export ECS_ZONE=cn-beijing-b
export ECS_SSH_PASSWORD=<ECS instance SSH password>
export ECS_UPGRADE_KERNEL=true
export MACHINE_DOCKER_INSTALL_URL= http://acs-public-mirror.oss-cn-hangzhou.aliyuncs.com/docker-engine/experimental/intranet
export ENGINE_REGISTRY_MIRROR=https://6udu7vtl.mirror.aliyuncs.com
export ECS_SLB_ID=<yourSLB instance ID>

Then, the following command is used to create the three ECS instances on Alibaba Cloud. This will automatically upgrade the operating system kernel, install the beta-version Docker engine with the latest configuration, and add the newly created Server Load Balancer instance as the specified Server Load Balancer for the backend servers.
docker-machine create -d aliyunecs node-1
docker-machine create -d aliyunecs node-2
docker-machine create -d aliyunecs node-3

Once the three ECS instances have been created, the following command is run to log onto node-1, initialize the Docker Swarm cluster, and display the master node's ECS intranet "eth0" address.
docker-machine ssh node-1 ip addr show eth0
docker-machine ssh node-1 docker swarm init --advertise-addr <NODE1_IP>
The result of the command:

Swarm initialized: current node (xxxxxxx) is now a manager.

To add a worker to this swarm, run the following command:
    docker swarm join \
    --token SWMTKN-1-1xxxxxxx-xxxxxxx \
    10.xx.xx.xx:2377

To add a manager to this swarm, run the following command:
    docker swarm join \
    --token SWMTKN-1-1xxxxxxx-xxxxxxx \
    10.xx.xx.xx:2377
After that, node-2 and node-3 are added to the Docker Swarm cluster as workers and docker-machine ssh <NODE> is executed. Then, the result of the first command will be copied and added to the worker nodes.
docker-machine ssh node-2 docker swarm join --token <WORKER_TOKEN> <NODE1_IP>:2377
docker-machine ssh node-3 docker swarm join --token <WORKER_TOKEN> <NODE1_IP>:2377

Now, the entire Docker cluster has been created, and the WordPress application deployment can be carried out.\

Generate Distributed Application Bundle (DAB)
This article uses WordPress as an example. Its Docker Compose template is:
version: '2'
services:
  wordpress:
    image: wordpress:4.5.3
    environment:
      - WORDPRESS_DB_PASSWORD=password
      - WORDPRESS_AUTH_KEY=changeme
      - WORDPRESS_SECURE_AUTH_KEY=changeme
      - WORDPRESS_LOGGED_IN_KEY=changeme
      - WORDPRESS_NONCE_KEY=changeme
      - WORDPRESS_AUTH_SALT=changeme
      - WORDPRESS_SECURE_AUTH_SALT=changeme
      - WORDPRESS_LOGGED_IN_SALT=changeme
      - WORDPRESS_NONCE_SALT=changeme
      - WORDPRESS_NONCE_AA=changeme
    ports:
      - 80:80
    restart: always
  mysql:
    image: mysql:5.7.13
    environment:
      - MYSQL_ROOT_PASSWORD=password
    restart: always

It is evident that it contains two services: "WordPress" and "MySQL." The docker-compose bundle command can be used to generate a DAB format file.
yili@yili-mbp:~/work/docker-experimental/wordpress$ docker-compose bundle
WARNING: Unsupported key 'restart' in services.wordpress - ignoring
WARNING: Unsupported key 'restart' in services.mysql - ignoring
Wrote bundle to wordpress.dab

Note: At present, DAB, Docker Service, and Stack capabilities are still incomplete. Various Docker Compose statements cannot be correctly processed yet, such as volume support.
The content of the produced "wordpress.dab" file is:
{
  "Services": {
    "mysql": {
      "Env": [
        "MYSQL_ROOT_PASSWORD=password"
      ],
      "Image": "mysql@sha256:a9a5b559f8821fe73d58c3606c812d1c044868d42c63817fa5125fd9d8b7b539",
      "Networks": [
        "default"
      ]
    },
    "wordpress": {
      "Env": [
        "WORDPRESS_AUTH_SALT=changeme",
        "WORDPRESS_LOGGED_IN_KEY=changeme",
        "WORDPRESS_AUTH_KEY=changeme",
        "WORDPRESS_NONCE_AA=changeme",
        "WORDPRESS_SECURE_AUTH_SALT=changeme",
        "WORDPRESS_NONCE_KEY=changeme",
        "WORDPRESS_DB_PASSWORD=password",
        "WORDPRESS_LOGGED_IN_SALT=changeme",
        "WORDPRESS_NONCE_SALT=changeme",
        "WORDPRESS_SECURE_AUTH_KEY=changeme"
      ],
      "Image": "wordpress@sha256:7bb9549fb6d80c230bec2da6bd181be8f30e5199687e53e5ad5744a3144eae1b",
      "Networks": [
        "default"
      ],
      "Ports": [
        {
          "Port": 80,
          "Protocol": "tcp"
        }
      ]
    }
  },
  "Version": "0.1"
}

For a detailed definition of the DAB format, refer to official documentation. **Note**: At present, DAB is still an experimental feature and its format may be changed in the future.

Create WordPress Stack
To deploy the application, log onto the cluster's master node. First, the relevant sample file from Github is retrieved.
git clone https://github.com/denverdino/docker-experimental
cd docker-experimental/wordpress/

Then, the docker deploy command is given to deploy the "wordpress" stack. By default, the Docker command finds a .dab file of the same name in the local directory and uses it as the stack deployment description.
root@node-1:~/docker-experimental/wordpress# docker deploy wordpress
Loading bundle from wordpress.dab
Creating network wordpress_default
Creating service wordpress_mysql
Creating service wordpress_wordpress

The corresponding service/task status can be viewed by using the docker service ls and docker stack tasks commands.
root@node-1:~/docker-experimental/wordpress# docker service ls
ID            NAME                 REPLICAS  IMAGE                                                                              COMMAND
3p616th5ypgc  wordpress_wordpress  1/1       wordpress@sha256:7bb9549fb6d80c230bec2da6bd181be8f30e5199687e53e5ad5744a3144eae1b  
dzyof5wz1s5y  wordpress_mysql      1/1       mysql@sha256:a9a5b559f8821fe73d58c3606c812d1c044868d42c63817fa5125fd9d8b7b539      
root@node-1:~/docker-experimental/wordpress# docker stack tasks wordpress
ID                         NAME                   SERVICE              IMAGE                                                                              LAST STATE       DESIRED STATE  NODE
2d9bjq6htn7xbcweocy192ubx  wordpress_wordpress.1  wordpress_wordpress  wordpress@sha256:7bb9549fb6d80c230bec2da6bd181be8f30e5199687e53e5ad5744a3144eae1b  Running 1 hours  Running        node-2
b31ol0r0eow2b4a5ut684dofy  wordpress_mysql.1      wordpress_mysql      mysql@sha256:a9a5b559f8821fe73d58c3606c812d1c044868d42c63817fa5125fd9d8b7b539      Running 1 hours  Running        node-3

Check the "wordpress_wordpress" service status by using the docker service inspect command:
root@node-1:~# docker service inspect wordpress_wordpress
[
    {
        "ID": "3p616th5ypgcrfxfb4a9v44ew",
        "Version": {
            "Index": 498
        },
        "CreatedAt": "2016-07-09T07:43:25.368994849Z",
        "UpdatedAt": "2016-07-09T07:43:25.378275593Z",
        "Spec": {
            "Name": "wordpress_wordpress",
            "Labels": {
                "com.docker.stack.namespace": "wordpress"
            },
            "TaskTemplate": {
                "ContainerSpec": {
                    "Image": "wordpress@sha256:7bb9549fb6d80c230bec2da6bd181be8f30e5199687e53e5ad5744a3144eae1b",
                    "Env": [
                        "WORDPRESS_AUTH_SALT=changeme",
                        "WORDPRESS_LOGGED_IN_KEY=changeme",
                        "WORDPRESS_AUTH_KEY=changeme",
                        "WORDPRESS_NONCE_AA=changeme",
                        "WORDPRESS_SECURE_AUTH_SALT=changeme",
                        "WORDPRESS_NONCE_KEY=changeme",
                        "WORDPRESS_DB_PASSWORD=password",
                        "WORDPRESS_LOGGED_IN_SALT=changeme",
                        "WORDPRESS_NONCE_SALT=changeme",
                        "WORDPRESS_SECURE_AUTH_KEY=changeme"
                    ]
                }
            },
            "Mode": {
                "Replicated": {
                    "Replicas": 1
                }
            },
            "Networks": [
                {
                    "Target": "5cb3ojhzk5snap2s3nsjtexsf",
                    "Aliases": [
                        "wordpress"
                    ]
                }
            ],
            "EndpointSpec": {
                "Mode": "vip",
                "Ports": [
                    {
                        "Protocol": "tcp",
                        "TargetPort": 80
                    }
                ]
            }
        },
        "Endpoint": {
            "Spec": {
                "Mode": "vip",
                "Ports": [
                    {
                        "Protocol": "tcp",
                        "TargetPort": 80
                    }
                ]
            },
            "Ports": [
                {
                    "Protocol": "tcp",
                    "TargetPort": 80,
                    "PublishedPort": 30000
                }
            ],
            "VirtualIPs": [
                {
                    "NetworkID": "6qj71vtgeipenr838hbjt1ash",
                    "Addr": "10.255.0.6/16"
                },
                {
                    "NetworkID": "5cb3ojhzk5snap2s3nsjtexsf",
                    "Addr": "10.0.0.5/24"
                }
            ]
        }
    }
]

The "wordpress_wordpress" service has now published the TCP port "30000" to provide external services. Now the question that arises is - how can one access this service?

Use Server Load Balancer to Create Service Routes
Alibaba Cloud's Server Load Balancer is a service designed to distribute traffic among multiple ECS instances. In the following example, Server Load Balancer is used to configure routes and carry out load balancing for Swarm cluster services.
First, add a listener port. Then select "http" and "8080" (or any other port) for "Frontend Protocol [Port]", "http", and "30000" for "Backend Protocol [Port]". **Note**: The backend port must be consistent with "PublishedPort" in the service status.
 
While configuring a health check, use "/license.txt" as the checking path. The service is considered healthy if WordPress service's Apache is accessible.
 
The configuration is now complete.
 
By accessing the Server Load Balancer instance's public IP address, the familiar WordPress settings interface becomes visible.

To scale the service and test load balancing performance, use the docker service scale command.
root@node-1:~# docker service scale wordpress_wordpress=6
wordpress_wordpress scaled to 6
root@node-1:~# docker service ps wordpress_wordpress
ID                         NAME                   SERVICE              IMAGE                                                                              LAST STATE         DESIRED STATE  NODE
2d9bjq6htn7xbcweocy192ubx  wordpress_wordpress.1  wordpress_wordpress  wordpress@sha256:7bb9549fb6d80c230bec2da6bd181be8f30e5199687e53e5ad5744a3144eae1b  Running 1 hours    Running        node-2
8w2dsfyyjb9cjiiw3d4j0111h  wordpress_wordpress.2  wordpress_wordpress  wordpress@sha256:7bb9549fb6d80c230bec2da6bd181be8f30e5199687e53e5ad5744a3144eae1b  Running 2 minutes  Running        node-1
42ym2i9lic0bvg1763acdudwc  wordpress_wordpress.3  wordpress_wordpress  wordpress@sha256:7bb9549fb6d80c230bec2da6bd181be8f30e5199687e53e5ad5744a3144eae1b  Running 2 minutes  Running        node-1
6f33yfwbrpj9g601z2c2gs3qu  wordpress_wordpress.4  wordpress_wordpress  wordpress@sha256:7bb9549fb6d80c230bec2da6bd181be8f30e5199687e53e5ad5744a3144eae1b  Running 2 minutes  Running        node-3
6xxkek3t8dgg48zs5mjwrakg8  wordpress_wordpress.5  wordpress_wordpress  wordpress@sha256:7bb9549fb6d80c230bec2da6bd181be8f30e5199687e53e5ad5744a3144eae1b  Running 2 minutes  Running        node-2
5idxgssy8t439jgoj65d6g4ln  wordpress_wordpress.6  wordpress_wordpress  wordpress@sha256:7bb9549fb6d80c230bec2da6bd181be8f30e5199687e53e5ad5744a3144eae1b  Running 2 minutes  Running        node-2

Brief Analysis of Docker Routing Mesh
The previous example increases one’s interest in the network configurations for corresponding services and routes. A brief analysis follows.
On the "node-1" node, execute this command:
root@node-1:~# docker network ls
NETWORK ID          NAME                DRIVER              SCOPE
29de61a371e7        bridge              bridge              local              
bdeb358293a3        docker_gwbridge     bridge              local              
78ee3e892931        host                host                local              
6qj71vtgeipe        ingress             overlay             swarm              
70d0aaba61cc        none                null                local              
5cb3ojhzk5sn        wordpress_default   overlay             swarm

Two overlay networks can be seen on the cluster: "ingress" and "wordpress_default". "Ingress" is the access route network that is automatically created once the cluster is created. "Wordpress_default" is also an automatic creation during "wordpress.dab" deployment.
If a value is not specified manually, the Swarm Manager will automatically allocate a PublishedPort in the range of 30000–32767 for services that require external access. In the above example, "wordpress_wordpress" was allocated the port 30000.
Each node in the Swarm cluster listens to the same PublishedPort to provide access routes for services. Therefore, configure a listener port on Server Load Balancer to perform forwarding and load balancing for the nodes in the cluster, regardless of whether or not the backend nodes are running service task containers. In the Swarm cluster, all nodes can connect to running task containers through the ingress network and ensure that incoming traffic is directed to the correct container instance.
The next diagram depicts load balancing of the ingress network in the routing mesh. Each service is allocated with a virtual IP address. Load balancing between these virtual IP addresses and service task containers is achieved through IPVS.
Note: Please do not confuse the port mapping mechanism in a routing mesh with the port publishing in a bridge network.


The following command can be used to view iptables for the traffic forwarding configuration between the ECS host ports and services' virtual IP addresses.
root@node-1:~# iptables -L DOCKER-INGRESS -t nat
Chain DOCKER-INGRESS (2 references)
target     prot opt source               destination        
DNAT       tcp  --  anywhere             anywhere             tcp dpt:30000 to:172.18.0.2:30000
RETURN     all  --  anywhere             anywhere

Conclusion
Docker 1.12 features a new DAB (Distributed Application Bundle) format that simplifies the deployment of multiple container applications on a cluster. However, current DAB capabilities and level of sophistication require improvisation. Also, some problems with Docker Compose compatibility require appropriate solutions. We hope to provide solutions to these problems, moving forward.