By Sajith Venkit, Alibaba Cloud Tech Share Author. Tech Share is Alibaba Cloud's incentive program to encourage the sharing of technical knowledge and best practices within the cloud community.
In this article, we will show how you can use Terraform to fully automate the build of a 3-node Docker Enterprise 2.1 cluster on Alibaba Cloud. If you are in the process of planning or setting up an Enterprise Docker Cluster for your containers, in any of the public cloud platforms, this article and the templates will be useful.
Even if you are not building a Docker Enterprise cluster, you may still find this article useful if you'd like to automate your infrastructure build in Alibaba Cloud.
As my objective was to show the art of possible using IaC (Infrastructure as Code) to automate build in Alibaba Cloud, I thought why not do that using the most popular platforms, hence decided to:
Docker Enterprise 2.1 is a Containers-as-a-Service (CaaS) platform that enables a secure software supply chain and deploys diverse applications for high availability across disparate infrastructure, both on-premises and in the cloud. It is a secure, scalable, and supported container platform for building and orchestrating applications across multi-tenant Linux, Windows Server 2016, and IBM Z environments.
One thing that I always loved about Docker is their Simplicity and Customer Centricity. That's exactly what they have done with the release of Enterprise 2.1 too. With Docker EE 2.1, you now have a freedom of choice as it can:
So, if you are an Enterprise customer who is looking to embark on a project to either migrate your legacy applications to Containers or keen to embrace DevOps for development of new applications, I'd strongly recommend Docker EE 2.1 as your CaaS platform.
You can start small and scale your cluster as you grow your container base. You can start utilizing the much simpler Docker Swarm for initial orchestration and switch to Kubernetes later if you really need it.
Docker EE 2.1 cluster also comes with components such as:
It also has enterprise security features like encrypted communication, application isolation, vulnerability scanning for images and much more.
Terraform is one of my favorite Orchestration/IaC tools out there. I just love the power and flexibility that Terraform offers for deploying new services to any public cloud platforms. You just define what you need and ask Terraform to Go and Build. It is that simple.
I chose Terraform for this automation as it is pretty much platform agnostic. Though you can't use the same templates for any cloud service provider it is quite easy to customize to a different provider once it is developed for a specific Cloud platform.
To learn more about Terraform, visit the HashiCorp website or read this good summary by MVP Alberto Roura's Tech Share article about Terraform.
For this demo, I chose to build a small 3-node Docker Enterprise 2.1 cluster.
1. One Alibaba Cloud ECS Linux server that hosts both Docker UCP and Docker Trusted Registry (DTR). Same node will also be configured as the Docker Swarm Manager and Kubernetes Master.
2. One Linux host which will be automatically joined as a worker node, in the Docker Swarm created by the UCP host.
3. One Windows host which will be automatically joined as a worker node, in the Docker Swarm created by the UCP host.
If you would like to get on with the cluster build right away, go to my GitHub repository and follow the instructions there.
Once you get the pre-requisites ready, you could get the cluster built in less than 30 minutes.
If you'd rather understand the templates first before jumping into action, the following sections will take you through the details of the Terraform files.
The terraform configuration files in my GitHub repository, and the bash & powershell scripts, have adequate comments to help you understand them better. In this article, I would like to focus on explaining the purpose of each of those template or script.
Terraform, as you may be aware, helps to deploy the Infrastructure by letting users define what they need, using the High-level Configuration Language (HCL). Terraform will build a detailed execution plan based on this definition and deploy the infrastructure on the platform specified.
Terraform supports a wide range of Cloud service providers and certainly the major Cloud providers and VMWare vSphere for on-premise Data Center.
The complete definition of our Docker Enterprise Cluster could be specified in a single .tf definition file but it is recommended to split into multiple .tf files for ease of maintaining and readability. All .tf files in the main folder form part of the root (main) module.
Terraform will load all configuration files (.tf files) in the folder, in alphabetical order. The order of variables, resources, etc. defined within the configuration doesn't matter. Terraform configurations are declarative, so references to other resources and variables do not depend on the order they're defined.
Input variables serve as parameters for the Terraform module. All variables must be declared in a .tf file (e.g., variables.tf) and their values could be passed in command line during execution OR in a separate .tfvars file (e.g., terraform.tfvars)
I recommend using UPPER CASE for the input variable names so they stand out from the terraform local variables, defined using locals and terraform resource names. I would also suggest to give the values for variables in a separate terraform.tfvars file, so you can choose to not send this file when sharing with others as it may contain your account secret keys, passwords etc. During execution, if terraform cannot find a .tfvars file and the default values for variables are not defined, it prompts the user.
provider.tf
This file defines the keys to connect to Alicloud and the region where you want the resources created
network-security.tf
This file defines the VPC, vSwitch, Security Group and the security/firewall rules to restrict access to the docker hosts.
The vSwitch and Security groups defined in this file will be mapped to our hosts, in the ECS (Elastic Compute Service) definition.
Few points to note:
This file defines the main host where Docker UCP and DTR will be installed. As part of Docker UCP installation, it will also initiate creation of Docker Swarm to which the other hosts will join as worker nodes.
Few points to note:
internet_max_bandwidth_out attribute ensures a Public IP is assigned as part of VM creation.connection definition for file provisioner to copy the installation scripts and for remote-exec provisioner to run the setup automatically for Docker EE, UCP and DTRssh key pair for the UCP host and the linux worker node is defined in key-pair.tf
linux-worker.tf
This file defines the linux worker node. Quite similar to the docker-host.tf file, except that this only has scripts to install Docker EE and join as worker node.
windows-worker.tf
This file defines the windows worker node. Key differences from the Linux node definition are:
password attribute instead of the key_name attribute used in Linux for ssh-key pair.user-data attribute which is used for bootstrapping the VM, including configuration of WinRM, so further scripts can be executed using WinRM
winRM instead of ssh used in Linux host.output.tf
This file defines the values returned by the module. These values will be printed once Terraform successfully completes the execution.
Values defined in output.tf can be printed anytime using the command terraform output, after a successful execution.
To retrieve further details of the Terraform execution, use terraform show
Scripts used to bootstrap Linux hosts are in the folder lin-files. They are called using the remote-exec provisioner in the ECS VM definition.
These scripts are used to setup Docker Enterprise, Docker UCP and Docker Trusted Registry as per the documentation in Docker website.
Scripts used to bootstrap Windows host are in the folder win-files. They are called using the remote-exec provisioner in the ECS VM definition.
These scripts are used to setup Docker Enterprise and also to join as worker node. Docker UCP and Docker Trusted Registry cannot be installed on Windows hosts. They can only be worker nodes.
Key point to note in the Windows scripts,
To ensure Invoke-RestMethod works fine when accessing the Docker API, following changes had to be done in the script prior to calling the Docker API.
This article was intended only to introduce you to the basic concepts of Docker Enterprise and how you can use Terraform to automate deployments in Alibaba Cloud. The definition files and scripts in the GitHub repository will help you setup a basic Docker Enterprise cluster and also provide you tips which you can use for other installations.
In subsequent articles, we will look at how to scale your cluster with a load balancer and additional nodes and also details on deploying containers using Docker Swarm or Kubernetes orchestration.
This section lists several useful references if you would like to learn more about Docker, Terraform, and Automation
1. Docker Enterprise
https://docs.docker.com/ee/
https://www.docker.com/products/docker-enterprise
2. Terraform
https://www.terraform.io
https://www.terraform.io/docs/providers/alicloud/index.html
3. My GitHub repository
https://github.com/sajiv3m/docker-terraform-alicloud
Setup and Deploy Geodjango App on Alibaba Cloud Part 1: Basic Setup
2,605 posts | 747 followers
FollowAlibaba Clouder - July 2, 2020
Alibaba Clouder - March 29, 2019
Alibaba Clouder - July 18, 2019
Alibaba Clouder - June 21, 2019
Alibaba Clouder - June 28, 2019
Alibaba Cloud Community - July 18, 2022
2,605 posts | 747 followers
FollowLearn More
ECS(Elastic Compute Service)
Elastic and secure virtual cloud servers to cater all your cloud hosting needs.
Learn More
Container Registry
A secure image hosting platform providing containerized image lifecycle management
Learn MoreMore Posts by Alibaba Clouder