By Yan Tsz Kin, Solutions Architect
The purpose of this article is to present a simple and practical example of provisioning an Alibaba Cloud Kubernetes (ACK) Multi-AZ Kubernetes cluster with a sample application on Alibaba Cloud using Terraform. This is also an example of Infrastructure as Code which is a key example of enabling best practices in DevOps.
I will walk through a sample project that is available in my GitHub repository: https://github.com/kin-alibaba/terraform-kubernetes
Our target infrastructure includes:
Our target architecture is illustrated in the diagram below:
Terraform allow us to describe the target environment, and then manage the lifecycle of the environment. In this case, Terraform files are modularized into multiple files for easier management in which variables are externalized to each "config" files.
In the accesskey.tf, update the Alibaba Cloud Access Key and Secret.
variable "access_key" {
default = ""
}
variable "secret_key" {
default = ""
}In vpc_config.tf, update the Region and Availability Zone information, you may check out the Region ID at https://www.alibabacloud.com/help/doc-detail/40654.htm, Zone ID is typically appending –zone name to Region ID, e.g. Hong Kong Zone C is having Region ID cn-hongkong, Zone ID is cn-hongkong-c, Singapore Zone C is ap-southeast1c.
variable "region" {
default = "cn-hongkong"
}
variable "azone1" {
default = "cn-hongkong-b"
}
variable "azone2" {
default = "cn-hongkong-c"
}
variable "azone3" {
default = "cn-hongkong-c"
}Make sure following files exist and empty:
This is critical for Terraform to proceed with Kubernetes provider after cluster provisioned.
Execute IaC.sh or command "terraform apply –auto-approve"
Create VPC and Vswitches
resource "alicloud_vpc" "vpc" {
name = "${var.vpc_name}"
cidr_block = "${var.vpc_cidr}"
}
resource "alicloud_vswitch" "vswitch1" {
availability_zone = "${var.azone1}"
name = "${var.vswitch_name1}"
cidr_block = "${var.vswitch_cidr1}"
vpc_id = "${alicloud_vpc.vpc.id}"
}
resource "alicloud_vswitch" "vswitch2" {
availability_zone = "${var.azone1}"
name = "${var.vswitch_name2}"
cidr_block = "${var.vswitch_cidr2}"
vpc_id = "${alicloud_vpc.vpc.id}"
}
resource "alicloud_vswitch" "vswitch3" {
availability_zone = "${var.azone3}"
name = "${var.vswitch_name3}"
cidr_block = "${var.vswitch_cidr3}"
vpc_id = "${alicloud_vpc.vpc.id}"
}Create NAT Gateway and EIP instance and associate to the VPC
resource "alicloud_nat_gateway" "nat_gateway" {
vpc_id = "${alicloud_vpc.vpc.id}"
specification = "Small"
name = "kin-k8s-tf-nat-gw"
depends_on = [
"alicloud_vswitch.vswitch1",
"alicloud_vswitch.vswitch2",
"alicloud_vswitch.vswitch3"
]
}
resource "alicloud_eip" "eip1" {
bandwidth = "20"
}
resource "alicloud_eip_association" "eip1_asso" {
allocation_id = "${alicloud_eip.eip1.id}"
instance_id = "${alicloud_nat_gateway.nat_gateway.id}"
}
resource "alicloud_snat_entry" "snat1" {
snat_table_id = "${alicloud_nat_gateway.nat_gateway.snat_table_ids}"
source_vswitch_id = "${alicloud_vswitch.vswitch1.id}"
snat_ip = "${alicloud_eip.eip1.ip_address}"
}
resource "alicloud_snat_entry" "snat2" {
snat_table_id = "${alicloud_nat_gateway.nat_gateway.snat_table_ids}"
source_vswitch_id = "${alicloud_vswitch.vswitch2.id}"
snat_ip = "${alicloud_eip.eip1.ip_address}"
}
resource "alicloud_snat_entry" "snat3" {
snat_table_id = "${alicloud_nat_gateway.nat_gateway.snat_table_ids}"
source_vswitch_id = "${alicloud_vswitch.vswitch3.id}"
snat_ip = "${alicloud_eip.eip1.ip_address}"
}Create RDS instance
resource "alicloud_db_instance" "rdsinstance" {
engine = "MySQL"
engine_version = "5.6"
instance_type = "rds.mysql.t1.small"
instance_storage = "5"
vswitch_id = "${alicloud_vswitch.vswitch1.id}"
security_ips = ["${var.vswitch_cidr1}", "${var.vswitch_cidr2}", "${var.vswitch_cidr3}", "${alicloud_eip.eip1.ip_address}"]
#depends_on = ["alicloud_cs_kubernetes.k8s-cluster"]
}
resource "alicloud_db_database" "rdsdb" {
instance_id = "${alicloud_db_instance.rdsinstance.id}"
name = "demodb"
character_set = "utf8"
}
resource "alicloud_db_account" "mysqlroot" {
instance_id = "${alicloud_db_instance.rdsinstance.id}"
name = "${var.db_credential["username"]}"
password = "${var.db_credential["password"]}"
}
resource "alicloud_db_account_privilege" "default" {
instance_id = "${alicloud_db_instance.rdsinstance.id}"
account_name = "${alicloud_db_account.mysqlroot.name}"
privilege = "ReadWrite"
db_names = ["${alicloud_db_database.rdsdb.name}"]
}Create ACK cluster
resource "alicloud_cs_kubernetes" "k8s-cluster" {
name = "${var.k8clu_name}"
vswitch_ids = [ #Indicates Multiple Availability Zone
"${alicloud_vswitch.vswitch1.id}",
"${alicloud_vswitch.vswitch2.id}",
"${alicloud_vswitch.vswitch3.id}"
]
master_instance_types = ["${var.master_type["zone1"]}","${var.master_type["zone1"]}","${var.master_type["zone2"]}"]
master_disk_category = "cloud_efficiency" #cloud_ssd or cloud_efficiency
master_disk_size = "40"
worker_instance_types = ["${var.worker_type["zone1"]}","${var.worker_type["zone1"]}","${var.worker_type["zone2"]}"]
worker_disk_category = "cloud_efficiency" #cloud_ssd or cloud_efficiency
worker_disk_size = "40"
worker_data_disk_category = "cloud_ssd" #cloud_ssd or cloud_efficiency
worker_data_disk_size = "40"
worker_numbers = [1,1,1]
key_name = "${alicloud_key_pair.k8s-ssh-key.key_name}" #for ECS ssh key auth, either key_name or password
#password = "${var.k8ssh["password"]}" #for ECS password auth, either key_name or password
new_nat_gateway = "false"
pod_cidr = "172.20.0.0/16"
service_cidr = "172.30.0.0/16"
slb_internet_enabled = "true" #for SLB of K8S API Server
enable_ssh = "true" #SSH login kubernetes
install_cloud_monitor = "true"
cluster_network_type = "terway"
kube_config = "${var.kube_cli["cfg"]}"
client_cert = "${var.kube_cli["client_cert"]}"
client_key = "${var.kube_cli["client_key"]}"
cluster_ca_cert = "${var.kube_cli["k8s_ca"]}"
depends_on = [
"alicloud_eip_association.eip1_asso",
"alicloud_snat_entry.snat1",
"alicloud_snat_entry.snat2",
"alicloud_snat_entry.snat3"
]
}Deploy application on ACK with Horizontal Pod Autoscaler enabled and expose the deployment to the Internet
variable "app1"{
default =
{
name = "sample"
namespace = "default"
min_replicas = 2
max_replicas = 10
cpu_threshold = 80
#image_repo = "registry-intl.ap-southeast-1.aliyuncs.com/kin-test-acr/demo"
image_repo = "seyantszkin/demo" #
image_ver = "v1.0"
svc_port = 80
container_port = 8080
svc_type = "LoadBalancer"
}
}
resource "kubernetes_deployment" "app1" {
metadata {
name = "${var.app1["name"]}"
labels {
app = "${var.app1["name"]}"
}
namespace = "${var.app1["namespace"]}"
}
spec {
replicas = "${var.app1["min_replicas"]}"
selector {
match_labels {
app = "${var.app1["name"]}"
}
}
template {
metadata {
labels {
app = "${var.app1["name"]}"
}
}
spec {
container {
env {
name = "MYSQL_HOST"
value = "${alicloud_db_instance.rdsinstance.connection_string}"
}
env {
name = "MYSQL_PORT"
value = "3306"
}
env {
name = "DB_USERNAME"
value = "${var.db_credential["username"]}"
}
env {
name = "DB_PASSWORD"
value = "${var.db_credential["password"]}"
}
image = "${var.app1["image_repo"]}:${var.app1["image_ver"]}"
name = "${var.app1["name"]}"
port {
container_port = "${var.app1["container_port"]}"
protocol = "TCP"
}
#resources {
# requests {}
#}
}
#image_pull_secrets {
# name = "${kubernetes_secret.reg_secret.metadata.0.name}"
#}
}
}
}
depends_on = [
"alicloud_cs_kubernetes.k8s-cluster",
"alicloud_db_database.rdsdb"
]
}
resource "kubernetes_horizontal_pod_autoscaler" "hpa1" {
metadata {
name = "${var.app1["name"]}"
}
spec {
max_replicas = "${var.app1["max_replicas"]}"
min_replicas = "${var.app1["min_replicas"]}"
target_cpu_utilization_percentage = "${var.app1["cpu_threshold"]}"
scale_target_ref {
kind = "Deployment"
name = "${var.app1["name"]}"
}
}
depends_on = ["kubernetes_deployment.app1"]
}
resource "kubernetes_service" "svc1" {
metadata {
name = "${var.app1["name"]}-svc"
namespace = "${var.app1["namespace"]}"
}
spec {
selector {
app = "${var.app1["name"]}"
}
port {
port = "${var.app1["svc_port"]}"
target_port = "${var.app1["container_port"]}"
protocol = "TCP"
}
session_affinity = "None"
type = "${var.app1["svc_type"]}"
}
depends_on = ["kubernetes_deployment.app1"]
}Bind a DNS A record to the Server Load balancer which exposes the application to the Internet
resource "alicloud_dns_record" "svc1" {
name = "alitest.org" # A domain name (e.g. alicloud.org) registered under your Alibaba Cloud account
host_record = "${var.app1["name"]}"
type = "A"
value = "${kubernetes_service.svc1.load_balancer_ingress.0.ip}"
}The output of the Terraform should look like following screenshot:
If the local machine that runs the Terraform has Kubernetes client installed, i.e. kubectl, the ~/.kube/config is downloaded, you may try connect to the Kubernetes cluster directly for further actions or tasks such as create Ingress which is not yet supported by Terraform Kubernetes provider.
Finally, the web application should be up and running as shown below.
Terraform is not able to repair, fix or retry in case of timeout of the provisioning processes. In addition, it is possible that metadata of Terraform may be corrupted when applying changes to the Terraform managed stack. Therefore, it is better to wrap up the provisioning process or even the Terraform command in a shell script to handle exceptions.
In this post, I have shown how Terraform can integrate resources on Alibaba Cloud with an application provisioned on ACK cluster. You can extend it to stack lifecycle management processes of your Alibaba Cloud resources.
To learn more about Kubernetes on Alibaba Cloud, visit https://www.alibabacloud.com/product/kubernetes
ACtivate 2019: Empower Your Digital Transformation with Alibaba Cloud
Consumer Insights: Empowering Customer 360 and Precision Marketing with Data Intelligence
2,605 posts | 747 followers
FollowAlibaba Container Service - April 17, 2024
Alibaba Clouder - January 13, 2021
Alibaba Clouder - January 12, 2021
Alibaba Clouder - January 5, 2021
Alibaba Clouder - June 9, 2020
Alibaba Clouder - September 10, 2020
2,605 posts | 747 followers
Follow
Container Service for Kubernetes
Alibaba Cloud Container Service for Kubernetes is a fully managed cloud container management service that supports native Kubernetes and integrates with other Alibaba Cloud products.
Learn MoreLearn More
Elastic Container Instance
An agile and secure serverless container instance service.
Learn MoreMore Posts by Alibaba Clouder
294618655049306610 April 12, 2019 at 7:39 am
very nice post, learned a lot