How to deploy a managed Kubernetes cluster by using Terraform - Terraform

The Container Service for Kubernetes (ACK) console provides a graphical interface to guide you through the process of creating a managed Kubernetes cluster. However, it can be cumbersome to use the console to create large numbers of managed clusters. Using Terraform can help you resolve this issue. This topic describes how to deploy a managed Kubernetes cluster by using Terraform.

Create a managed Kubernetes cluster

In alicloud_cs_managed_kubernetes, a Terraform resource document for Alibaba Cloud managed Kubernetes clusters, you can view the parameters that are used to create a managed Kubernetes cluster. The parameters are divided into input arguments and output attributes. The Argument Reference section lists the required and optional arguments and their characteristics. For example, name and name_prefix are two required arguments, but they are mutually exclusive and cannot be simultaneously specified. If the name argument is specified, the name value is used as the cluster name. If the name_prefix argument is specified, a cluster name that starts with the prefix specified by name_prefix is automatically generated. Before you use an account to create a node pool that has auto scaling enabled, grant the required permissions to the account. For more information, see Use Terraform to create a node pool that has auto scaling enabled.

Compile a cluster description by referring to the Argument Reference section in the preceding document. Sample code:

provider "alicloud" {
    region = "cn-zhangjiakou"
}
# The default resource name.
variable "name" {
  default = "my-first-kubernetes-demo"
}
# The name of the Simple Log Service project.
variable "log_project_name" {
  default = "my-first-kubernetes-sls-demo"
}
# The zone.
data "alicloud_zones" "default" {
  available_resource_creation = "VSwitch"
}
# The Elastic Compute Service (ECS) instance specifications of the worker nodes.
data "alicloud_instance_types" "default" {
  availability_zone    = data.alicloud_zones.default.zones[0].id
  cpu_core_count       = 2
  memory_size          = 4
  kubernetes_node_role = "Worker"
}
# The virtual private cloud (VPC).
resource "alicloud_vpc" "default" {
  vpc_name   = var.name
  cidr_block = "10.1.0.0/21"
}
# The vSwitch.
resource "alicloud_vswitch" "default" {
  vswitch_name = var.name
  vpc_id       = alicloud_vpc.default.id
  cidr_block   = "10.1.1.0/24"
  zone_id      = data.alicloud_zones.default.zones[0].id
}

# The managed Kubernetes cluster.
resource "alicloud_cs_managed_kubernetes" "default" {
  worker_vswitch_ids = [alicloud_vswitch.default.id]
  # The prefix of the Kubernetes cluster name. It is mutually exclusive with the name argument. If the name_prefix argument is specified, Terraform uses it to create a unique cluster name. Default value: Terraform-Creation. 
  name_prefix = var.name
  # Specify whether to create a NAT gateway when the Kubernetes cluster is created. Default value: true. 
  new_nat_gateway = true
  # The pod CIDR block. If you set cluster_network_type to flannel, this parameter is required. The pod CIDR block cannot be the same as the VPC CIDR block or the CIDR blocks of other Kubernetes clusters in the VPC. You cannot change the pod CIDR block after the cluster is created. Maximum number of hosts in the cluster: 256. 
  pod_cidr = "172.20.0.0/16"
  # The Service CIDR block. The Service CIDR block cannot be the same as the VPC CIDR block or the CIDR blocks of other Kubernetes clusters in the VPC. You cannot change the Service CIDR block after the cluster is created. 
  service_cidr = "172.21.0.0/20"
  # Specify whether to create an Internet-facing Server Load Balancer (SLB) instance for the API server of the cluster. Default value: false. 
  slb_internet_enabled = true
}

resource "alicloud_cs_kubernetes_node_pool" "default" {
  name         = var.name
  cluster_id   = alicloud_cs_managed_kubernetes.default.id
  vswitch_ids  = [alicloud_vswitch.default.id]
  # The password that is used to log on to a cluster node over SSH. You must specify one of the password, key_name, and kms_encrypted_password fields. 
  password = "Yourpassword1234"
  # The number of worker nodes in the cluster. 
  desired_size = 2
  # Specify whether to install the CloudMonitor agent on the nodes in the cluster. 
  install_cloud_monitor = true
  # The ECS instance types of the worker nodes. Specify one type for a cluster deployed in a single zone or three types for a cluster deployed across multiple zones. You can use the instance_types data source to obtain available instance types for Kubernetes master nodes.
  instance_types        = ["ecs.n4.large"]
  # The type of system disk used by the nodes. Valid values: cloud_ssd and cloud_efficiency. Default value: cloud_efficiency. 
  system_disk_category  = "cloud_efficiency"
  system_disk_size      = 40
  data_disks {
    category = "cloud_ssd"
    size = "100"
  }
}

Save the preceding configuration as a main.tf description file. Run the terraform init and terraform apply commands in the current directory of the file.

Run the terraform init command to initialize Terraform.

$ terraform init                                                                    

Initializing the backend...

Initializing provider plugins...
- Finding latest version of aliyun/alicloud...
- Installing aliyun/alicloud v1.214.1...
- Installed aliyun/alicloud v1.214.1 (verified checksum)

Terraform has created a lock file .terraform.lock.hcl to record the provider
selections it made above. Include this file in your version control repository
so that Terraform can guarantee to make the same selections by default when
you run "terraform init" in the future.

╷
│ Warning: Incomplete lock file information for providers
│ 
│ Due to your customized provider installation methods, Terraform was forced to calculate lock file checksums locally for the following providers:
│   - aliyun/alicloud
│ 
│ The current .terraform.lock.hcl file only includes checksums for darwin_amd64, so Terraform running on another platform will fail to install these providers.
│ 
│ To calculate additional checksums for another platform, run:
│   terraform providers lock -platform=linux_amd64
│ (where linux_amd64 is the platform to generate)
╵

Terraform has been successfully initialized!

You may now begin working with Terraform. Try running "terraform plan" to see
any changes that are required for your infrastructure. All Terraform commands
should now work.

If you ever set or change modules or backend configuration for Terraform,
rerun this command to reinitialize your working directory. If you forget, other
commands will detect it and remind you to do so if necessary.

Run the terraform apply command to create resources.

$ terraform apply  

data.alicloud_zones.default: Reading...
data.alicloud_zones.default: Read complete after 1s [id=2604238681]
data.alicloud_instance_types.default: Reading...
data.alicloud_instance_types.default: Read complete after 1s [id=1017980362]

Terraform used the selected providers to generate the following execution plan. Resource actions are indicated with the following symbols:
  + create

Terraform will perform the following actions:

  # alicloud_cs_kubernetes_node_pool.default will be created
  + resource "alicloud_cs_kubernetes_node_pool" "default" {
      + cluster_id                 = (known after apply)
      + deployment_set_id          = (known after apply)
      + desired_size               = 2
      + format_disk                = (known after apply)
      + id                         = (known after apply)
      + image_id                   = (known after apply)
      + image_type                 = (known after apply)
      + install_cloud_monitor      = true
      + instance_charge_type       = "PostPaid"
      + instance_types             = [
          + "ecs.n4.large",
        ]
      + internet_charge_type       = (known after apply)
      + internet_max_bandwidth_out = (known after apply)
      + keep_instance_name         = (known after apply)
      + name                       = "my-first-kubernetes-demo"
      + node_count                 = (known after apply)
      + node_name_mode             = (known after apply)
      + password                   = (sensitive value)
      + platform                   = (known after apply)
      + resource_group_id          = (known after apply)
      + runtime_name               = (known after apply)
      + runtime_version            = (known after apply)
      + scaling_group_id           = (known after apply)
      + scaling_policy             = (known after apply)
      + security_group_id          = (known after apply)
      + security_group_ids         = (known after apply)
      + spot_strategy              = (known after apply)
      + system_disk_category       = "cloud_efficiency"
      + system_disk_size           = 40
      + unschedulable              = false
      + vpc_id                     = (known after apply)
      + vswitch_ids                = (known after apply)

      + data_disks {
          + category = "cloud_ssd"
          + size     = 100
        }
    }

  # alicloud_cs_managed_kubernetes.default will be created
  + resource "alicloud_cs_managed_kubernetes" "default" {
      + availability_zone            = (known after apply)
      + certificate_authority        = (known after apply)
      + cluster_domain               = "cluster.local"
      + cluster_spec                 = (known after apply)
      + connections                  = (known after apply)
      + control_plane_log_project    = (known after apply)
      + control_plane_log_ttl        = (known after apply)
      + deletion_protection          = false
      + id                           = (known after apply)
      + install_cloud_monitor        = (known after apply)
      + is_enterprise_security_group = (known after apply)
      + load_balancer_spec           = "slb.s1.small"
      + name                         = (known after apply)
      + name_prefix                  = "my-first-kubernetes-demo"
      + nat_gateway_id               = (known after apply)
      + new_nat_gateway              = true
      + node_cidr_mask               = 24
      + node_port_range              = (known after apply)
      + os_type                      = "Linux"
      + platform                     = (known after apply)
      + pod_cidr                     = "172.20.0.0/16"
      + proxy_mode                   = "ipvs"
      + resource_group_id            = (known after apply)
      + rrsa_metadata                = (known after apply)
      + security_group_id            = (known after apply)
      + service_cidr                 = "172.21.0.0/20"
      + slb_id                       = (known after apply)
      + slb_internet                 = (known after apply)
      + slb_internet_enabled         = true
      + slb_intranet                 = (known after apply)
      + version                      = (known after apply)
      + vpc_id                       = (known after apply)
      + worker_auto_renew_period     = (known after apply)
      + worker_disk_size             = (known after apply)
      + worker_instance_charge_type  = (known after apply)
      + worker_period                = (known after apply)
      + worker_period_unit           = (known after apply)
      + worker_ram_role_name         = (known after apply)
      + worker_vswitch_ids           = (known after apply)
    }

  # alicloud_vpc.default will be created
  + resource "alicloud_vpc" "default" {
      + cidr_block            = "10.1.0.0/21"
      + create_time           = (known after apply)
      + id                    = (known after apply)
      + ipv6_cidr_block       = (known after apply)
      + ipv6_cidr_blocks      = (known after apply)
      + name                  = (known after apply)
      + resource_group_id     = (known after apply)
      + route_table_id        = (known after apply)
      + router_id             = (known after apply)
      + router_table_id       = (known after apply)
      + secondary_cidr_blocks = (known after apply)
      + status                = (known after apply)
      + user_cidrs            = (known after apply)
      + vpc_name              = "my-first-kubernetes-demo"
    }

  # alicloud_vswitch.default will be created
  + resource "alicloud_vswitch" "default" {
      + availability_zone    = (known after apply)
      + cidr_block           = "10.1.1.0/24"
      + create_time          = (known after apply)
      + id                   = (known after apply)
      + ipv6_cidr_block      = (known after apply)
      + ipv6_cidr_block_mask = (known after apply)
      + name                 = (known after apply)
      + status               = (known after apply)
      + vpc_id               = (known after apply)
      + vswitch_name         = "my-first-kubernetes-demo"
      + zone_id              = "cn-zhangjiakou-a"
    }

Plan: 4 to add, 0 to change, 0 to destroy.

Do you want to perform these actions?
  Terraform will perform the actions described above.
  Only 'yes' will be accepted to approve.

  Enter a value:

The terraform init command downloads required provider plug-ins. The terraform apply command figures out the operations to be performed based on the main.tf file. The preceding log shows that a resource named alicloud_cs_managed_kubernetes.default is to be created. You must enter yes to confirm the operation. After you confirm the operation, it takes about 5 minutes to create the resource. Terraform then generates log entries similar to the following ones:

Do you want to perform these actions?
  Terraform will perform the actions described above.
  Only 'yes' will be accepted to approve.

  Enter a value: yes

alicloud_vpc.default: Creating...
alicloud_vpc.default: Creation complete after 4s [id=vpc-8vbkpc7n9gp5mft7kxh7t]
alicloud_vswitch.default: Creating...
alicloud_vswitch.default: Creation complete after 3s [id=vsw-8vbkdhovthzlwirs4et9c]
alicloud_cs_managed_kubernetes.default: Creating...
alicloud_cs_managed_kubernetes.default: Still creating... [10s elapsed]
......
alicloud_cs_managed_kubernetes.default: Still creating... [3m40s elapsed]
alicloud_cs_managed_kubernetes.default: Creation complete after 3m42s [id=cfd0a48c499804b94b59a4f6da963f6d5]
alicloud_cs_kubernetes_node_pool.default: Creating...
alicloud_cs_kubernetes_node_pool.default: Still creating... [10s elapsed]
alicloud_cs_kubernetes_node_pool.default: Still creating... [20s elapsed]
alicloud_cs_kubernetes_node_pool.default: Still creating... [30s elapsed]
alicloud_cs_kubernetes_node_pool.default: Creation complete after 33s [id=cfd0a48c499804b94b59a4f6da963f6d5:np378764a2c81d4a8eb85bad53cf3ccf5c]

Apply complete! Resources: 4 added, 0 changed, 0 destroyed.

When the Apply complete! Resources: 4 added message appears, the cluster is created. You can also log on to the console to view this cluster.

Modify a managed Kubernetes cluster

Terraform providers provide the ability to modify specific arguments. In most cases, arguments of resources that are not marked ForceNew can all be modified.

The following code shows a template in which specific arguments are modified:

provider "alicloud" {
    region = "cn-zhangjiakou"
}
# The default resource name.
variable "name" {
  default = "my-first-kubernetes-demo"
}
# The name of the Simple Log Service project.
variable "log_project_name" {
  default = "my-first-kubernetes-sls-demo"
}
# The zone.
data "alicloud_zones" "default" {
  available_resource_creation = "VSwitch"
}
# The ECS instance specifications of the worker nodes.
data "alicloud_instance_types" "default" {
  availability_zone    = data.alicloud_zones.default.zones[0].id
  cpu_core_count       = 2
  memory_size          = 4
  kubernetes_node_role = "Worker"
}
#The VPC.
resource "alicloud_vpc" "default" {
  vpc_name   = var.name
  cidr_block = "10.1.0.0/21"
}
# The vSwitch.
resource "alicloud_vswitch" "default" {
  vswitch_name = var.name
  vpc_id       = alicloud_vpc.default.id
  cidr_block   = "10.1.1.0/24"
  zone_id      = data.alicloud_zones.default.zones[0].id
}

# The managed Kubernetes cluster.
resource "alicloud_cs_managed_kubernetes" "default" {
  worker_vswitch_ids = [alicloud_vswitch.default.id]
  # The prefix of the Kubernetes cluster name. It is mutually exclusive with the name argument. If the name_prefix argument is specified, Terraform uses it to create a unique cluster name. Default value: Terraform-Creation. 
  name_prefix = var.name
  # Specify whether to create a NAT gateway when the Kubernetes cluster is created. Default value: true. 
  new_nat_gateway = true
  # The pod CIDR block. If you set cluster_network_type to flannel, this parameter is required. The pod CIDR block cannot be the same as the VPC CIDR block or the CIDR blocks of other Kubernetes clusters in the VPC. You cannot change the pod CIDR block after the cluster is created. Maximum number of hosts in the cluster: 256. 
  pod_cidr = "172.20.0.0/16"
  # The Service CIDR block. The Service CIDR block cannot be the same as the VPC CIDR block or the CIDR blocks of other Kubernetes clusters in the VPC. You cannot change the Service CIDR block after the cluster is created. 
  service_cidr = "172.21.0.0/20"
  # Specify whether to create an Internet-facing SLB instance for the API server of the cluster. Default value: false. 
  slb_internet_enabled = true
  
  # Export the certificate related files of the cluster to the /tmp directory.
  client_cert = "/tmp/client-cert.pem"
  client_key = "/tmp/client-key.pem"
  cluster_ca_cert = "/tmp/cluster-ca-cert.pem"
}

resource "alicloud_cs_kubernetes_node_pool" "default" {
  name         = var.name
  cluster_id   = alicloud_cs_managed_kubernetes.default.id
  vswitch_ids  = [alicloud_vswitch.default.id]
  # The password that is used to log on to a cluster node over SSH. You must specify one of the password, key_name, and kms_encrypted_password fields. 
  password = "Yourpassword1234"
  # The number of worker nodes in the cluster. 
  desired_size = 3
  # Specify whether to install the CloudMonitor agent on the nodes in the cluster. 
  install_cloud_monitor = true
  # The ECS instance types of the worker nodes. Specify one type for a cluster deployed in a single zone or three types for a cluster deployed across multiple zones. You can use the instance_types data source to obtain available instance types for Kubernetes master nodes.
  instance_types        = ["ecs.n4.large"]
  # The type of system disk used by the nodes. Valid values: cloud_ssd and cloud_efficiency. Default value: cloud_efficiency. 
  system_disk_category  = "cloud_efficiency"
  system_disk_size      = 40
  data_disks {
    category = "cloud_ssd"
    size = "100"
  }
}

data "alicloud_cs_cluster_credential" "auth" {
  cluster_id                 = alicloud_cs_managed_kubernetes.default.id
  temporary_duration_minutes = 60
  output_file = "/tmp/config"
}

Similarly, you can run the terraform apply command to modify a cluster. After the command is run, the following log output is displayed. Type yes and press the Enter key to change the cluster name to test-managed-kubernetes-updated, scale out the number of worker nodes to 3, and export the certificate and connection files to the local /tmp directory.

terraform apply
data.alicloud_zones.default: Reading...
alicloud_vpc.default: Refreshing state... [id=vpc-8vbr6t6i2xl49hjzald45]
data.alicloud_zones.default: Read complete after 0s [id=2604238681]
data.alicloud_instance_types.default: Reading...
alicloud_vswitch.default: Refreshing state... [id=vsw-8vbkp6rcqkn4ljf1a7tb3]
alicloud_cs_managed_kubernetes.default: Refreshing state... [id=cdfe383b2114c40f582270860c39cb3cb]
data.alicloud_instance_types.default: Read complete after 1s [id=3527274229]
alicloud_cs_kubernetes_node_pool.default: Refreshing state... [id=cdfe383b2114c40f582270860c39cb3cb:npf17c80f735d645e88b4ea61b689e15b8]

Terraform used the selected providers to generate the following execution plan. Resource actions are indicated with the following symbols:
  ~ update in-place
 <= read (data resources)

Terraform will perform the following actions:

  # data.alicloud_cs_cluster_credential.auth will be read during apply
  # (depends on a resource or a module with changes pending)
 <= data "alicloud_cs_cluster_credential" "auth" {
      + certificate_authority      = (known after apply)
      + cluster_id                 = "cdfe383b2114c40f582270860c39cb3cb"
      + cluster_name               = (known after apply)
      + expiration                 = (known after apply)
      + id                         = (known after apply)
      + kube_config                = (sensitive value)
      + output_file                = "/tmp/config"
      + temporary_duration_minutes = 60
    }

  # alicloud_cs_kubernetes_node_pool.default will be updated in-place
  ~ resource "alicloud_cs_kubernetes_node_pool" "default" {
      ~ desired_size               = 2 -> 3
        id                         = "cdfe383b2114c40f582270860c39cb3cb:npf17c80f735d645e88b4ea61b689e15b8"
      ~ instance_types             = [
          - "ecs.n1.medium",
          + "ecs.sn1.medium",
        ]
        name                       = "my-first-kubernetes-demo"
        tags                       = {}
        # (26 unchanged attributes hidden)

        # (1 unchanged block hidden)
    }

  # alicloud_cs_managed_kubernetes.default will be updated in-place
  ~ resource "alicloud_cs_managed_kubernetes" "default" {
      + client_cert                  = "/tmp/client-cert.pem"
      + client_key                   = "/tmp/client-key.pem"
      + cluster_ca_cert              = "/tmp/cluster-ca-cert.pem"
        id                           = "cdfe383b2114c40f582270860c39cb3cb"
        name                         = "my-first-kubernetes-demo20240116105632726000000002"
        tags                         = {}
        # (28 unchanged attributes hidden)

        # (1 unchanged block hidden)
    }

Plan: 0 to add, 2 to change, 0 to destroy.

Do you want to perform these actions?
  Terraform will perform the actions described above.
  Only 'yes' will be accepted to approve.

  Enter a value: yes

alicloud_cs_managed_kubernetes.default: Modifying... [id=cdfe383b2114c40f582270860c39cb3cb]
alicloud_cs_managed_kubernetes.default: Modifications complete after 3s [id=cdfe383b2114c40f582270860c39cb3cb]
data.alicloud_cs_cluster_credential.auth: Reading...
alicloud_cs_kubernetes_node_pool.default: Modifying... [id=cdfe383b2114c40f582270860c39cb3cb:npf17c80f735d645e88b4ea61b689e15b8]
data.alicloud_cs_cluster_credential.auth: Read complete after 0s [id=87210520]
alicloud_cs_kubernetes_node_pool.default: Still modifying... [id=cdfe383b2114c40f582270860c39cb3cb:npf17c80f735d645e88b4ea61b689e15b8, 10s elapsed]
alicloud_cs_kubernetes_node_pool.default: Still modifying... [id=cdfe383b2114c40f582270860c39cb3cb:npf17c80f735d645e88b4ea61b689e15b8, 20s elapsed]
alicloud_cs_kubernetes_node_pool.default: Still modifying... [id=cdfe383b2114c40f582270860c39cb3cb:npf17c80f735d645e88b4ea61b689e15b8, 30s elapsed]
alicloud_cs_kubernetes_node_pool.default: Modifications complete after 35s [id=cdfe383b2114c40f582270860c39cb3cb:npf17c80f735d645e88b4ea61b689e15b8]

Apply complete! Resources: 0 added, 2 changed, 0 destroyed.

After the terraform apply command is run, the cluster information displayed in the console indicates that the cluster has reached the desired state. On the local machine, you can use the exported kubectl configuration file to connect to the created cluster.