Manage cluster add-ons using Terraform - Container Service for Kubernetes

Container Service for Kubernetes (ACK) offers a rich set of add-ons to extend cluster capabilities. This topic describes how to configure add-ons in Terraform to manage your workloads across various scenarios.

Add-on types

ACK manages two types of cluster add-ons: system add-ons and optional add-ons. For more information about add-ons, see Add-ons.

System add-ons

System add-ons are essential components required to run an ACK cluster. They are installed by default when you create a cluster. Examples include the following:

kube-apiserver
kube-controller-manager
cloud-controller-manager
kube-proxy
CoreDNS

Optional add-ons

Optional add-ons are non-essential components provided by ACK that you can choose to install to extend cluster functionality. These fall into categories such as application management, logging and monitoring, storage, networking, and security.

Generate Terraform parameters for managing add-ons using the console

You can use the console to generate Terraform parameter configurations for add-ons. This improves the efficiency and usability of add-on management.

Log on to the ACK console. In the left navigation pane, click Clusters.
On the Clusters page, find the one you want to manage and click its name. In the left navigation pane, click Add-ons.
On the Add-ons page, search for and locate your target add-on. On the add-on card, click Install or Configure as needed.
In the dialog box that appears, click Equivalent Code. In the panel on the right, click the Terraform tab to view the relevant parameters for installing or configuring the add-on. You can then copy and use these parameters.

Add-on management practices

You can use Terraform to specify which add-ons to install during cluster creation and perform full lifecycle management after the cluster is created. The following sections describe how to manage the lifecycle of cluster add-ons and provide best practices for common scenarios.

Specify add-ons to install during cluster creation

You can specify add-ons to install when you create a cluster. The following resources are involved:

Managed Kubernetes cluster: alicloud_cs_managed_kubernetes
ACK Dedicated cluster: alicloud_cs_kubernetes
ACK Edge cluster: alicloud_cs_edge_kubernetes
ACK serverless cluster: alicloud_cs_serverless_kubernetes

All the preceding resources support defining add-ons to install during cluster creation using the addons attribute. The addons attribute is defined as follows:

# Example for a managed Kubernetes cluster.
resource "alicloud_cs_managed_kubernetes" "default" {
  # Other parameters.
  # ...

  # addons is a list structure. Define the addons attribute in the resource to install the add-on during cluster creation.
  addons {
    # The name of the add-on. You can query available and installed add-ons and their versions using the alicloud_cs_kubernetes_addons data source.
    name = "XXX"

    # Custom parameters for the add-on. Some add-ons support custom configuration. Use this field to specify custom parameters. For details, see the section "Modify custom configuration parameters for cluster add-ons."
    config = jsonencode(
      {
        ....
      }
    )

    # This Boolean parameter defaults to false. ACK installs certain add-ons by default to simplify cluster management. To skip installing these during cluster creation, set disabled = true.
    disabled = XXX
  }
}

Important

You can install add-ons by specifying them in the addons block of a cluster resource only during cluster creation. After the cluster is created, you cannot manage the add-on lifecycle, such as by upgrading, uninstalling, or updating configurations, by modifying the addons block. To manage the lifecycle of add-ons after cluster creation, see Manage the lifecycle of add-ons after a cluster is created.

The following table lists how to configure add-ons in ACK.

Add-on name	Add-on type	Description	Terraform configuration
appcenter	Application management	Provides a unified application center for managing multi-cluster application deployments and application lifecycles.	`addon { name = "appcenter" }`
progressive-delivery-tool	Application management	Enables progressive phased releases for applications.	`addon { name = "progressive-delivery-tool" }`
alicloud-monitor-controller	Logging and monitoring	Integrates ACK with Cloud Monitor.	`addon { name = "alicloud-monitor-controller" }`
metrics-server	Logging and monitoring	An enhanced version of the open source community metrics component. It collects monitoring data and provides Metrics API for data consumption and Horizontal Pod Autoscaler (HPA) capabilities.	`addon { name = "metrics-server" }`
ack-node-problem-detector	Logging and monitoring	An enhanced version of the open source community project for monitoring anomalous node events and integrating with third-party monitoring platforms.	`addons { name = "ack-node-problem-detector" }`
ags-metrics-collector	Logging and monitoring	A monitoring service component for genomics customers. It monitors detailed resource usage of nodes in genomics workflows.	`addons { name = "ags-metrics-collector" }`
ack-arms-prometheus	Logging and monitoring	Uses Alibaba Cloud Prometheus to monitor ACK clusters.	`addons { name = "arms-prometheus" }`
loongcollector	Logging and monitoring	Uses Simple Log Service (SLS) to collect Kubernetes container logs.	`addons { name = "loongcollector" }`
csi-plugin	Storage	Supports mounting and unmounting volumes. This add-on is installed by default if you select the CSI plug-in to enable Alibaba Cloud storage integration during cluster creation.	`addons { name = "csi-plugin" }`
csi-provisioner	Storage	Supports automatic volume creation. This add-on is installed by default if you select the CSI plug-in to enable Alibaba Cloud storage integration during cluster creation.	`addons { name = "csi-plugin" }`
storage-operator	Storage	Manages the lifecycle of storage add-ons.	`addons { name = "storage-operator" }`
alicloud-disk-controller	Storage	Supports automatic creation of cloud disk volumes.	`addons { name = "alicloud-disk-controller" }`
flexvolume	Storage	FlexVolume is an early volume extension mechanism from the Kubernetes community that supports mounting and unmounting volumes. This component is installed by default when you create a cluster and select the FlexVolume plugin to connect to Alibaba Cloud storage.	`addons { name = "flexvolume" }`
nginx-ingress-controller	Network Component	The Nginx Ingress Controller parses Ingress forwarding rules. When it receives a request, it matches the Ingress rule and forwards traffic to the backend service.	`addons { name = "nginx-ingress-controller" }`
terway-eniip	Network components	An open source Alibaba Cloud CNI plug-in based on virtual private cloud (VPC). It uses standard Kubernetes network policies to define access rules between containers. You can use Terway to enable internal communication within a Kubernetes cluster. This add-on is installed by default if you select the Terway network plug-in during cluster creation.	`addons { name = "terway-eniip" }`
ack-node-local-dns	Networking	A DNS local caching solution based on the open source NodeLocal DNSCache project.	`addons { name = "ack-node-local-dns" }`
aliyun-acr-credential-helper	Security	Enables passwordless pulling of private images from ACR Default or Enterprise Edition in an ACK cluster.	`addons { name = "aliyun-acr-credential-helper" }`
gatekeeper	Security	Helps manage and apply Open Policy Agent (OPA) policies in the cluster, such as namespace label management.	`addons { name = "gatekeeper" }`
kritis-validation-hook	Security	A key component for verifying container image signatures in trusted container deployment.	`addons { name = "kritis-validation-hook" }`
security-inspector	Security	A key component for security inspection.	`addons { name = "security-inspector" }`
ack-kubernetes-webhook-injector	Security	A Kubernetes component that dynamically adds or removes pod IPs from Alibaba Cloud product whitelists, eliminating manual configuration.	`addons { name = "ack-kubernetes-webhook-injector" }`
ack-arena	Other	Simplifies the installation of open source Arena, enabling one-click installation from the console.	`addons { name = "ack-arena" }`
ack-cost-exporter	Other	A plug-in for data processing in ACK cost analysis.	`addons { name = "ack-cost-exporter" }`
ack-kubernetes-cronhpa-controller	Other	Enables scheduled scaling for application workloads.	`addons { name = "ack-kubernetes-cronhpa-controller" }`
ack-virtual-node	Other	Built on the open source Virtual Kubelet project with extended support for Aliyun Provider and extensive optimizations to seamlessly connect Kubernetes with Elastic Container Instance (ECI).	`addons { name = "ack-virtual-node" }`
aesm	Other	Intel® SGX Architectural Enclave Service Manager (Intel® SGX AESM) is a system component for Intel® SGX that provides enclave launch support, key provisioning, and remote attestation services.	`addons { name = "aesm" }`
aliyun-acr-acceleration-suite	Other	A client-side plug-in that provides on-demand image loading acceleration. It is deployed as a DaemonSet on worker nodes.	`addons { name = "aliyun-acr-acceleration-suite" }`
migrate-controller	Other	A Kubernetes application migration component developed based on the open source Velero project.	`addons { name = "migrate-controller" }`
resource-controller	Other	A key component for dynamic pod resource control. Install this add-on to use CPU topology-aware scheduling in ACK Pro clusters.	`addons { name = "resource-controller" }`
sandboxed-container-controller	Other	A dedicated controller for sandboxed container runtime that enhances and extends basic sandboxed container functionality.	`addons { name = "sandboxed-container-controller" }`
sandboxed-container-helper	Other	A component for diagnostics and operations management of sandboxed containers.	`addons { name = "sandboxed-container-helper" }`
sgx-device-plugin	Other	A Kubernetes Device Plugin jointly developed by the Alibaba Cloud Container Service team and Ant Financial secure computing team for Intel SGX. It simplifies using SGX in containers.	`addons { name = "sgx-device-plugin" }`

Manage add-on lifecycles after cluster creation

To manage the lifecycle of add-ons, you must have a Kubernetes cluster. If you do not have one, you must create one first.

For add-ons in a cluster, you can use the alicloud_cs_kubernetes_addon resource to manage their lifecycle, including installation, upgrades, uninstallation, and custom configuration updates. The attributes and definition of alicloud_cs_kubernetes_addon are as follows:

resource "alicloud_cs_kubernetes_addon" "addon-example" {
  # Cluster ID.
  cluster_id = "XXXX"

  # Add-on name. Use the alicloud_cs_kubernetes_addons data source to query all installed and installable add-ons and their versions for the current cluster.
  name = "XXXX"

  # Add-on version.
  version = "XXXX"

  # Custom parameters for the add-on, formatted as a JSON string. Use Terraform's built-in jsonencode function or provide a JSON string directly (with proper escaping). Some add-ons support custom parameters. For details, see the section "Modify custom configuration parameters for cluster add-ons."
  config = jsonencode(
    {
      ....
    }
  )
}

You can configure custom parameters by providing a JSON string directly, but you must escape special characters. For example, the nginx-ingress-controller add-on supports the following two configuration methods:

Configure parameters using jsonencode:

config = jsonencode(   
  {       
    IngressSlbNetworkType="internet"       
    IngressSlbSpec="slb.s2.small"     
  }  
)

Configure parameters using a direct string:

config = "{\"IngressSlbNetworkType\":\"internet\",\"IngressSlbSpec\":\"slb.s2.small\"}"

Import existing cluster add-ons into Terraform management

For add-ons that are already installed in a cluster, you can use terraform import to bring them under Terraform management. The following example shows how to import the nginx-ingress-controller add-on.

Create a new file with a .tf extension, or use an existing .tf file, and define a resource.
The alicloud_cs_kubernetes_addon resource manages cluster add-ons. Leave it empty for now.
```
resource "alicloud_cs_kubernetes_addon" "nginx-ingress-controller" {
}
```
Run the following command to import the nginx-ingress-controller add-on that is already installed in the cluster.
Terraform pulls the add-on configuration from the cluster and writes it to a file with a .state extension.
```
terraform import alicloud_cs_kubernetes_addon.nginx-ingress-controller <cluster_id>:nginx-ingress-controller
```
Run the terraform plan command and review the output to see the differences between the nginx-ingress-controller configuration in the cluster and your defined resource.
Based on the differences and the content of the .state file, update the resource information that you defined in step 1. Repeat this process until running terraform plan shows no differences between your local configuration and the add-on configuration in the cluster. At that point, the import is complete.
```
resource "alicloud_cs_kubernetes_addon" "nginx-ingress-controller" {
  cluster_id = "XXXXX"
  name = "nginx-ingress-controller"
  version = "v1.2.1-aliyun.1"
  config = jsonencode(
    {
      IngressSlbNetworkType = "internet"
      IngressSlbSpec        = "slb.s2.small"
    }
  )
}
```

Install cluster add-ons

You can use the alicloud_cs_kubernetes_addon resource to install add-ons in an existing cluster. The following example uses the gatekeeper add-on.

Define the add-on to install in a .tf file. Specify the following information:
- Cluster ID.
- Add-on name and version:
  You can query available add-on names and versions using the alicloud_cs_kubernetes_addons data source. This returns only the latest installable version for each add-on. To install a historical version, check the release notes of the add-on and specify the corresponding version number.
- (Optional) Custom configuration:
  Modify the config field to customize the add-on. You can use the built-in jsonencode function of Terraform to build your configuration. You can query configurable parameters using the alicloud_cs_kubernetes_addon_metadata data source. For more information, see Modify custom configuration parameters for cluster add-ons.
Expand to view details
```
resource "alicloud_cs_kubernetes_addon" "gatekeeper" {
  cluster_id = "ce36b7c61e126430b8b245730ca6d****"
  name = "gatekeeper"
  version = "v3.8.1.113-geb7947ef-aliyun"
  config = jsonencode(
    {
      AdmissionPodCpuLimit      = "1000m"
      AdmissionPodCpuRequest    = "100m"
      AdmissionPodMemoryLimit   = "512Mi"
      AdmissionPodMemoryRequest = "256Mi"
      AdmissionPodNumber        = 3
      AuditInterval             = 1800
      AuditPodCpuLimit          = "1000m"
      AuditPodCpuRequest        = "100m"
      AuditPodMemoryLimit       = "512Mi"
      AuditPodMemoryRequest     = "256Mi"
      EnableAuditPod            = false
      EnableMutatingWebhook     = false
    }
  )
}
```

Run the following command to install the add-on in the cluster.

terraform apply

Expected output:

Plan: 1 to add, 0 to change, 0 to destroy.

Do you want to perform these actions?
  Terraform will perform the actions described above.
  Only 'yes' will be accepted to approve.

  Enter a value: yes

alicloud_cs_kubernetes_addon.gatekeeper: Creating...
alicloud_cs_kubernetes_addon.gatekeeper: Still creating... [10s elapsed]
alicloud_cs_kubernetes_addon.gatekeeper: Creation complete after 16s [id=XXXXX:gatekeeper]

Apply complete! Resources: 1 added, 0 changed, 0 destroyed.

When you see Apply complete!, the add-on installation is complete.

Upgrade cluster add-ons

You can use the alicloud_cs_kubernetes_addons data source to query the available upgrade versions for an add-on. If a new version is available, you can upgrade the add-on by changing the version number. The following example uses the gatekeeper add-on.

Expand to view details

resource "alicloud_cs_kubernetes_addon" "gatekeeper" {
  cluster_id = "ce36b7c61e126430b8b245730ca6d****"
  name = "gatekeeper"

  # Change version to the desired upgrade version.
  version = "XXXXXXXXX"
  config = jsonencode(
    {
      AdmissionPodCpuLimit      = "1000m"
      AdmissionPodCpuRequest    = "100m"
      AdmissionPodMemoryLimit   = "512Mi"
      AdmissionPodMemoryRequest = "256Mi"
      AdmissionPodNumber        = 3
      AuditInterval             = 1800
      AuditPodCpuLimit          = "1000m"
      AuditPodCpuRequest        = "100m"
      AuditPodMemoryLimit       = "512Mi"
      AuditPodMemoryRequest     = "256Mi"
      EnableAuditPod            = false
      EnableMutatingWebhook     = false
    }
  )
}

Run terraform apply to upgrade the add-on. A success message indicates that the upgrade is complete.

Modify custom configuration parameters for cluster add-ons

Some ACK add-ons support custom parameter configuration. You can use the alicloud_cs_kubernetes_addons resource to update your add-on configuration. For the gatekeeper add-on, you can modify the config field to change the settings.

Expand to view details

resource "alicloud_cs_kubernetes_addon" "gatekeeper" {
  cluster_id = "ce36b7c61e126430b8b245730ca6d****"
  name = "gatekeeper"
  version = "v3.8.1.113-geb7947ef-aliyun"

  # Modify properties in config and apply to update the cluster add-on configuration.

}

To view all configurable parameters for an add-on, you can query them using the alicloud_cs_kubernetes_addon_metadata data source. The response is in the JSON Schema format. For the gatekeeper add-on, add the following content to your .tf file:

# Define a data source to get the configurable parameter schema for the gatekeeper add-on.
data "alicloud_cs_kubernetes_addon_metadata" "default" {
  cluster_id = "ce36b7c61e126430b8b245730ca6d****"
  name       = "gatekeeper"
  version    = "v3.8.1.113-geb7947ef-aliyun"
}

# Output the result.
output "addon_config_schema" {
  value = data.alicloud_cs_kubernetes_addon_metadata.default.config_schema
}

Run terraform apply. The output is a JSON Schema in which the properties field defines all supported parameters. Based on this schema, you can specify any supported configuration parameter. The parameter descriptions include the following:

default: The default value.
description: The parameter description.
pattern: The regular expression that defines allowed value formats.
type: The field type.

Expand to view details

addon_config_schema = <<EOT
{
  "$schema": "http://json-schema.org/draft-07/schema#",
  "properties": {
    "AdmissionPodCpuLimit": {
      "default": "1000m",
      "description": "cpu limit for gatekeeper",
      "pattern": "^(|[1-9][0-9]*(m|\\.\\d+)?)$",
      "type": "string"
    },
    "AdmissionPodCpuRequest": {
      "default": "100m",
      "description": "cpu request for gatekeeper",
      "pattern": "^[1-9][0-9]*(m|\\.\\d+)?$",
      "type": "string"
    },
    "AdmissionPodMemoryLimit": {
      "default": "512Mi",
      "description": "memory limit for gatekeeper",
      "pattern": "^(|[1-9][0-9]*(\\.\\d+)?(K|Ki|M|Mi|G|Gi|T|Ti)?)$",
      "type": "string"
    },

    ......
  },
  "title": "Config",
  "type": "object"
}
EOT

Configure networking add-ons

In ACK, you can use the Terway networking mode to implement container networking capabilities. For more information, see Networking.

The following examples show how to configure networking add-ons using Terraform.

Expand to view details

# Use Terway as the networking add-on with pod-exclusive ENI mode (default).
# In this mode, the number of pods per node is limited by the ECS ENI quota.

resource "alicloud_cs_managed_kubernetes" "default" {
  # Other parameters.
  # ...

  addons {
    name = "terway-eni"
  }
}

# Use Terway with IPVlan mode.
# Uses IPVlan + eBPF for shared ENI virtualization. Requires Alibaba Cloud Linux 2.
resource "alicloud_cs_managed_kubernetes" "default" {
  # Other parameters.
  # ...

  addons {
    name   =  "terway-eniip",
    config = "{\"IPVlan\":\"true\",\"NetworkPolicy\":\"false\"}"
  }
}

# Use Terway with IPVlan mode and enable NetworkPolicy support.
# Uses IPVlan + eBPF for shared ENI virtualization. Requires Alibaba Cloud Linux 2.
# Provides policy-based network control in IPVlan mode.
resource "alicloud_cs_managed_kubernetes" "default" {
  # Other parameters.
  # ...

  addons {
    name   =  "terway-eniip",
    config = "{\"IPVlan\":\"true\",\"NetworkPolicy\":\"true\"}"
  }
}

Configure storage add-ons

ACK provides storage add-ons that are based on FlexVolume and Container Storage Interface (CSI). FlexVolume is deprecated, and ACK primarily maintains CSI storage add-ons. If you do not specify any storage add-ons when you create a cluster with Terraform, CSI is installed by default. You can define storage add-ons in Terraform as follows:

Expand to view details

# When using CSI, include csi-plugin and csi-provisioner.
# To enable dynamic storage classes for default NAS file systems and CNFS, also install storage-operator.
resource "alicloud_cs_managed_kubernetes" "default" {
  # Other parameters.
  # ...

  addons {
    name = "csi-plugin"
  }
  addons {
    name = "csi-provisioner"
  }
  addons {
    name   = "storage-operator"
    config = "{\"CnfsOssEnable\":\"false\",\"CnfsNasEnable\":\"true\"}"
  }
}

Configure logging add-ons

The log collection add-on of ACK stores collected logs in Simple Log Service (SLS). It supports the following two log storage options:

Use an existing SLS project for log storage.
Automatically create a new SLS project for log storage during cluster creation.

The configuration differs between these scenarios. The following Terraform examples illustrate both scenarios:

Expand to view details

# Automatically create an SLS project for log storage.
resource "alicloud_cs_managed_kubernetes" "default" {
  # Other parameters.
  # ...

  addons {
    name = "loongcollector"
  }
}

# Automatically create an SLS project and enable the Ingress dashboard.
resource "alicloud_cs_managed_kubernetes" "default" {
  # Other parameters.
  # ...

  addons {
    name = "loongcollector"
    config = "{\"IngressDashboardEnabled\":\"true\"}"
  }
}


# Use an existing SLS project for log storage and enable the Ingress dashboard.
resource "alicloud_cs_managed_kubernetes" "default" {
  # Other parameters.
  # ...

  addons {
    name = "loongcollector"
    config = "{\"IngressDashboardEnabled\":\"true\",\"sls_project_name\":\"k8s-log-c55c35ff493df47b88783bea48827****\"}"
  }
}

# Install and configure ack-node-problem-detector for Event Hub.
# Automatically create an SLS project for Event Hub log storage.
resource "alicloud_cs_managed_kubernetes" "default" {
  # Other parameters.
  # ...

  addons {
    name   = "ack-node-problem-detector"
    config = "{\"sls_project_name\":\"\"}"
  }
}

# Install and configure ack-node-problem-detector for Event Hub.
# Use an existing SLS project for Event Hub log storage. This can share the same logstore as loongcollector.
resource "alicloud_cs_managed_kubernetes" "default" {
  # Other parameters.
  # ...

  addons {
    name   = "ack-node-problem-detector"
    config = "{\"sls_project_name\":\"k8s-log-c55c35ff493df47b88783bea48827****\"}"
  }
}

Configure monitoring add-ons

ACK provides monitoring add-ons, including the CloudMonitor agent for ECS nodes and Prometheus Service. You can install the CloudMonitor agent on ECS nodes by setting the install_cloud_monitor parameter in Terraform.

Expand to view details

# Install Prometheus monitoring.
resource "alicloud_cs_managed_kubernetes" "default" {
  # Other parameters.
  # ...
  addons {
    name = "arms-prometheus"
  }
}
# Install the CloudMonitor agent on ECS nodes.
resource "alicloud_cs_kubernetes_node_pool" "default" {
  # Other parameters.
  # ...
  install_cloud_monitor = true
}

Configure Ingress routing add-ons

ACK provides two traffic ingress solutions: Nginx Ingress and ALB Ingress.

Nginx Ingress: An optimized version of the community ingress-nginx that provides flexible and reliable routing for your Kubernetes cluster. For more information, see Overview of Nginx Ingress.
ALB Ingress: A fully managed, highly reliable ALB Ingress that provides flexible and reliable routing for your Kubernetes cluster. For more information, see Access services through ALB Ingress.

The following examples show how to configure routing add-ons using Terraform:

View details

# Use nginx-ingress-controller routing.
# For Internet-facing SLB, set IngressSlbNetworkType to "internet".
# For internal SLB, set IngressSlbNetworkType to "intranet".
resource "alicloud_cs_managed_kubernetes" "default" {
  # Other parameters.
  # ...

  addons {
    name   =  "nginx-ingress-controller",
    config = "{\"IngressSlbNetworkType\":\"internet\",\"IngressSlbSpec\":\"slb.s2.small\"}"
  }
}

# Use ALB Ingress routing.
resource "alicloud_cs_managed_kubernetes" "default" {
  # Other parameters.
  # ...

  addons {
    name   = "alb-ingress-controller",
    config = "{\"albIngress\":{\"CreateDefaultALBConfig\":false}}"  # Do not create by default.
#   config = "{\"albIngress\":{\"LoadBalancerId\":\"alb-vl8uiXXXXXxdr\",\"CreateDefaultALBConfig\":true}}" # Use an existing ALB instance.
#   config = "{\"albIngress\":{\"AddressType\":\"Internet\",\"ZoneMappings\":{\"cn-hangzhou-l\":[\"vsw-uf6XXXXXoyb4qe\"],\"cn-hangzhou-m\":[\"vsw-uf6XXXX0rlkiq\"]},\"CreateDefaultALBConfig\":true}}" # Create new ALB (select at least two zones).
  }
}

Disable default add-ons

To simplify cluster management, ACK installs certain add-ons by default. If you do not need an add-on during cluster creation, you can disable it by setting disabled = true. The following example disables nginx-ingress-controller:

# Prevent installation of nginx-ingress-controller.

resource "alicloud_cs_managed_kubernetes" "default" {
  # Other parameters.
  # ...

  addons {
    name     =  "nginx-ingress-controller",
    disabled = true
  }
}

Default add-ons installed when no add-ons are specified

If you do not specify any add-ons when you create a cluster, the following add-ons are installed by default.

Cluster type	Add-on type	Default add-on name	Description
ACK cluster	System components	kube-scheduler	Uses Kube Scheduler for cluster resource scheduling.
		cloud-controller-manager	Uses Cloud Controller Manager to create load balancers for K8s applications and manage node route entries.
		kube-apiserver	APIServer is the bus and ingress gateway for the K8s cluster.
		kube-controller-manager	KCM manages internal resources in the K8s cluster.
	Logging and monitoring	alicloud-monitor-controller	Monitors application container lifecycle and state changes.
	Logging and monitoring	metrics-server	Metrics Server provides resource monitoring metrics for application containers to support cluster autoscaling.
	Storage	csi-plugin	Uses csi-plugin for volume lifecycle management (recommended).
		csi-provisioner	Uses csi-provisioner for volume creation and deletion (recommended).
		storage-operator	Uses storage-operator for storage operations management (recommended).
	Networking	CoreDNS	Kubernetes cluster DNS server.
		Gateway API	Gateway API gateway resource model.
		terway-eniip	Terway network plug-in.
		nginx-ingress-controller (installed by default in Pro edition)	Ingress controller based on Nginx traffic forwarding.
ACK serverless cluster	System components	kube-scheduler	Uses Kube Scheduler for cluster resource scheduling.
		ack-virtual-node	Uses virtual nodes and ECI elasticity.
		cloud-controller-manager	Uses Cloud Controller Manager to create load balancers for K8s applications and manage node route entries.
		kube-apiserver	APIServer is the bus and ingress gateway for the K8s cluster.
		kube-controller-manager	KCM manages internal resources in the K8s cluster.
	Networking	CoreDNS	K8s cluster DNS server.
ACK Edge cluster	System	kube-scheduler	Uses Kube Scheduler for cluster resource scheduling
		cloud-controller-manager	Uses Cloud Controller Manager to create load balancers for K8s applications and manage node route entries.
		kube-apiserver	APIServer is the bus and ingress gateway for the K8s cluster.
		kube-controller-manager	KCM manages internal resources in the K8s cluster.
	Logging and monitoring	alicloud-monitor-controller	Monitors application container lifecycle and state changes.
	Logging and monitoring	metrics-server	Metrics Server provides resource monitoring metrics for application containers to support cluster autoscaling
	Networking	CoreDNS	Kubernetes cluster DNS server.
	Networking	terway-eniip	Terway network plug-in.
	Other	edge-controller-manager	-
		edge-tunnel-agent	Edge-tunnel uses a client-server architecture to build a reverse O&M channel between cloud and edge.
		edge-tunnel-server	Edge-tunnel uses a client-server architecture to build a reverse O&M channel between cloud and edge.
		yurt-app-manager	Uses yurt-app-manager to provide node pool and unitized deployment capabilities for ACK@Edge.

Common configuration examples

The following are common configuration examples:

Select Terway for networking.
Select either Container Storage Interface (CSI) or FlexVolume for storage. We recommend that you use CSI because FlexVolume is deprecated.
Select either Nginx Ingress or ALB Ingress for routing based on your business needs.
Install other add-ons as needed. You can freely combine them.

Example 1: No add-ons configured

# If no add-ons are configured during cluster creation, only default add-ons are installed.
# This is the minimal cluster configuration. Replace variables with your own values.
resource "alicloud_cs_managed_kubernetes" "default" {
  name                         = var.name
  cluster_spec                 = "ack.pro.small"
  is_enterprise_security_group = true
  pod_cidr                     = "172.20.0.0/16"
  service_cidr                 = "172.21.0.0/20"
  worker_vswitch_ids           = [var.vswitch_id]
}

Example 2: Use Terway networking

# Create a cluster with Terway networking.
# Use pod-exclusive ENI mode.

resource "alicloud_cs_managed_kubernetes" "default" {
  name                         = var.name
  cluster_spec                 = "ack.pro.small"
  is_enterprise_security_group = true
  pod_vswitch_ids              = [var.vswitch_id]
  service_cidr                 = "172.21.0.0/20"
  worker_vswitch_ids           = [var.vswitch_id]

  addons {
    name = "terway-eni"
  }
}

# Create a cluster with Terway networking.
# Use IPVlan mode and enable network policies.
resource "alicloud_cs_managed_kubernetes" "default" {
  name                         = var.name
  cluster_spec                 = "ack.pro.small"
  is_enterprise_security_group = true
  service_cidr                 = "172.21.0.0/20"
  pod_vswitch_ids              = [var.vswitch_id]
  worker_vswitch_ids           = [var.vswitch_id]

  addons {
    name   =  "terway-eniip"
    config = "{\"IPVlan\":\"true\",\"NetworkPolicy\":\"true\"}"
  }
}

Example 3: Generic template with Terway, CSI, and Nginx Ingress

# Generic template: Terway + CSI + nginx-ingress.

resource "alicloud_cs_managed_kubernetes" "default" {
  name                         = var.name
  cluster_spec                 = "ack.pro.small"
  is_enterprise_security_group = true
  service_cidr                 = "172.21.0.0/20"
  pod_vswitch_ids              = [var.vswitch_id]
  worker_vswitch_ids           = [var.vswitch_id]

  addons {
    name = "terway-eniip",
    config = "{\"IPVlan\":\"true\",\"NetworkPolicy\":\"false\"}"
  }
  addons {
    name = "csi-plugin"
  }
  addons {
    name = "csi-provisioner"
  }
  addons {
    name = "storage-operator"
    config = "{\"CnfsOssEnable\":\"false\",\"CnfsNasEnable\":\"true\"}"
  }
  addons {
    name = "loongcollector"
    config = "{\"IngressDashboardEnabled\":\"true\"}"
  }
  addons {
    name = "ack-node-problem-detector"
    config = "{\"sls_project_name\":\"\"}"
  }
  addons {
    name = "nginx-ingress-controller"
    config = "{\"IngressSlbNetworkType\":\"internet\",\"IngressSlbSpec\":\"slb.s2.small\"}"
  }
  addons {
    name = "ack-node-local-dns"
  }
  addons {
    name = "arms-prometheus"
  }
  addons {
    name = "alicloud-monitor-controller"
    config = "{\"group_contact_ids\":\"[10619]\"}"
  }
}