Manage add-ons with Terraform - Container Service for Kubernetes

Container Service for Kubernetes offers a rich set of add-ons to extend cluster capabilities. This topic explains how to use Terraform to configure add-ons to manage workloads across various scenarios.

Add-on types

Cluster add-ons in ACK fall into two types: system add-ons and optional add-ons. For more information, see Add-ons.

System add-ons

System add-ons are essential for an ACK cluster to operate. They are installed by default when you create a cluster. For example:

kube-apiserver
kube-controller-manager
cloud-controller-manager
kube-proxy
CoreDNS

Optional add-ons

Optional add-ons are not essential for a cluster to operate, but you can install them as needed to extend its capabilities. These add-ons are grouped into several categories, including application management, logging and monitoring, storage, networking, and security.

Generate add-onTerraformparameters from the Console

Use the ACK console to generate Terraform configurations to manage add-ons.

Log on to the ACK console. In the left navigation pane, click Clusters.
On the Clusters page, click the name of your cluster. In the left navigation pane, click Components and Add-ons.
On the Add-ons page, find the desired add-on and click Install or Configuration on its card.
In the dialog box, click Console-to-Code. In the panel on the right, click the Terraform tab. The console displays the corresponding Terraform configuration. You can copy this code.

Add-on management practices

Use Terraform to specify which add-ons to install during cluster creation and to manage their lifecycle. The following sections describe how to manage the add-on lifecycle and provide best practices for various scenarios.

Specify add-ons during cluster creation

You can specify which add-ons to install when you create a cluster. The following Terraform resources are used to create types of clusters:

ACK managed cluster: alicloud_cs_managed_kubernetes
ACK dedicated cluster: alicloud_cs_kubernetes
ACK Edge cluster: alicloud_cs_edge_kubernetes
ACK Serverless cluster: alicloud_cs_serverless_kubernetes

You can use theaddons attribute in any of these resources to specify the add-ons to install during cluster creation. Theaddons attribute is defined as follows:

# The following example uses an ACK managed cluster.
resource "alicloud_cs_managed_kubernetes" "default" {
  # Other parameters.
  # ...

  # The addons block is a list. Define this block in the resource to specify which add-ons to install during cluster creation.
  addons {
    # The name of the add-on. You can query available add-ons and their versions by using the alicloud_cs_kubernetes_addons data source.
    name = "XXX"

    # Custom parameters for the add-on. For more information, see the "Modify the custom parameters of a cluster add-on" section.
    config = jsonencode(
      {
        ....
      }
    )

    # The default value is false. ACK installs some add-ons by default to simplify cluster management. To prevent these default add-ons from being installed, set disabled = true.
    disabled = XXX
  }
}

Important

After the cluster is created, you cannot use the addons block to manage the add-on lifecycle, such as upgrading, uninstalling, or reconfiguring add-ons. To manage add-ons after cluster creation, see Manage the lifecycle of add-ons after the cluster is created.

The following table describes the add-ons that you can configure in ACK.

Add-on	Type	Description	Configuration
appcenter	Application management	A centralized application center for managing application deployments and lifecycles across multiple clusters.	`addon { name = "appcenter" }`
progressive-delivery-tool	Application management	Enables progressive delivery for applications.	`addon { name = "progressive-delivery-tool" }`
alicloud-monitor-controller	Logging and monitoring	A system add-on that integrates ACK with CloudMonitor.	`addon { name = "alicloud-monitor-controller" }`
metrics-server	Logging and monitoring	An enhanced monitoring add-on based on the open-source Metrics Server. It collects resource metrics, exposes them through the Metrics API, and enables the Horizontal Pod Autoscaler (HPA).	`addon { name = "metrics-server" }`
ack-node-problem-detector	Logging and monitoring	An enhanced add-on based on the open-source Node Problem Detector project. It monitors node exception events and integrates with third-party monitoring platforms.	`addons { name = "ack-node-problem-detector" }`
ags-metrics-collector	Logging and monitoring	A monitoring add-on for genomics computing customers that tracks detailed resource usage of each node in a genomics workflow.	`addons { name = "ags-metrics-collector" }`
ack-arms-prometheus	Logging and monitoring	Enables cluster monitoring with Managed Service for Prometheus.	`addons { name = "arms-prometheus" }`
loongcollector	Logging and monitoring	Collects Kubernetes container logs with Log Service.	`addons { name = "loongcollector" }`
csi-plugin	Storage	Supports volume mounting and unmounting. This add-on is installed by default if you select the CSI plugin for storage integration when you create the cluster.	`addons { name = "csi-plugin" }`
csi-provisioner	Storage	Supports automatic volume creation. This add-on is installed by default if you select the CSI plugin for storage integration when you create the cluster.	`addons { name = "csi-provisioner" }`
storage-operator	Storage	Manages the lifecycle of storage add-ons.	`addons { name = "storage-operator" }`
alicloud-disk-controller	Storage	Supports automatic creation of disk volumes.	`addons { name = "alicloud-disk-controller" }`
flexvolume	Storage	An early-stage storage volume extension mechanism from the Kubernetes community. Flexvolume supports volume mounting and unmounting. This add-on is installed by default if you select the flexvolume plugin for storage integration when you create the cluster.	`addons { name = "flexvolume" }`
nginx-ingress-controller	Networking	Parses Ingress forwarding rules. When the Ingress controller receives a request, it matches the request against Ingress forwarding rules and forwards the request to the backend Service.	`addons { name = "nginx-ingress-controller" }`
terway-eniip	Networking	An open-source Container Network Interface (CNI) plugin developed by Alibaba Cloud based on Virtual Private Cloud (VPC). It lets you define access policies between containers using standard Kubernetes network policies and establish network connectivity within Kubernetes clusters. This add-on is installed by default if you select the Terway network plugin when you create the cluster.	`addons { name = "terway-eniip" }`
ack-node-local-dns	Networking	A local DNS caching solution based on the open-source NodeLocal DNSCache project.	`addons { name = "ack-node-local-dns" }`
aliyun-acr-credential-helper	Security	Enables password-less pulling of private images from Container Registry Personal or Enterprise Edition instances.	`addons { name = "aliyun-acr-credential-helper" }`
gatekeeper	Security	Helps you manage and enforce Open Policy Agent (OPA) policies in your cluster and enables features such as namespace label management.	`addons { name = "gatekeeper" }`
kritis-validation-hook	Security	A key add-on that verifies container image signatures when you deploy trusted containers.	`addons { name = "kritis-validation-hook" }`
security-inspector	Security	A key add-on that enables the security inspection feature.	`addons { name = "security-inspector" }`
ack-kubernetes-webhook-injector	Security	A Kubernetes add-on that dynamically adds Pod IP addresses to or removes them from the whitelists of various Alibaba Cloud services, eliminating the need for manual configuration.	`addons { name = "ack-kubernetes-webhook-injector" }`
ack-arena	Other	Simplifies the installation of open-source Arena, enabling one-click installation from the console.	`addons { name = "ack-arena" }`
ack-cost-exporter	Other	A data processing plugin for the ACK cost analysis feature.	`addons { name = "ack-cost-exporter" }`
ack-kubernetes-cronhpa-controller	Other	Enables scheduled scaling for workloads.	`addons { name = "ack-kubernetes-cronhpa-controller" }`
ack-virtual-node	Other	Based on the open-source Virtual Kubelet project, this add-on extends support for the Aliyun Provider with numerous optimizations for seamless integration between Kubernetes and Elastic Container Instance (ECI).	`addons { name = "ack-virtual-node" }`
aesm	Other	The Intel® SGX Architectural Enclave Service Manager (Intel® SGX AESM) is a system component of Intel® SGX. It provides services such as SGX enclave launch support, key provisioning, and remote attestation.	`addons { name = "aesm" }`
aliyun-acr-acceleration-suite	Other	A client plugin for accelerated, on-demand image loading. It is deployed as a DaemonSet on worker nodes.	`addons { name = "aliyun-acr-acceleration-suite" }`
migrate-controller	Other	A Kubernetes application migration add-on developed based on the open-source Velero project.	`addons { name = "migrate-controller" }`
resource-controller	Other	A key add-on that dynamically controls Pod resources. This add-on is required to use topology-aware CPU scheduling in ACK Pro clusters.	`addons { name = "resource-controller" }`
sandboxed-container-controller	Other	A dedicated controller for the sandboxed container runtime that enhances and extends its basic functions.	`addons { name = "sandboxed-container-controller" }`
sandboxed-container-helper	Other	An add-on for diagnostics and O&M of sandboxed containers.	`addons { name = "sandboxed-container-helper" }`
sgx-device-plugin	Other	Jointly developed by Alibaba Cloud and Ant Group, this Kubernetes device plugin for Intel SGX simplifies the use of SGX in containers.	`addons { name = "sgx-device-plugin" }`

Manage the add-on lifecycle

To manage the lifecycle of an add-on, you need an existing Kubernetes cluster. If you do not have a cluster, create one first.

Use the alicloud_cs_kubernetes_addon Resource to manage the lifecycle of an add-on in your cluster, including its installation, upgrade, uninstallation, and custom configuration. The following code shows the definition and attributes of alicloud_cs_kubernetes_addon:

resource "alicloud_cs_kubernetes_addon" "addon-example" {
  # The cluster ID.
  cluster_id = "XXXX"

  # The add-on name. Use the alicloud_cs_kubernetes_addons Data Source to query all installed and available add-ons and their versions.
  name = "XXXX"

  # The add-on version.
  version = "XXXX"

  # The custom configuration for the add-on, as a JSON string. Use Terraform's jsonencode function or a raw JSON string with escaped characters. For details on custom parameters, see "Modify the custom configuration of a cluster add-on".
  config = jsonencode(
    {
      ....
    }
  )
}

You can also provide custom parameters as a raw JSON string, but you must escape the characters. For example, the nginx-ingress-controller add-on can be configured in the following two ways:

Using jsonencode:

config = jsonencode(   
  {       
    IngressSlbNetworkType="internet"       
    IngressSlbSpec="slb.s2.small"     
  }  
)

Using a raw string:

config = "{\"IngressSlbNetworkType\":\"internet\",\"IngressSlbSpec\":\"slb.s2.small\"}"

Import installed add-ons into Terraform

Use the terraform import command to bring add-ons already installed in your cluster under Terraform's management. The following example demonstrates how to import the nginx-ingress-controller add-on.

Create a new file with a .tf extension, or use an existing .tf file, and define a Resource.
The alicloud_cs_kubernetes_addon Resource manages cluster add-ons. Leave the Resource block empty for now.
```
resource "alicloud_cs_kubernetes_addon" "nginx-ingress-controller" {
}
```
Run the following command to import the installed nginx-ingress-controller add-on.
Terraform pulls the configuration of the nginx-ingress-controller add-on from the cluster and writes it to a .state file.
```
terraform import alicloud_cs_kubernetes_addon.nginx-ingress-controller <cluster_id>:nginx-ingress-controller
```
Run the terraform plan command. The output shows the differences between the nginx-ingress-controller add-on's configuration in the cluster and the Resource you defined.
Based on the diff and the content of the .state file, update the Resource you defined in Step 1. The import is complete when terraform plan reports that no changes are needed.
```
resource "alicloud_cs_kubernetes_addon" "nginx-ingress-controller" {
  cluster_id = "XXXXX"
  name = "nginx-ingress-controller"
  version = "v1.2.1-aliyun.1"
  config = jsonencode(
    {
      IngressSlbNetworkType = "internet"
      IngressSlbSpec        = "slb.s2.small"
    }
  )
}
```

Install a cluster add-on

Use the alicloud_cs_kubernetes_addon Resource to install an add-on in an existing cluster. The following example shows how to install the gatekeeper add-on.

In your .tf file, define the add-on Resource and specify the following arguments.
- The cluster ID.
- The add-on name and version:
  
  Use the alicloud_cs_kubernetes_addons Data Source to query the names and versions of available add-ons. The query result returns only the latest installable version for each add-on. To install an earlier version, check the add-on's release notes and specify the desired version number.
- (Optional) The custom configuration for the add-on:
  
  Set your custom configuration in the config argument. You can use Terraform's built-in jsonencode function to construct your desired configuration. Use the alicloud_cs_kubernetes_addon_metadata Data Source to query the available parameters for the add-on. For more information, see Modify the custom configuration of a cluster add-on.
Details
```
resource "alicloud_cs_kubernetes_addon" "gatekeeper" {
  cluster_id = "ce36b7c61e126430b8b245730ca6d****"
  name = "gatekeeper"
  version = "v3.8.1.113-geb7947ef-aliyun"
  config = jsonencode(
    {
      AdmissionPodCpuLimit      = "1000m"
      AdmissionPodCpuRequest    = "100m"
      AdmissionPodMemoryLimit   = "512Mi"
      AdmissionPodMemoryRequest = "256Mi"
      AdmissionPodNumber        = 3
      AuditInterval             = 1800
      AuditPodCpuLimit          = "1000m"
      AuditPodCpuRequest        = "100m"
      AuditPodMemoryLimit       = "512Mi"
      AuditPodMemoryRequest     = "256Mi"
      EnableAuditPod            = false
      EnableMutatingWebhook     = false
    }
  )
}
```

Run the following command to install the add-on in the cluster.

terraform apply

Expected output:

Plan: 1 to add, 0 to change, 0 to destroy.

Do you want to perform these actions?
  Terraform will perform the actions described above.
  Only 'yes' will be accepted to approve.

  Enter a value: yes

alicloud_cs_kubernetes_addon.gatekeeper: Creating...
alicloud_cs_kubernetes_addon.gatekeeper: Still creating... [10s elapsed]
alicloud_cs_kubernetes_addon.gatekeeper: Creation complete after 16s [id=XXXXX:gatekeeper]

Apply complete! Resources: 1 added, 0 changed, 0 destroyed.

The Apply complete! message indicates the add-on is successfully installed.

Upgrade a cluster add-on

You can use the alicloud_cs_kubernetes_addons Data Source to query for available upgrade versions of an add-on. If a newer version is available, upgrade the add-on by modifying the version number in your configuration.

Details

resource "alicloud_cs_kubernetes_addon" "gatekeeper" {
  cluster_id = "ce36b7c61e126430b8b245730ca6d****"
  name = "gatekeeper"

  # Modify the version to the desired upgrade version.
  version = "XXXXXXXXX"
  config = jsonencode(
    {
      AdmissionPodCpuLimit      = "1000m"
      AdmissionPodCpuRequest    = "100m"
      AdmissionPodMemoryLimit   = "512Mi"
      AdmissionPodMemoryRequest = "256Mi"
      AdmissionPodNumber        = 3
      AuditInterval             = 1800
      AuditPodCpuLimit          = "1000m"
      AuditPodCpuRequest        = "100m"
      AuditPodMemoryLimit       = "512Mi"
      AuditPodMemoryRequest     = "256Mi"
      EnableAuditPod            = false
      EnableMutatingWebhook     = false
    }
  )
}

Run the terraform apply command to upgrade the add-on. The upgrade is complete when the command succeeds.

Modify custom parameters

Some ACK add-ons allow you to specify custom parameters. Update an add-on's configuration by modifying the config argument in the alicloud_cs_kubernetes_addon Resource. The following example uses the gatekeeper add-on.

Details

resource "alicloud_cs_kubernetes_addon" "gatekeeper" {
  cluster_id = "ce36b7c61e126430b8b245730ca6d****"
  name = "gatekeeper"
  version = "v3.8.1.113-geb7947ef-aliyun"

  # You can modify the properties in the config block and apply the changes to update the add-on's configuration.

}

Use the alicloud_cs_kubernetes_addon_metadata Data Source to view all configurable parameters that an add-on supports. For example, to query the parameters for the gatekeeper add-on, add the following to your .tf file.

# Define a Data Source to get the schema of configurable parameters for the gatekeeper add-on.
data "alicloud_cs_kubernetes_addon_metadata" "default" {
  cluster_id = "ce36b7c61e126430b8b245730ca6d****"
  name       = "gatekeeper"
  version    = "v3.8.1.113-geb7947ef-aliyun"
}

# Display the schema by using an output.
output "addon_config_schema" {
  value = data.alicloud_cs_kubernetes_addon_metadata.default.config_schema
}

Run the terraform apply command. The command returns the result in JSON Schema format. The properties attribute within this schema defines all supported configuration parameters. Based on the returned schema, you can specify any of the supported parameters. The parameter attributes are described as follows:

default: The default value.
description: A description of the parameter.
pattern: A regular expression that validates the parameter's value.
type: The data type.

Details

addon_config_schema = <<EOT
{
  "$schema": "http://json-schema.org/draft-07/schema#",
  "properties": {
    "AdmissionPodCpuLimit": {
      "default": "1000m",
      "description": "cpu limit for gatekeeper",
      "pattern": "^(|[1-9][0-9]*(m|\\.\\d+)?)$",
      "type": "string"
    },
    "AdmissionPodCpuRequest": {
      "default": "100m",
      "description": "cpu request for gatekeeper",
      "pattern": "^[1-9][0-9]*(m|\\.\\d+)?$",
      "type": "string"
    },
    "AdmissionPodMemoryLimit": {
      "default": "512Mi",
      "description": "memory limit for gatekeeper",
      "pattern": "^(|[1-9][0-9]*(\\.\\d+)?(K|Ki|M|Mi|G|Gi|T|Ti)?)$",
      "type": "string"
    },

    ......
  },
  "title": "Config",
  "type": "object"
}
EOT

Configure network add-ons

In ACK, you can use the Terway network add-on for container networking. For more information, see Network.

The following example shows how to configure the network add-on with Terraform.

Details

# Configure the Terway network add-on in exclusive elastic network interface (ENI) mode. This is the default.
# In this mode, the ECS instance's ENI quota limits the number of Pods per node.

resource "alicloud_cs_managed_kubernetes" "default" {
  # Other parameters.
  # ...

  addons {
    name = "terway-eniip"
  }
}

# Configure the Terway network add-on in IPVLAN mode.
# This mode uses IPVLAN and eBPF for ENI sharing and requires Alibaba Cloud Linux 2.
resource "alicloud_cs_managed_kubernetes" "default" {
  # Other parameters.
  # ...

  addons {
    name   =  "terway-eniip"
    config = "{\"IPVLAN\":\"true\",\"NetworkPolicy\":\"false\"}"
  }
}

# Configure the Terway network add-on in IPVLAN mode with network policy support enabled.
# This configuration also uses IPVLAN and eBPF for ENI sharing and requires Alibaba Cloud Linux 2.
# It enables policy-based network control for Pods.
resource "alicloud_cs_managed_kubernetes" "default" {
  # Other parameters.
  # ...

  addons {
    name   =  "terway-eniip"
    config = "{\"IPVLAN\":\"true\",\"NetworkPolicy\":\"true\"}"
  }
}

Configure storage components

ACK supports two types of volume add-ons: FlexVolume and CSI. FlexVolume is deprecated, and CSI is the recommended replacement. The CSI add-on is installed by default when you create an ACK cluster using Terraform, unless you specify a different volume add-on. The following example shows how to define volume add-ons in your Terraform configuration.

Details

# The CSI volume add-on includes the csi-plugin and csi-provisioner components. Install the storage-operator add-on to enable default dynamically provisioned volumes for NAS and CNFS file systems.
resource "alicloud_cs_managed_kubernetes" "default" {
  # Other parameters.
  # ...

  addons {
    name = "csi-plugin"
  }
  addons {
    name = "csi-provisioner"
  }
  addons {
    name   = "storage-operator"
    config = "{\"CnfsOssEnable\":\"false\",\"CnfsNasEnable\":\"true\"}"
  }
}

Configure the logging add-on

The ACK logging add-on collects and stores logs in Simple Log Service (SLS). It supports the following two storage methods:

Store logs in an existing SLS project.
Create a new SLS project automatically during cluster creation.

The configuration differs between these two scenarios. The following Terraform examples show how to configure each scenario:

Examples

# Automatically create an SLS project to store logs.
resource "alicloud_cs_managed_kubernetes" "default" {
  # Other parameters.
  # ...

  addons {
    name = "loongcollector"
  }
}

# Automatically create an SLS project to store logs and enable the automatic creation of an Ingress dashboard.
resource "alicloud_cs_managed_kubernetes" "default" {
  # Other parameters.
  # ...

  addons {
    name = "loongcollector"
    config = "{\"IngressDashboardEnabled\":\"true\"}"
  }
}


# Use an existing SLS project to store logs and enable the automatic creation of an Ingress dashboard.
resource "alicloud_cs_managed_kubernetes" "default" {
  # Other parameters.
  # ...

  addons {
    name = "loongcollector"
    config = "{\"IngressDashboardEnabled\":\"true\",\"sls_project_name\":\"k8s-log-c55c35ff493df47b88783bea48827****\"}"
  }
}

# Install and configure node-problem-detector.
# Store events in an automatically created SLS project.
resource "alicloud_cs_managed_kubernetes" "default" {
  # Other parameters.
  # ...

  addons {
    name   = "ack-node-problem-detector"
    config = "{\"sls_project_name\":\"\"}"
  }
}

# Install and configure node-problem-detector.
# Store events in an existing SLS project, which can share the same Logstore with loongcollector.
resource "alicloud_cs_managed_kubernetes" "default" {
  # Other parameters.
  # ...

  addons {
    name   = "ack-node-problem-detector"
    config = "{\"sls_project_name\":\"k8s-log-c55c35ff493df47b88783bea48827****\"}"
  }
}

Configure monitoring add-ons

ACK provides monitoring add-ons, including the CloudMonitor agent for ECS nodes and Managed Service for Prometheus. In Terraform, you can install the CloudMonitor agent on ECS nodes by configuring the install_cloud_monitor parameter.

Click to expand

# Install Managed Service for Prometheus.
resource "alicloud_cs_managed_kubernetes" "default" {
  # Other parameters.
  # ...
  addons {
    name = "arms-prometheus"
  }
}
# Install the CloudMonitor agent on ECS nodes.
resource "alicloud_cs_kubernetes_node_pool" "default" {
  # Other parameters.
  # ...
  install_cloud_monitor = true
}

Configure Ingress add-ons

ACK provides the nginx-ingress-controller and alb-ingress-controller add-ons to route application traffic in a Kubernetes cluster.

nginx-ingress-controller: An optimized version of the community ingress-nginx that provides flexible and reliable routing services for your Kubernetes cluster. For more information, see Overview of NGINX Ingresses.
alb-ingress-controller: A fully managed and highly reliable Ingress add-on that provides flexible and reliable routing services for your Kubernetes cluster. For more information, see Access Services by using an ALB Ingress.

The following example demonstrates how to use Terraform to configure the Ingress add-ons:

Example

# Use the nginx-ingress-controller to route traffic.
# If you use an Internet-facing SLB instance, set the IngressSlbNetworkType parameter to "internet" in the Config.
# If you use an internal-facing SLB instance, set the IngressSlbNetworkType parameter to "intranet" in the Config.
resource "alicloud_cs_managed_kubernetes" "default" {
  # Other parameters.
  # ...

  addons {
    name   =  "nginx-ingress-controller"
    config = "{\"IngressSlbNetworkType\":\"internet\",\"IngressSlbSpec\":\"slb.s2.small\"}"
  }
}

# Use the alb-ingress-controller to route traffic.
resource "alicloud_cs_managed_kubernetes" "default" {
  # Other parameters.
  # ...

  addons {
    name   = "alb-ingress-controller"
    config = "{\"albIngress\":{\"CreateDefaultALBConfig\":false}}"  # Do not create a default ALB instance.
#   config = "{\"albIngress\":{\"LoadBalancerId\":\"alb-vl8uiXXXXXxdr\",\"CreateDefaultALBConfig\":true}}" # Specify an existing ALB instance.
#   config = "{\"albIngress\":{\"AddressType\":\"Internet\",\"ZoneMappings\":{\"cn-hangzhou-l\":[\"vsw-uf6XXXXXoyb4qe\"],\"cn-hangzhou-m\":[\"vsw-uf6XXXX0rlkiq\"]},\"CreateDefaultALBConfig\":true}}" # Create a new ALB instance. You must specify at least two availability zones.
  }
}

Disable default add-ons

ACK installs some add-ons by default to simplify cluster management. If you do not need these add-ons when you create a cluster, you can disable them by setting disabled = true. For example, to disable the nginx-ingress-controller add-on:

# Disable the nginx-ingress-controller add-on.

resource "alicloud_cs_managed_kubernetes" "default" {
  # Other parameters.
  # ...

  addons {
    name     =  "nginx-ingress-controller"
    disabled = true
  }
}

Add-ons installed by default

The following add-ons are installed by default when you create a cluster.

Cluster type	Add-on type	Add-on name	Description
ACK cluster	System add-ons	kube-scheduler	Schedules resources in the cluster.
		cloud-controller-manager	Creates load balancers for Kubernetes applications and manages node route entries.
		kube-apiserver	Serves as the cluster's central control point and API gateway.
		kube-controller-manager	Manages the state of core Kubernetes resources.
	Logging and monitoring	alicloud-monitor-controller	Monitors the lifecycle and status of application containers.
	Logging and monitoring	metrics-server	Provides container resource metrics for the cluster's auto scaling mechanism.
	Storage	csi-plugin	Manages the volume lifecycle. (Recommended)
		csi-provisioner	Provisions and deletes volumes. (Recommended)
		storage-operator	Manages the O&M of storage add-ons. (Recommended)
	Network	CoreDNS	Serves as the DNS server for the cluster.
		Gateway API	Provides the Gateway API resource model.
		terway-eniip	Provides container networking by using the Terway network add-on.
		nginx-ingress-controller (Installed by default in Pro edition clusters)	An Ingress controller that uses NGINX to route traffic.
ACK Serverless cluster	System add-ons	kube-scheduler	Schedules resources in the cluster.
		ack-virtual-node	Enables elastic scaling with virtual nodes and Elastic Container Instance (ECI).
		cloud-controller-manager	Creates load balancers for Kubernetes applications and manages node route entries.
		kube-apiserver	Serves as the cluster's central control point and API gateway.
		kube-controller-manager	Manages the state of core Kubernetes resources.
	Network	CoreDNS	Serves as the DNS server for the cluster.
ACK Edge cluster	System add-ons	kube-scheduler	Schedules resources in the cluster.
		cloud-controller-manager	Creates load balancers for Kubernetes applications and manages node route entries.
		kube-apiserver	Serves as the cluster's central control point and API gateway.
		kube-controller-manager	Manages the state of core Kubernetes resources.
	Logging and monitoring	alicloud-monitor-controller	Monitors the lifecycle and status of application containers.
	Logging and monitoring	metrics-server	Provides container resource metrics for the cluster's auto scaling mechanism.
	Network	CoreDNS	Serves as the DNS server for the cluster.
	Network	terway-eniip	Provides container networking by using the Terway network add-on.
	Others	edge-controller-manager	-
		edge-tunnel-agent	Establishes a reverse O&M tunnel for cloud-to-edge communication.
		edge-tunnel-server	Establishes a reverse O&M tunnel for cloud-to-edge communication.
		yurt-app-manager	Provides node pool management and cell-based deployment for ACK Edge clusters.

Configuration examples

This section provides common configuration examples.

Use Terway as the network add-on.
For the storage add-on, choose either CSI or FlexVolume. Use CSI, as FlexVolume is no longer maintained.
For the routing add-on, choose either nginx-ingress-controller or alb-ingress-controller based on your business requirements.
You can install and combine other add-ons as needed.

Example 1: Minimal configuration

# If you create a cluster without configuring any add-ons, only the default add-ons are installed.
# This is the minimal configuration for a cluster. Replace the variables with your custom variables.
resource "alicloud_cs_managed_kubernetes" "default" {
  name                         = var.name
  cluster_spec                 = "ack.pro.small"
  is_enterprise_security_group = true
  pod_cidr                     = "172.20.0.0/16"
  service_cidr                 = "172.21.0.0/20"
  worker_vswitch_ids           = [var.vswitch_id]
}

Example 2: Configure the Terway network add-on

# Create a cluster that uses the Terway network add-on.
# This configuration uses the Assign One ENI to Each Pod mode.

resource "alicloud_cs_managed_kubernetes" "default" {
  name                         = var.name
  cluster_spec                 = "ack.pro.small"
  is_enterprise_security_group = true
  pod_vswitch_ids              = [var.vswitch_id]
  service_cidr                 = "172.21.0.0/20"
  worker_vswitch_ids           = [var.vswitch_id]

  addons {
    name = "terway-eni"
  }
}

# Create a cluster that uses the Terway network add-on.
# This configuration uses IPVLAN mode and enables network policies.
resource "alicloud_cs_managed_kubernetes" "default" {
  name                         = var.name
  cluster_spec                 = "ack.pro.small"
  is_enterprise_security_group = true
  service_cidr                 = "172.21.0.0/20"
  pod_vswitch_ids              = [var.vswitch_id]
  worker_vswitch_ids           = [var.vswitch_id]

  addons {
    name   =  "terway-eniip"
    config = "{\"IPVlan\":\"true\",\"NetworkPolicy\":\"true\"}"
  }
}

Example 3: Common template with Terway, CSI, and nginx-ingress-controller

# Common template with the Terway, CSI, and nginx-ingress-controller add-ons.

resource "alicloud_cs_managed_kubernetes" "default" {
  name                         = var.name
  cluster_spec                 = "ack.pro.small"
  is_enterprise_security_group = true
  service_cidr                 = "172.21.0.0/20"
  pod_vswitch_ids              = [var.vswitch_id]
  worker_vswitch_ids           = [var.vswitch_id]

  addons {
    name = "terway-eniip"
    config = "{\"IPVlan\":\"true\",\"NetworkPolicy\":\"false\"}"
  }
  addons {
    name = "csi-plugin"
  }
  addons {
    name = "csi-provisioner"
  }
  addons {
    name = "storage-operator"
    config = "{\"CnfsOssEnable\":\"false\",\"CnfsNasEnable\":\"true\"}"
  }
  addons {
    name = "loongcollector"
    config = "{\"IngressDashboardEnabled\":\"true\"}"
  }
  addons {
    name = "ack-node-problem-detector"
    config = "{\"sls_project_name\":\"\"}"
  }
  addons {
    name = "nginx-ingress-controller"
    config = "{\"IngressSlbNetworkType\":\"internet\",\"IngressSlbSpec\":\"slb.s2.small\"}"
  }
  addons {
    name = "ack-node-local-dns"
  }
  addons {
    name = "arms-prometheus"
  }
  addons {
    name = "alicloud-monitor-controller"
    config = "{\"group_contact_ids\":\"[10619]\"}"
  }
}