How to configure a Classic Load Balancer for an ACK cluster by using annotations - Container Service for Kubernetes

You can use annotations in a Service YAML file to configure Classic Load Balancer (CLB) instances, listeners, and backend server groups. This lets you use a wide range of load balancing features.

Index

Categorization	Feature classification	Configuration link
Notes on annotations
Typical operations for CLB instances	Create an SLB instance	Create an Internet-facing SLB instance Create an internal-facing SLB instance Create an SLB instance that is billed on a pay-by-bandwidth basis Specify primary and secondary zones when you create an SLB instance Create an SLB instance that is billed on a pay-by-CU basis Create an IPv6 SLB instance
	Specify an existing instance	Use an existing SLB instance Use an existing SLB instance and forcibly overwrite its existing listeners
	Configure an SLB instance	Specify an SLB instance type Specify a vSwitch for an SLB instance Specify an IP address for an internal-facing SLB instance Add extra tags to an SLB instance Specify a name for an SLB instance Specify the resource group to which an SLB instance belongs Set a hostname for a Service
	Enable instance protection	Enable deletion protection for an SLB instance Enable configuration read-only mode for an SLB instance Retain an automatically created SLB instance when you delete a Service
Typical operations for listeners	Session persistence settings	Configure the session persistence timeout period for a TCP listener Configure session persistence (insert cookie) for HTTP and HTTPS listeners
	Port and protocol configurations	Specify a redirection port for an SLB instance Create a listener with a health check Create a UDP listener Create an HTTP listener Create an HTTPS listener Configure TCP and UDP protocols for a listener at the same time Configure the Proxy Protocol for TCP and UDP listeners
	Advanced configuration	Set the scheduling algorithm for an SLB instance Configure an access control policy group for an SLB instance Configure a security policy for a listener Configure connection draining for a listener Configure additional request headers for a listener Set the idle connection timeout period for a listener Specify the request timeout period for a listener Specify the connection timeout period for a listener Disable the HTTP/2 feature for a listener
Typical operations for backend server groups	Configuration management	Use worker nodes with a specific label as backend servers Use the nodes where pods reside as backend servers Remove unschedulable nodes from the backend server group of the CLB instance Directly attach pod ENIs to the backend server group of the CLB instance Reuse an existing vServer group Set the weight of a Service to receive traffic Ignore backend server weight updates
References

Notes on annotations

Annotations are case-sensitive.
Before you use an annotation, check the supported Cloud Controller Manager (CCM) version for the feature in this topic. To upgrade the CCM component, see Manage components. For more information about the change history of the CCM component, see Cloud Controller Manager.
As of September 11, 2019, alicloud was updated to alibaba-cloud in the annotations field.
For example:
Before the update: service.beta.kubernetes.io/alicloud-loadbalancer-id
After the update: service.beta.kubernetes.io/alibaba-cloud-loadbalancer-id
The system remains compatible with the alicloud format. No changes are required.

Typical operations for CLB instances

Create an Internet-facing SLB instance

apiVersion: v1
kind: Service
metadata:
  name: nginx
  namespace: default
spec:
  ports:
  - port: 80
    protocol: TCP
    targetPort: 80
  selector:
    run: nginx
  type: LoadBalancer

Create an internal-facing SLB instance

Annotation: service.beta.kubernetes.io/alibaba-cloud-loadbalancer-address-type

Description

Supported CCM version

Specifies that the SLB instance is an internal-facing instance. Valid values:

internet: The service is accessed over the Internet. The Address Type of the CLB instance must be Internet.
intranet: The service is accessed over a private network. The Address Type of the CLB instance must be Internal.

Default value: internet

v1.9.3 and later

apiVersion: v1
kind: Service
metadata:
  annotations:
    service.beta.kubernetes.io/alibaba-cloud-loadbalancer-address-type: "intranet"
  name: nginx
  namespace: default
spec:
  ports:
  - port: 80
    protocol: TCP
    targetPort: 80
  selector:
    run: nginx
  type: LoadBalancer

Specify an SLB instance type

Annotation: Multiple, as shown in the following table.

Annotation

Description

Supported CCM version

service.beta.kubernetes.io/alibaba-cloud-loadbalancer-instance-charge-type

The billing method of the instance. Valid values:

PayBySpec: pay-by-specification.
PayByCLCU: pay-by-CU.

Default value: PayBySpec

Important

If the Cloud Controller Manager version is v2.5.0 or later, the default value is changed to PayByCLCU.
You cannot specify an instance type for a pay-by-CU SLB instance. This means that you cannot set the value of this parameter to PayByCLCU and use the service.beta.kubernetes.io/alibaba-cloud-loadbalancer-spec annotation at the same time.

v2.4.0 and later

service.beta.kubernetes.io/alibaba-cloud-loadbalancer-spec

The type of the SLB instance. You can use this parameter to create a CLB instance of a specific type or update the type of an existing CLB instance. Valid values:

slb.s1.small
slb.s2.small
slb.s2.medium
slb.s3.small
slb.s3.medium
slb.s3.large

Default value: slb.s1.small

For more information about the valid values of this parameter, see CreateLoadBalancer.

Important

If you modify the type of a pay-by-specification CLB instance in the CLB console, the instance type may be changed back to the original one by CCM. Proceed with caution.

v1.9.3 and later

apiVersion: v1
kind: Service
metadata:
  annotations:
    service.beta.kubernetes.io/alibaba-cloud-loadbalancer-instance-charge-type: "PayBySpec"
    service.beta.kubernetes.io/alibaba-cloud-loadbalancer-spec: "slb.s1.small"
  name: nginx
  namespace: default
spec:
  ports:
  - port: 443
    protocol: TCP
    targetPort: 443
  selector:
    run: nginx
  type: LoadBalancer

Use an existing SLB instance

Annotation: service.beta.kubernetes.io/alibaba-cloud-loadbalancer-id

Description

Supported CCM version

Important

To prevent unexpected behavior, such as cluster unavailability or traffic interruptions, do not reuse the API Server CLB instance or a CLB instance created by CCM. Manually create a new instance in the Classic Load Balancer (CLB) console.

The ID of the SLB instance. Use this annotation to specify an existing CLB instance.

By default, the listeners of an existing SLB instance are not overwritten. To forcibly overwrite the listeners, set service.beta.kubernetes.io/alibaba-cloud-loadbalancer-force-override-listeners to "true". This configuration overwrites only the listeners of the current Service and does not affect other listeners.
Note
By default, the existing listeners of a reused SLB instance are not overwritten. This is due to the following reasons:
- If services are bound to the listeners of the existing SLB instance, forcibly overwriting the listeners may cause service interruptions.
- CCM supports only a limited number of backend configurations and cannot process complex configurations. If you have complex backend configuration requirements, you can configure listeners in the console without overwriting them.
For these reasons, we recommend that you do not forcibly overwrite listeners. You can forcibly overwrite the listeners of an existing SLB instance if the listener ports are no longer in use.
(For versions earlier than v2.10.0) You cannot add extra tags (annotation: service.beta.kubernetes.io/alibaba-cloud-loadbalancer-additional-resource-tags) when you use an existing SLB instance.

v1.9.3.81-gca19cd4-aliyun and later

apiVersion: v1
kind: Service
metadata:
  annotations:
    service.beta.kubernetes.io/alibaba-cloud-loadbalancer-id: "${YOUR_LOADBALANCER_ID}"
  name: nginx
  namespace: default
spec:
  ports:
  - port: 443
    protocol: TCP
    targetPort: 443
  selector:
    run: nginx
  type: LoadBalancer

Use an existing SLB instance and forcibly overwrite its existing listeners

Annotation: Multiple, as shown in the following table. This operation forcibly overwrites existing listeners. If a listener port conflict occurs, the existing listener is deleted.

Annotation

Description

Supported CCM version

service.beta.kubernetes.io/alibaba-cloud-loadbalancer-id

Important

The ID of the SLB instance. Use this annotation to specify an existing CLB instance.

By default, the listeners of an existing SLB instance are not overwritten. To forcibly overwrite the listeners, set service.beta.kubernetes.io/alibaba-cloud-loadbalancer-force-override-listeners to "true". This configuration overwrites only the listeners of the current Service and does not affect other listeners.
Note
By default, the existing listeners of a reused SLB instance are not overwritten. This is due to the following reasons:
- If services are bound to the listeners of the existing SLB instance, forcibly overwriting the listeners may cause service interruptions.
- CCM supports only a limited number of backend configurations and cannot process complex configurations. If you have complex backend configuration requirements, you can configure listeners in the console without overwriting them.
For these reasons, we recommend that you do not forcibly overwrite listeners. You can forcibly overwrite the listeners of an existing SLB instance if the listener ports are no longer in use.
(For versions earlier than v2.10.0) You cannot add extra tags (annotation: service.beta.kubernetes.io/alibaba-cloud-loadbalancer-additional-resource-tags) when you use an existing SLB instance.

service.beta.kubernetes.io/alibaba-cloud-loadbalancer-force-override-listeners

Specifies whether to forcibly overwrite the listeners of the CLB instance when you bind the instance. Valid values:

"true": overwrites the listeners. This overwrites only the listeners of the current Service and does not affect other listeners.
"false": Does not overwrite existing data.

Default value: "false"

Important

If you reuse an existing CLB instance and set force-override to "true", do not allow multiple Services to reuse the same listener of the CLB instance. Otherwise, listener configuration conflicts may occur.

v1.9.3.81-gca19cd4-aliyun and later

apiVersion: v1
kind: Service
metadata:
  annotations:
    service.beta.kubernetes.io/alibaba-cloud-loadbalancer-id: "${YOUR_LOADBALANCER_ID}"
    service.beta.kubernetes.io/alibaba-cloud-loadbalancer-force-override-listeners: "true"
  name: nginx
  namespace: default
spec:
  ports:
  - port: 443
    protocol: TCP
    targetPort: 443
  selector:
    run: nginx
  type: LoadBalancer

Specify primary and secondary zones when you create an SLB instance

Annotation: Multiple, as shown in the following table. The primary and secondary zones cannot be modified after the instance is created.

SLB instances in some regions do not support primary and secondary zones. For more information, see the CLB instance creation page.

Annotation	Description	Supported CCM version
service.beta.kubernetes.io/alibaba-cloud-loadbalancer-master-zoneid	The ID of the zone in which the primary backend server is deployed.	v1.9.3.10-gfb99107-aliyun and later
service.beta.kubernetes.io/alibaba-cloud-loadbalancer-slave-zoneid	The ID of the zone in which the secondary backend server is deployed.	v1.9.3.10-gfb99107-aliyun and later

apiVersion: v1
kind: Service
metadata:
  annotations:
    service.beta.kubernetes.io/alibaba-cloud-loadbalancer-master-zoneid: "cn-hangzhou-k"
    service.beta.kubernetes.io/alibaba-cloud-loadbalancer-slave-zoneid: "cn-hangzhou-j"
  name: nginx
  namespace: default
spec:
  ports:
  - port: 80
    protocol: TCP
    targetPort: 80
  selector:
    run: nginx
  type: LoadBalancer

Create an SLB instance that is billed on a pay-by-bandwidth basis

Annotation: Multiple, as shown in the following table. The following two annotations are required.

Annotation

Description

Supported CCM version

service.beta.kubernetes.io/alibaba-cloud-loadbalancer-charge-type

The billing method of the SLB instance. Valid values:

paybytraffic
paybybandwidth

Default value: paybytraffic

v1.9.3 and later

service.beta.kubernetes.io/alibaba-cloud-loadbalancer-bandwidth

The bandwidth of the SLB instance. This parameter specifies the peak bandwidth. Default value: 50. This parameter applies only to Internet-facing SLB instances. For more information about other limits, see Modify the billing method of an Internet-facing SLB instance.

v1.9.3.10-gfb99107-aliyun and later

apiVersion: v1
kind: Service
metadata:
  annotations:
    service.beta.kubernetes.io/alibaba-cloud-loadbalancer-charge-type: "paybybandwidth"
    service.beta.kubernetes.io/alibaba-cloud-loadbalancer-bandwidth: "2"
  name: nginx
  namespace: default
spec:
  ports:
  - port: 443
    protocol: TCP
    targetPort: 443
  selector:
    run: nginx
  type: LoadBalancer

Specify a vSwitch for an SLB instance

Annotation: Multiple, as shown in the following table. The following two annotations are required.

Annotation

Description

Supported CCM version

service.beta.kubernetes.io/alibaba-cloud-loadbalancer-address-type

Specifies that the SLB instance is an internal-facing instance. Valid values:

internet: The service is accessed over the Internet. The Address Type of the CLB instance must be Internet.
intranet: The service is accessed over a private network. The Address Type of the CLB instance must be Internal.

Default value: internet

v1.9.3 and later

service.beta.kubernetes.io/alibaba-cloud-loadbalancer-vswitch-id

The ID of the vSwitch to which the SLB instance belongs. The vSwitch must belong to the same VPC as the Kubernetes cluster.

When you set this parameter, you must also set service.beta.kubernetes.io/alibaba-cloud-loadbalancer-address-type to "intranet".

You can log on to the VPC console to query the vSwitch ID.

v1.9.3 and later

apiVersion: v1
kind: Service
metadata:
  annotations:
   service.beta.kubernetes.io/alibaba-cloud-loadbalancer-address-type: "intranet"
   service.beta.kubernetes.io/alibaba-cloud-loadbalancer-vswitch-id: "${YOUR_VSWITCH_ID}"
  name: nginx
  namespace: default
spec:
  ports:
  - port: 443
    protocol: TCP
    targetPort: 443
  selector:
    run: nginx
  type: LoadBalancer

Specify an IP address for an internal-facing SLB instance

Annotation: Multiple, as shown in the following table. The following three annotations are required.

Annotation	Description	Supported CCM version
service.beta.kubernetes.io/alibaba-cloud-loadbalancer-address-type	Specifies that the SLB instance is an internal-facing instance. Valid values: `internet`: The service is accessed over the Internet. This is the default value. The Address Type of the CLB instance must be Internet. `intranet`: The service is accessed over a private network. The Address Type of the CLB instance must be Internal. Default value: `internet`	v1.9.3 and later
service.beta.kubernetes.io/alibaba-cloud-loadbalancer-vswitch-id	The ID of the vSwitch to which the SLB instance belongs. The vSwitch must belong to the same VPC as the Kubernetes cluster. When you set this parameter, you must also set `service.beta.kubernetes.io/alibaba-cloud-loadbalancer-address-type` to intranet. You can log on to the VPC console to query the vSwitch ID.	v1.9.3 and later
service.beta.kubernetes.io/alibaba-cloud-loadbalancer-ip	The IP address of the internal-facing SLB instance. The IP address must be within the destination CIDR block of the vSwitch. Only IPv4 addresses are supported. This parameter must be used together with `service.beta.kubernetes.io/alibaba-cloud-loadbalancer-vswitch-id`. The IP address cannot be changed after the instance is created.	v2.7.0 and later

apiVersion: v1
kind: Service
metadata:
  annotations:
    service.beta.kubernetes.io/alibaba-cloud-loadbalancer-address-type: "intranet"
    service.beta.kubernetes.io/alibaba-cloud-loadbalancer-vswitch-id: "${YOUR_VSWITCH_ID}"
    service.beta.kubernetes.io/alibaba-cloud-loadbalancer-ip: "192.168.x.x"
  name: nginx
  namespace: default
spec:
  type: LoadBalancer
  ports:
    - port: 80
      targetPort: 80
      name: http
  selector:
    app: nginx

Add extra tags to an SLB instance

Annotation: service.beta.kubernetes.io/alibaba-cloud-loadbalancer-additional-resource-tags

Description	Supported CCM version
The list of tags that you want to add. Separate multiple tags with commas (,), for example, `"k1=v1,k2=v2"`. For CCM v2.10.0 and later, you can modify the tags of created instances and reused instances. Important After you add this annotation to a Service to specify extra tags, the modifications that you make to the tags of the corresponding SLB instance in the console may be overwritten.	v1.9.3 and later

apiVersion: v1
kind: Service
metadata:
  annotations:
    service.beta.kubernetes.io/alibaba-cloud-loadbalancer-additional-resource-tags: "Key1=Value1,Key2=Value2" 
  name: nginx
  namespace: default
spec:
  ports:
  - port: 80
    protocol: TCP
    targetPort: 80
  selector:
    run: nginx
  type: LoadBalancer

Create an IPv6 SLB instance

Annotation: service.beta.kubernetes.io/alibaba-cloud-loadbalancer-ip-version

Description

Supported CCM version

The IP version of the SLB instance. The IP version cannot be changed after the instance is created. When you use this parameter, the proxy mode of kube-proxy in the cluster must be IPVS. Valid values:

ipv4: IPv4.
ipv6: IPv6. The generated IPv6 address can be accessed only in an IPv6 environment.

Default value: ipv4

v1.9.3.220-g24b1885-aliyun and later

apiVersion: v1
kind: Service
metadata:
  annotations:
    service.beta.kubernetes.io/alibaba-cloud-loadbalancer-ip-version: "ipv6"
  name: nginx
spec:
  ports:
  - port: 80
    protocol: TCP
    targetPort: 80
  selector:
    app: nginx
  type: LoadBalancer

Enable deletion protection for an SLB instance

Annotation: service.beta.kubernetes.io/alibaba-cloud-loadbalancer-delete-protection

Description

Supported CCM version

Deletion protection for the SLB instance. Valid values:

on
off

Default value: on

Important

For an SLB instance created for a Service of the LoadBalancer type, if you manually enable deletion protection in the CLB console, you can still run the kubectl delete svc {your-svc-name} command to delete the SLB instance that is associated with the Service.

v1.9.3.313-g748f81e-aliyun and later

apiVersion: v1
kind: Service
metadata:
  annotations:
    service.beta.kubernetes.io/alibaba-cloud-loadbalancer-delete-protection: "on"
  name: nginx
spec:
  externalTrafficPolicy: Local
  ports:
  - port: 80
    protocol: TCP
    targetPort: 80
  selector:
    app: nginx
  type: LoadBalancer

Enable configuration read-only mode for an SLB instance

Annotation: service.beta.kubernetes.io/alibaba-cloud-loadbalancer-modification-protection

Description

Supported CCM version

Configuration read-only mode for the SLB instance. Valid values:

ConsoleProtection
NonProtection

Default value: ConsoleProtection

v1.9.3.313-g748f81e-aliyun and later

apiVersion: v1
kind: Service
metadata:
  annotations:
    service.beta.kubernetes.io/alibaba-cloud-loadbalancer-modification-protection: "ConsoleProtection"
  name: nginx
spec:
  externalTrafficPolicy: Local
  ports:
  - port: 80
    protocol: TCP
    targetPort: 80
  selector:
    app: nginx
  type: LoadBalancer

Specify a name for an SLB instance

Annotation: service.beta.kubernetes.io/alibaba-cloud-loadbalancer-name

Description	Supported CCM version
The name of the SLB instance. The name must be 2 to 128 characters in length, start with a letter or a Chinese character, and can contain digits, periods (.), underscores (_), and hyphens (-).	v1.9.3.313-g748f81e-aliyun and later

apiVersion: v1
kind: Service
metadata:
  annotations:
    service.beta.kubernetes.io/alibaba-cloud-loadbalancer-name: "your-svc-name"
  name: nginx
spec:
  externalTrafficPolicy: Local
  ports:
  - port: 80
    protocol: TCP
    targetPort: 80
  selector:
    app: nginx
  type: LoadBalancer

Specify the resource group to which an SLB instance belongs

Annotation: service.beta.kubernetes.io/alibaba-cloud-loadbalancer-resource-group-id

Description	Supported CCM version
The ID of the resource group to which the SLB instance belongs. The resource group ID cannot be changed after it is specified. You can query resource group IDs on the Alibaba Cloud Resource Management platform.	v1.9.3.313-g748f81e-aliyun and later

apiVersion: v1
kind: Service
metadata:
  annotations:
    service.beta.kubernetes.io/alibaba-cloud-loadbalancer-resource-group-id: "rg-xxxx"
  name: nginx
spec:
  externalTrafficPolicy: Local
  ports:
  - port: 80
    protocol: TCP
    targetPort: 80
  selector:
    app: nginx
  type: LoadBalancer

Set a hostname for a Service

Annotation: Multiple, as shown in the following table.

Annotation

Description

Supported CCM version

service.beta.kubernetes.io/alibaba-cloud-loadbalancer-protocol-port

Separate multiple values with commas (,), for example, https:443,http:80.

v1.9.3 and later

service.beta.kubernetes.io/alibaba-cloud-loadbalancer-hostname

Set a hostname for the Service. The hostname must conform to DNS naming conventions.

Note the following items:

After you add this annotation, the EXTERNAL-IP of the Service is changed from the default CLB IP address to your_service_name. When you access the CLB IP address from within the cluster, traffic is routed through the CLB instance and then forwarded to the cluster.
After you add this annotation, if the listener protocol is TCP or UDP, a loopback access issue occurs when you access the CLB IP address from within the cluster. For more information, see A client fails to access a CLB instance.
This annotation does not automatically associate the CLB instance with a domain name. To do this, go to the domain name service page to purchase a domain name and manually associate the CLB instance with the domain name. For more information about how to purchase a domain name, see Purchase a domain name.

v2.3.0 and later

apiVersion: v1
kind: Service
metadata:
  annotations:
    service.beta.kubernetes.io/alibaba-cloud-loadbalancer-protocol-port: "http:80"
    service.beta.kubernetes.io/alibaba-cloud-loadbalancer-hostname: "${your_service_hostname}"
  name: nginx-svc
  namespace: default
spec:
  ports:
  - name: http
    port: 80
    protocol: TCP
    targetPort: 80
  selector:
    app: nginx
  type: LoadBalancer

Expected output:

NAME         TYPE           CLUSTER-IP       EXTERNAL-IP            PORT(S)                      AGE
nginx-svc    loadBalancer   47.100.XX.XX     www.example.com        80:30248/TCP,443:32670/TCP   10s

Create an SLB instance that is billed on a pay-by-CU basis

Annotation: service.beta.kubernetes.io/alibaba-cloud-loadbalancer-instance-charge-type

Description

Supported CCM version

The billing method of the instance. Valid values:

PayBySpec: pay-by-specification. This is the default value.
PayByCLCU: Pay-as-you-go.

Default value: PayBySpec

Important

If the Cloud Controller Manager version is v2.5.0 or later, the default value is changed to PayByCLCU.
You cannot specify an instance type for a pay-by-CU SLB instance. This means that you cannot set the value of this parameter to PayByCLCU and use the service.beta.kubernetes.io/alibaba-cloud-loadbalancer-spec annotation at the same time.

v2.4.0 and later

apiVersion: v1
kind: Service
metadata:
  annotations:
    service.beta.kubernetes.io/alibaba-cloud-loadbalancer-instance-charge-type: "PayByCLCU"
  name: nginx
  namespace: default
spec:
  ports:
  - port: 80
    protocol: TCP
    targetPort: 80
  selector:
    run: nginx
  type: LoadBalancer

Retain an automatically created SLB instance when you delete a Service

Annotation: service.beta.kubernetes.io/alibaba-cloud-loadbalancer-preserve-lb-on-delete

Description

Supported CCM version

When you delete a Service of the LoadBalancer type, the CLB instance created by the Service is retained, and the kubernetes.do.not.delete and ack.aliyun.com tags are removed from the CLB instance. The existing servers in the vServer group are retained.

When this feature is enabled, a Warning event of the PreservedOnDelete type is generated during Service synchronization. After you configure this annotation, we recommend that you check whether this event exists to confirm that the feature is enabled.

Valid values:

Not empty: enables the retention feature.
Empty or not set: disables the retention feature.

Important

Perform this operation by deleting the Service instead of modifying the Service type. Otherwise, the Service may be incorrectly re-associated with the previously retained CLB instance.

v2.10.0 and later

apiVersion: v1
kind: Service
metadata:
  annotations:
    service.beta.kubernetes.io/alibaba-cloud-loadbalancer-preserve-lb-on-delete: "true"
  name: nginx
  namespace: default
spec:
  ports:
  - port: 80
    protocol: TCP
    targetPort: 80
  selector:
    run: nginx
  type: LoadBalancer

Typical operations for listeners

Configure the session persistence timeout period for a TCP listener

Annotation: service.beta.kubernetes.io/alibaba-cloud-loadbalancer-persistence-timeout

Description

Supported CCM version

The session persistence timeout period. This parameter applies only to TCP listeners. If multiple TCP listener ports are configured for the SLB instance, this configuration is applied to all TCP listeners by default.

Unit: seconds. Valid values: 0 to 3600. Default value: 0. A value of 0 indicates that session persistence is disabled. For more information, see CreateLoadBalancerTCPListener.

v1.9.3 and later

apiVersion: v1
kind: Service
metadata:
  annotations:
    service.beta.kubernetes.io/alibaba-cloud-loadbalancer-persistence-timeout: "1800"
  name: nginx
  namespace: default
spec:
  ports:
  - port: 443
    protocol: TCP
    targetPort: 443
  selector:
    run: nginx
  type: LoadBalancer

Configure session persistence (insert cookie) for HTTP and HTTPS listeners

Annotation: Multiple, as shown in the following table. When you insert a cookie, the following four annotations are required.

This feature is supported only by SLB instances that use HTTP or HTTPS.
If you configure multiple HTTP or HTTPS listener ports, session persistence is applied to all HTTP and HTTPS listeners by default.

Annotation	Description	Supported CCM version
service.beta.kubernetes.io/alibaba-cloud-loadbalancer-sticky-session	Specifies whether to enable session persistence. This parameter is valid only for HTTP and HTTPS listeners. Valid values: `on` `off` Default value: `off` For more information, see CreateLoadBalancerHTTPListener and CreateLoadBalancerHTTPSListener.	v1.9.3 and later
service.beta.kubernetes.io/alibaba-cloud-loadbalancer-sticky-session-type	The method that is used to process cookies. This parameter is valid only for HTTP and HTTPS listeners. This parameter is required when `service.beta.kubernetes.io/alibaba-cloud-loadbalancer-sticky-session` is set to `on`. Valid values: `insert`: inserts a cookie. `server`: rewrites a cookie. For more information, see CreateLoadBalancerHTTPListener and CreateLoadBalancerHTTPSListener.	v1.9.3 and later
service.beta.kubernetes.io/alibaba-cloud-loadbalancer-cookie-timeout	The cookie timeout period. This parameter is required when `service.beta.kubernetes.io/alibaba-cloud-loadbalancer-sticky-session` is set to `on` and `service.beta.kubernetes.io/alibaba-cloud-loadbalancer-sticky-session-type` is set to `insert`. Unit: seconds. Valid values: 1 to 86400. For more information, see CreateLoadBalancerHTTPListener and CreateLoadBalancerHTTPSListener.	v1.9.3 and later
service.beta.kubernetes.io/alibaba-cloud-loadbalancer-cookie	The name of the cookie that is configured on the server. The name must be 1 to 200 characters in length and can contain only ASCII letters and digits. It cannot contain commas (,), semicolons (;), or spaces. It cannot start with a dollar sign ($). This parameter is required when `service.beta.kubernetes.io/alibaba-cloud-loadbalancer-sticky-session` is set to `on` and `service.beta.kubernetes.io/alibaba-cloud-loadbalancer-sticky-session-type` is set to `server`. For more information, see CreateLoadBalancerHTTPListener and CreateLoadBalancerHTTPSListener.	v1.9.3 and later
service.beta.kubernetes.io/alibaba-cloud-loadbalancer-protocol-port	Separate multiple values with commas (,), for example, `https:443,http:80`.	v1.9.3 and later

apiVersion: v1
kind: Service
metadata:
  annotations:
    service.beta.kubernetes.io/alibaba-cloud-loadbalancer-sticky-session: "on"
    service.beta.kubernetes.io/alibaba-cloud-loadbalancer-sticky-session-type: "insert"
    service.beta.kubernetes.io/alibaba-cloud-loadbalancer-cookie-timeout: "1800"
    service.beta.kubernetes.io/alibaba-cloud-loadbalancer-protocol-port: "http:80"
  name: nginx
  namespace: default
spec:
  ports:
  - port: 80
    protocol: TCP
    targetPort: 80
  selector:
    run: nginx
  type: LoadBalancer

Configure an access control policy group for an SLB instance

Annotation: Multiple, as shown in the following table. The following three annotations are required.

Before you use this annotation to create an SLB instance with access control enabled, you must create an access control policy group in the Classic Load Balancer (CLB) console and record its ID (acl-id).

Annotation	Description	Supported CCM version
service.beta.kubernetes.io/alibaba-cloud-loadbalancer-acl-status	Specifies whether to enable access control. Valid values: `on` `off` Default value: `off`	v1.9.3.164-g2105d2e-aliyun and later
service.beta.kubernetes.io/alibaba-cloud-loadbalancer-acl-id	The ID of the access control policy group that is bound to the listener. This parameter is required when `service.beta.kubernetes.io/alibaba-cloud-loadbalancer-acl-status` is set to `"on"`.	v1.9.3.164-g2105d2e-aliyun and later
service.beta.kubernetes.io/alibaba-cloud-loadbalancer-acl-type	The type of access control. Valid values: `white`: forwards only requests from the IP addresses or CIDR blocks that are specified in the selected access control policy group. A whitelist is suitable for scenarios in which an application allows access only from specific IP addresses. Using a whitelist may pose risks to your services. After you set a whitelist, only IP addresses in the whitelist can access the SLB listener. If you enable a whitelist but do not add any IP addresses to the access control policy group, the SLB listener forwards all requests. `black`: does not forward any requests from the IP addresses or CIDR blocks that are specified in the selected access control policy group. A blacklist is suitable for scenarios in which an application denies access from specific IP addresses. If you enable a blacklist but do not add any IP addresses to the access control policy group, the SLB listener forwards all requests. This parameter is required when the AclStatus parameter is set to on.	v1.9.3.164-g2105d2e-aliyun and later

apiVersion: v1
kind: Service
metadata:
  annotations:
    service.beta.kubernetes.io/alibaba-cloud-loadbalancer-acl-status: "on"
    service.beta.kubernetes.io/alibaba-cloud-loadbalancer-acl-id: "${YOUR_ACL_ID}" # You cannot configure multiple policy groups.
    service.beta.kubernetes.io/alibaba-cloud-loadbalancer-acl-type: "white"
  name: nginx
  namespace: default
spec:
  ports:
  - port: 443
    protocol: TCP
    targetPort: 443
  selector:
    run: nginx
  type: LoadBalancer

Specify a redirection port for an SLB instance

Port forwarding redirects requests from an HTTP port to an HTTPS port.

Annotation: Multiple, as shown in the following table. The following three annotations are required.

Annotation	Description	Supported CCM version
service.beta.kubernetes.io/alibaba-cloud-loadbalancer-protocol-port	Separate multiple values with commas (,), for example, `https:443,http:80`.	v1.9.3 and later
service.beta.kubernetes.io/alibaba-cloud-loadbalancer-cert-id	The ID of the certificate on Alibaba Cloud. Log on to the CLB console and view the certificate ID on the Certificate Management page. Note To create a certificate, see Select an Alibaba Cloud-issued certificate.	v1.9.3.164-g2105d2e-aliyun and later
service.beta.kubernetes.io/alibaba-cloud-loadbalancer-forward-port	Redirects HTTP requests to a specified HTTPS port, for example, `80:443`.	v1.9.3.164-g2105d2e-aliyun and later

apiVersion: v1
kind: Service
metadata:
  annotations:
    service.beta.kubernetes.io/alibaba-cloud-loadbalancer-protocol-port: "https:443,http:80"
    service.beta.kubernetes.io/alibaba-cloud-loadbalancer-cert-id: "${YOUR_CERT_ID}"
    service.beta.kubernetes.io/alibaba-cloud-loadbalancer-forward-port: "80:443"
  name: nginx
  namespace: default
spec:
  ports:
  - name: https
    port: 443
    protocol: TCP
    targetPort: 80
  - name: http
    port: 80
    protocol: TCP
    targetPort: 80
  selector:
    run: nginx
  type: LoadBalancer

Set the scheduling algorithm for an SLB instance

Annotation: service.beta.kubernetes.io/alibaba-cloud-loadbalancer-scheduler

Description

Supported CCM version

The scheduling algorithm. Valid values:

wrr: weighted round-robin. Backend servers with higher weights receive more requests.
rr: round-robin. Requests are distributed to backend servers in sequence.
sch: source IP hashing. Requests from the same source IP address are distributed to the same backend server.
tch: four-tuple hashing. Requests from the same stream (identified by source IP address, destination IP address, source port, and destination port) are distributed to the same backend server.

Default value: rr.

For more information about the valid values of this parameter, see the Scheduler field in the API documentation for creating the corresponding listener type, such as Create a TCP listener.

v1.9.3 and later

apiVersion: v1
kind: Service
metadata:
  annotations:
    service.beta.kubernetes.io/alibaba-cloud-loadbalancer-scheduler: "wrr"
  name: nginx
  namespace: default
spec:
  ports:
  - port: 443
    protocol: TCP
    targetPort: 443
  selector:
    run: nginx
  type: LoadBalancer

Create a UDP listener

apiVersion: v1
kind: Service
metadata:
  name: nginx
  namespace: default
spec:
  ports:
  - port: 80
    protocol: UDP
    targetPort: 80
  selector:
    run: nginx
  type: LoadBalancer

Create an HTTP listener

Annotation: service.beta.kubernetes.io/alibaba-cloud-loadbalancer-protocol-port

Description	Supported CCM version
Separate multiple values with commas (,), for example, `https:443,http:80`.	v1.9.3 and later

apiVersion: v1
kind: Service
metadata:
  annotations:
    service.beta.kubernetes.io/alibaba-cloud-loadbalancer-protocol-port: "http:80"
  name: nginx
  namespace: default
spec:
  ports:
  - port: 80
    protocol: TCP
    targetPort: 80
  selector:
    run: nginx
  type: LoadBalancer

Create an HTTPS listener

Annotation: Multiple, as shown in the following table.

HTTPS requests are decrypted at the CLB layer and then sent to backend pods as HTTP requests.

Annotation

Description

Supported CCM version

service.beta.kubernetes.io/alibaba-cloud-loadbalancer-protocol-port

Separate multiple values with commas (,), for example, https:443,http:80.

v1.9.3 and later

service.beta.kubernetes.io/alibaba-cloud-loadbalancer-cert-id

The ID of the certificate on Alibaba Cloud.

Log on to the CLB console and view the certificate ID on the Certificate Management page.

Note

To create a certificate, see Select an Alibaba Cloud-issued certificate.

v1.9.3.164-g2105d2e-aliyun and later

apiVersion: v1
kind: Service
metadata:
  annotations:
    service.beta.kubernetes.io/alibaba-cloud-loadbalancer-protocol-port: "https:443"
    service.beta.kubernetes.io/alibaba-cloud-loadbalancer-cert-id: "${YOUR_CERT_ID}"
  name: nginx
  namespace: default
spec:
  ports:
  - port: 443
    protocol: TCP
    targetPort: 80
  selector:
    run: nginx
  type: LoadBalancer

SSL protocol error occurs when you access the CLB instance from within the cluster

Symptoms

After you create an HTTPS listener for a Service, you can access the Service from outside the cluster. However, when you run the curl command to access the HTTPS port of the CLB instance associated with the Service from a node or pod within the cluster, the following error is returned:

curl: (35) error:1408F10B:SSL routines:ssl3_get_record:wrong version number

Cause

This issue is caused by the IPVS rules on the node:

Service configuration: The HTTPS listener of the CLB instance supports only HTTP as a backend service. Therefore, the Service can only forward traffic from port: 443 to targetPort: 80. As a result, ACK creates an IPVS rule on the node to directly forward traffic destined for port 443 to the backend port 80.
Layer 4 forwarding: IPVS works at Layer 4 of the TCP/IP protocol stack. It only forwards TCP packets and does not parse application-layer protocols such as TLS or HTTPS.
Protocol mismatch: The HTTPS request (TLS handshake data) initiated by the client (curl) is directly forwarded by IPVS to the backend service's HTTP port 80. Because this port is not configured for TLS, it cannot parse the TLS request. As a result, an HTTP 400 error is returned, and the client reports an SSL protocol error.

Solution

Add the service.beta.kubernetes.io/alibaba-cloud-loadbalancer-hostname annotation to the Service. This annotation prevents the generation of IPVS rules on the node and forces traffic within the cluster to be routed through the CLB instance. This ensures that TLS is correctly processed on the CLB instance. For more information about the detailed steps, see Set a hostname for a Service.

Create a listener with a health check

Configure a TCP health check

Annotation: Multiple, as shown in the following table. All of the following annotations are required.

Health checks are enabled for TCP ports by default.

Annotation	Description	Supported CCM version
service.beta.kubernetes.io/alibaba-cloud-loadbalancer-health-check-switch	Specifies whether to enable health checks for TCP and UDP listeners. Valid values: `on` `off` Default value: `on`	v2.6.0 and later
service.beta.kubernetes.io/alibaba-cloud-loadbalancer-health-check-type	The health check type. Valid values: `tcp` `http` Default value: `tcp`. For more information, see CreateLoadBalancerTCPListener.	v1.9.3 and later
service.beta.kubernetes.io/alibaba-cloud-loadbalancer-health-check-connect-timeout	The period of time to wait for a response from a health check. This parameter applies to TCP health checks. If a backend ECS instance does not respond within the specified period of time, the health check fails. Unit: seconds. Valid values: 1 to 300. If the value of `service.beta.kubernetes.io/alibaba-cloud-loadbalancer-health-check-connect-timeout` is smaller than the value of `service.beta.kubernetes.io/alibaba-cloud-loadbalancer-health-check-interval`, the former is invalid and the timeout period is the value of the latter. Default value: 5. For more information, see CreateLoadBalancerTCPListener.	v1.9.3 and later
service.beta.kubernetes.io/alibaba-cloud-loadbalancer-healthy-threshold	The number of consecutive successful health checks required before the health check status of a backend server is changed from `fail` to `success`. Valid values: 2 to 10. Default value: 3. For more information, see CreateLoadBalancerTCPListener.	v1.9.3 and later
service.beta.kubernetes.io/alibaba-cloud-loadbalancer-unhealthy-threshold	The number of consecutive failed health checks required before the health check status of a backend server is changed from success to fail. Valid values: 2 to 10. Default value: 3. For more information, see CreateLoadBalancerTCPListener.	v1.9.3 and later
service.beta.kubernetes.io/alibaba-cloud-loadbalancer-health-check-interval	The interval between two consecutive health checks. Unit: seconds. Valid values: 1 to 50. Default value: 2. For more information, see CreateLoadBalancerTCPListener.	v1.9.3 and later

apiVersion: v1
kind: Service
metadata:
  annotations:
    service.beta.kubernetes.io/alibaba-cloud-loadbalancer-health-check-switch: "on"
    service.beta.kubernetes.io/alibaba-cloud-loadbalancer-health-check-type: "tcp"
    service.beta.kubernetes.io/alibaba-cloud-loadbalancer-health-check-connect-timeout: "8"
    service.beta.kubernetes.io/alibaba-cloud-loadbalancer-healthy-threshold: "4"
    service.beta.kubernetes.io/alibaba-cloud-loadbalancer-unhealthy-threshold: "4"
    service.beta.kubernetes.io/alibaba-cloud-loadbalancer-health-check-interval: "3"
  name: nginx
  namespace: default
spec:
  ports:
  - port: 80
    protocol: TCP
    targetPort: 80
  selector:
    run: nginx
  type: LoadBalancer

Configure a UDP health check

Annotation: Multiple, as shown in the following table. All of the following annotations are required.

Health checks are enabled for UDP ports by default.

Annotation	Description	Supported CCM version
service.beta.kubernetes.io/alibaba-cloud-loadbalancer-health-check-switch	Specifies whether to enable health checks for TCP and UDP listeners. Valid values: `on` `off` Default value: `on`	v2.6.0 and later
service.beta.kubernetes.io/alibaba-cloud-loadbalancer-health-check-connect-timeout	The period of time to wait for a response from a health check. This parameter applies to TCP health checks. If a backend ECS instance does not respond within the specified period of time, the health check fails. Unit: seconds. Valid values: 1 to 300. If the value of `service.beta.kubernetes.io/alibaba-cloud-loadbalancer-health-check-connect-timeout` is smaller than the value of `service.beta.kubernetes.io/alibaba-cloud-loadbalancer-health-check-interval`, the former is invalid and the timeout period is the value of the latter. Default value: 5. For more information, see CreateLoadBalancerUDPListener.	v1.9.3 and later
service.beta.kubernetes.io/alibaba-cloud-loadbalancer-healthy-threshold	The number of consecutive successful health checks required before the health check status of a backend server is changed from fail to success. Valid values: 2 to 10. Default value: 3. For more information, see CreateLoadBalancerUDPListener.	v1.9.3 and later
service.beta.kubernetes.io/alibaba-cloud-loadbalancer-unhealthy-threshold	The number of consecutive failed health checks required before the health check status of a backend server is changed from success to fail. Valid values: 2 to 10. Default value: 3. For more information, see CreateLoadBalancerUDPListener.	v1.9.3 and later
service.beta.kubernetes.io/alibaba-cloud-loadbalancer-health-check-interval	The interval between two consecutive health checks. Unit: seconds. Valid values: 1 to 50. Default value: 2. For more information, see CreateLoadBalancerUDPListener.	v1.9.3 and later

apiVersion: v1
kind: Service
metadata:
  annotations:
    service.beta.kubernetes.io/alibaba-cloud-loadbalancer-health-check-switch: "on"
    service.beta.kubernetes.io/alibaba-cloud-loadbalancer-health-check-interval: "5"
    service.beta.kubernetes.io/alibaba-cloud-loadbalancer-health-check-connect-timeout: "10"
    service.beta.kubernetes.io/alibaba-cloud-loadbalancer-healthy-threshold: "3"
    service.beta.kubernetes.io/alibaba-cloud-loadbalancer-unhealthy-threshold: "3"
  name: nginx
  namespace: default
spec:
  ports:
  - port: 80
    protocol: UDP
    targetPort: 80
  selector:
    run: nginx
  type: LoadBalancer

Disable health checks for TCP and UDP listeners

Annotation: service.beta.kubernetes.io/alibaba-cloud-loadbalancer-health-check-switch

Description

Supported CCM version

Specifies whether to enable health checks for TCP and UDP listeners. Valid values:

on
off

Default value: on

v2.6.0 and later

apiVersion: v1
kind: Service
metadata:
  annotations:
    service.beta.kubernetes.io/alibaba-cloud-loadbalancer-health-check-switch: "off" # Disable health checks.
  name: nginx
  namespace: default
spec:
  ports:
  - port: 443
    protocol: TCP
    targetPort: 443
  selector:
    run: nginx
  type: LoadBalancer

Configure an HTTP health check

Annotation: Multiple, as shown in the following table.

Annotation	Description	Supported CCM version
service.beta.kubernetes.io/alibaba-cloud-loadbalancer-health-check-flag	Valid values: `on`: For TCP listeners, the default value is `on` and cannot be changed. You do not need to change this annotation for TCP listeners. `off`: For HTTP listeners, the default value is `off`. Default value: `off`	v1.9.3 and later
service.beta.kubernetes.io/alibaba-cloud-loadbalancer-health-check-type	The health check type. Valid values: `tcp` `http` Default value: `tcp` For more information, see CreateLoadBalancerHTTPListener.	v1.9.3 and later
service.beta.kubernetes.io/alibaba-cloud-loadbalancer-health-check-uri	The URI that is used for health checks. You do not need to configure this annotation for TCP health checks.	v1.9.3 and later
service.beta.kubernetes.io/alibaba-cloud-loadbalancer-health-check-httpcode	The HTTP status codes that indicate a successful health check. Separate multiple status codes with commas (,). Valid values: `http_2xx` `http_3xx` `http_4xx` `http_5xx` Default value: `http_2xx` For more information, see CreateLoadBalancerHTTPListener.	v1.9.3 and later
service.beta.kubernetes.io/alibaba-cloud-loadbalancer-health-check-domain	The domain name that is used for health checks. Valid values: `$_ip`: the private IP address of a backend server. If you specify an IP address or do not specify this parameter, the SLB instance uses the private IP address of each backend server as the domain name for health checks. `domain`: The domain name must be 1 to 80 characters in length and can contain letters, digits, periods (.), and hyphens (-). For more information, see CreateLoadBalancerHTTPListener.	v1.9.3 and later
service.beta.kubernetes.io/alibaba-cloud-loadbalancer-health-check-timeout	The period of time to wait for a response from a health check. This parameter applies to HTTP health checks. If a backend ECS instance does not respond within the specified period of time, the health check fails. Unit: seconds. Valid values: 1 to 300. If the value of `service.beta.kubernetes.io/alibaba-cloud-loadbalancer-health-check-timeout` is smaller than the value of service.beta.kubernetes.io/alibaba-cloud-loadbalancer-health-check-interval, `service.beta.kubernetes.io/alibaba-cloud-loadbalancer-health-check-timeout` is invalid and the timeout period is the value of `service.beta.kubernetes.io/alibaba-cloud-loadbalancer-health-check-interval`. For more information, see CreateLoadBalancerHTTPListener.	v1.9.3 and later
service.beta.kubernetes.io/alibaba-cloud-loadbalancer-healthy-threshold	The number of consecutive successful health checks required before the health check status of a backend server is changed from fail to success. Valid values: 2 to 10. Default value: 3. For more information, see CreateLoadBalancerHTTPListener.	v1.9.3 and later
service.beta.kubernetes.io/alibaba-cloud-loadbalancer-unhealthy-threshold	The number of consecutive failed health checks required before the health check status of a backend server is changed from success to fail. Valid values: 2 to 10. Default value: 3. For more information, see CreateLoadBalancerHTTPListener.	v1.9.3 and later
service.beta.kubernetes.io/alibaba-cloud-loadbalancer-health-check-interval	The interval between two consecutive health checks. Unit: seconds. Valid values: 1 to 50. Default value: 2. For more information, see CreateLoadBalancerHTTPListener.	v1.9.3 and later
service.beta.kubernetes.io/alibaba-cloud-loadbalancer-protocol-port	Separate multiple values with commas (,), for example, `https:443,http:80`.	v1.9.3 and later
service.beta.kubernetes.io/alibaba-cloud-loadbalancer-health-check-method	The health check method for an HTTP listener. Valid values: `head` `get`	v2.3.0 and later

apiVersion: v1
kind: Service
metadata:
  annotations:
    service.beta.kubernetes.io/alibaba-cloud-loadbalancer-health-check-flag: "on"
    service.beta.kubernetes.io/alibaba-cloud-loadbalancer-health-check-type: "http"
    service.beta.kubernetes.io/alibaba-cloud-loadbalancer-health-check-uri: "/test/index.html"
    service.beta.kubernetes.io/alibaba-cloud-loadbalancer-healthy-threshold: "4"
    service.beta.kubernetes.io/alibaba-cloud-loadbalancer-unhealthy-threshold: "4"
    service.beta.kubernetes.io/alibaba-cloud-loadbalancer-health-check-timeout: "10"
    service.beta.kubernetes.io/alibaba-cloud-loadbalancer-health-check-interval: "3"
    service.beta.kubernetes.io/alibaba-cloud-loadbalancer-protocol-port: "http:80"
    # Optional. Set the HTTP status codes for the health check.
    service.beta.kubernetes.io/alibaba-cloud-loadbalancer-health-check-httpcode: "http_4xx"
    # Optional. Set the domain name for the health check.
    service.beta.kubernetes.io/alibaba-cloud-loadbalancer-health-check-domain: "www.aliyun.com"
    # Optional. Set the health check method.
    service.beta.kubernetes.io/alibaba-cloud-loadbalancer-health-check-method: "head"
  name: nginx
  namespace: default
spec:
  ports:
  - port: 80
    protocol: TCP
    targetPort: 80
  selector:
    run: nginx
  type: LoadBalancer

Configure connection draining for a listener

Annotation: Multiple, as shown in the following table. All of the following annotations are required.

This feature is supported only by TCP and UDP.

Annotation

Description

Supported CCM version

service.beta.kubernetes.io/alibaba-cloud-loadbalancer-connection-drain

Specifies whether to enable connection draining. Valid values:

on
off

v2.0.1 and later

service.beta.kubernetes.io/alibaba-cloud-loadbalancer-connection-drain-timeout

The connection draining timeout period. Unit: seconds. Valid values: 10 to 900.

v2.0.1 and later

apiVersion: v1
kind: Service
metadata:
  annotations:
    service.beta.kubernetes.io/alibaba-cloud-loadbalancer-connection-drain: "on"
    service.beta.kubernetes.io/alibaba-cloud-loadbalancer-connection-drain-timeout: "30"
  name: nginx
  namespace: default
spec:
  ports:
  - port: 80
    protocol: TCP
    targetPort: 80
  selector:
    run: nginx
  type: LoadBalancer

Configure additional request headers for a listener

Annotation: Multiple, as shown in the following table.

This feature is supported only by HTTP and HTTPS.

Annotation	Description	Supported CCM version
service.beta.kubernetes.io/alibaba-cloud-loadbalancer-protocol-port	Separate multiple values with commas (,), for example, `https:443,http:80`.	v1.9.3 and later
service.beta.kubernetes.io/alibaba-cloud-loadbalancer-xforwardedfor-proto	Specifies whether to use the X-Forwarded-Proto header to retrieve the listener protocol of the CLB instance. Valid values: `on` `off` Default value: `off`	v2.1.0 and later
service.beta.kubernetes.io/alibaba-cloud-loadbalancer-xforwardedfor-slbport	Specifies whether to use the X-Forwarded-For-SLBPORT header to retrieve the listener port of the SLB instance. Valid values: `on` `off` Default value: `off`	v2.9.1 and later
service.beta.kubernetes.io/alibaba-cloud-loadbalancer-xforwardedfor-clientsrcport	Specifies whether to use the X-Forwarded-For-ClientSrcPort header to retrieve the port of the client that accesses the SLB instance. Valid values: `on` `off` Default value: `off`	v2.9.1 and later

apiVersion: v1
kind: Service
metadata:
  annotations:
    service.beta.kubernetes.io/alibaba-cloud-loadbalancer-protocol-port: "http:80"
    service.beta.kubernetes.io/alibaba-cloud-loadbalancer-xforwardedfor-proto: "on"
    service.beta.kubernetes.io/alibaba-cloud-loadbalancer-xforwardedfor-slbport: "on"
    service.beta.kubernetes.io/alibaba-cloud-loadbalancer-xforwardedfor-clientsrcport: "on"
  name: nginx
  namespace: default
spec:
  ports:
  - port: 80
    protocol: TCP
    targetPort: 80
  selector:
    run: nginx
  type: LoadBalancer

Set the idle connection timeout period for a listener

Annotation: Multiple, as shown in the following table.

This feature is supported only by HTTP and HTTPS.

Annotation

Description

Supported CCM version

service.beta.kubernetes.io/alibaba-cloud-loadbalancer-protocol-port

Separate multiple values with commas (,), for example, https:443,http:80.

v1.9.3 and later

service.beta.kubernetes.io/alibaba-cloud-loadbalancer-idle-timeout

The idle connection timeout period for the listener. Unit: seconds. Valid values: 1 to 60.

Default value: 15

v2.1.0 and later

apiVersion: v1
kind: Service
metadata:
  annotations:
    service.beta.kubernetes.io/alibaba-cloud-loadbalancer-protocol-port: "http:80"
    service.beta.kubernetes.io/alibaba-cloud-loadbalancer-idle-timeout: "30"
  name: nginx
  namespace: default
spec:
  ports:
  - port: 80
    protocol: TCP
    targetPort: 80
  selector:
    run: nginx
  type: LoadBalancer

Disable the HTTP/2 feature for a listener

Annotation: Multiple, as shown in the following table.

This feature is supported only by HTTPS.

Annotation	Description	Supported CCM version
service.beta.kubernetes.io/alibaba-cloud-loadbalancer-protocol-port	Separate multiple values with commas (,), for example, `https:443,http:80`.	v1.9.3 and later
service.beta.kubernetes.io/alibaba-cloud-loadbalancer-cert-id	The ID of the certificate on Alibaba Cloud. Log on to the CLB console and view the certificate ID on the Certificate Management page. Note To create a certificate, see Select an Alibaba Cloud-issued certificate.	v1.9.3.164-g2105d2e-aliyun and later
service.beta.kubernetes.io/alibaba-cloud-loadbalancer-http2-enabled	Specifies whether to enable HTTP/2. Valid values: `on` `off` Default value: `on`	v2.1.0 and later

apiVersion: v1
kind: Service
metadata:
  annotations:
    service.beta.kubernetes.io/alibaba-cloud-loadbalancer-protocol-port: "https:443"
    service.beta.kubernetes.io/alibaba-cloud-loadbalancer-cert-id: "${YOUR_CERT_ID}"
    service.beta.kubernetes.io/alibaba-cloud-loadbalancer-http2-enabled: "off"
  name: nginx
  namespace: default
spec:
  ports:
  - port: 443
    protocol: TCP
    targetPort: 80
  selector:
    run: nginx
  type: LoadBalancer

Specify the request timeout period for a listener

Annotation: Multiple, as shown in the following table.

This feature is supported only by HTTP and HTTPS.

Annotation

Description

Supported CCM version

service.beta.kubernetes.io/alibaba-cloud-loadbalancer-protocol-port

Separate multiple values with commas (,), for example, https:443,http:80.

v1.9.3 and later

service.beta.kubernetes.io/alibaba-cloud-loadbalancer-request-timeout

The request timeout period. Unit: seconds. Valid values: 1 to 180. Default value: 60

If a backend server does not respond within the timeout period, the SLB instance stops waiting and returns an HTTP 504 error code to the client.

v2.3.0 and later

apiVersion: v1
kind: Service
metadata:
  annotations:
    service.beta.kubernetes.io/alibaba-cloud-loadbalancer-protocol-port: "http:80"
    service.beta.kubernetes.io/alibaba-cloud-loadbalancer-request-timeout: "60"
  name: nginx
  namespace: default
spec:
  ports:
  - port: 80
    protocol: TCP
    targetPort: 80
  selector:
    run: nginx
  type: LoadBalancer

Specify the connection timeout period for a listener

Annotation: service.beta.kubernetes.io/alibaba-cloud-loadbalancer-established-timeout

This feature is supported only by TCP.

Description	Supported CCM version
The connection timeout period. Unit: seconds. Valid values: 10 to 900. For more information, see CreateLoadBalancerTCPListener.	v2.3.0 and later

apiVersion: v1
kind: Service
metadata:
  annotations:
    service.beta.kubernetes.io/alibaba-cloud-loadbalancer-established-timeout: "60"
  name: nginx
  namespace: default
spec:
  ports:
  - port: 80
    protocol: TCP
    targetPort: 80
  selector:
    run: nginx
  type: LoadBalancer

Configure a security policy for a listener

Annotation: Multiple, as shown in the following table.

This feature is supported only by HTTPS.

Annotation	Description	Supported CCM version
service.beta.kubernetes.io/alibaba-cloud-loadbalancer-protocol-port	Separate multiple values with commas (,), for example, `https:443,http:80`.	v1.9.3 and later
service.beta.kubernetes.io/alibaba-cloud-loadbalancer-cert-id	The ID of the certificate on Alibaba Cloud. Log on to the CLB console and view the certificate ID on the Certificate Management page. Note To create a certificate, see Select an Alibaba Cloud-issued certificate.	v1.9.3.164-g2105d2e-aliyun and later
service.beta.kubernetes.io/alibaba-cloud-loadbalancer-tls-cipher-policy	A security policy includes the supported TLS versions and cipher suites for HTTPS. For more information, see CreateLoadBalancerHTTPSListener. Valid values: `tls_cipher_policy_1_0` `tls_cipher_policy_1_1` `tls_cipher_policy_1_2` `tls_cipher_policy_1_2_strict` `tls_cipher_policy_1_2_strict_with_1_3` Default value: `tls_cipher_policy_1_0`	v2.4.0 and later

apiVersion: v1
kind: Service
metadata:
  annotations:
    service.beta.kubernetes.io/alibaba-cloud-loadbalancer-protocol-port: "https:443,http:80"
    service.beta.kubernetes.io/alibaba-cloud-loadbalancer-cert-id: "${YOUR_CERT_ID}"
    service.beta.kubernetes.io/alibaba-cloud-loadbalancer-tls-cipher-policy: "tls_cipher_policy_1_2"
  name: nginx
  namespace: default
spec:
  ports:
  - name: https
    port: 443
    protocol: TCP
    targetPort: 443
  - name: http
    port: 80
    protocol: TCP
    targetPort: 80
  selector:
    run: nginx
  type: LoadBalancer

Configure TCP and UDP protocols for a listener at the same time

Note

This feature requires a Kubernetes cluster of v1.24 or later. For more information about how to upgrade a cluster, see Upgrade an ACK cluster.

apiVersion: v1
kind: Service
metadata:
  name: nginx
  namespace: default
spec:
  ports:
  - name: tcp
    port: 80
    protocol: TCP
    targetPort: 80
  - name: udp
    port: 80
    protocol: UDP
    targetPort: 81
  selector:
    app: nginx
  sessionAffinity: None
  type: LoadBalancer

Configure the Proxy Protocol for TCP and UDP listeners

Annotation: service.beta.kubernetes.io/alibaba-cloud-loadbalancer-proxy-protocol

Description

Supported CCM version

Configures the Proxy Protocol for TCP and UDP listeners. After you configure the Proxy Protocol, you can use it to pass client source IP addresses to backend servers. Valid values:

on
off

Default value: off

Important

This feature does not support smooth online migration. To switch the Proxy Protocol, you must stop your services for the upgrade. Proceed with caution.

v2.6.0 and later

apiVersion: v1
kind: Service
metadata:
  annotations:
    service.beta.kubernetes.io/alibaba-cloud-loadbalancer-proxy-protocol: "on"
  name: nginx
  namespace: default
spec:
  ports:
  - port: 443
    protocol: TCP
    targetPort: 443
  selector:
    run: nginx
  type: LoadBalancer

Typical operations for backend server groups

Use worker nodes with a specific label as backend servers

Annotation: service.beta.kubernetes.io/alibaba-cloud-loadbalancer-backend-label

Description	Supported CCM version
Specifies the worker nodes to be attached to the backend server group of the CLB instance based on labels. Separate multiple labels with commas (,), for example, `"k1=v1,k2=v2"`. Multiple labels have an AND relationship.	v1.9.3 and later

apiVersion: v1
kind: Service
metadata:
  annotations:
    service.beta.kubernetes.io/alibaba-cloud-loadbalancer-backend-label: "failure-domain.beta.kubernetes.io/zone=ap-southeast-5a"
  name: nginx
  namespace: default
spec:
  ports:
  - port: 443
    protocol: TCP
    targetPort: 443
  selector:
    run: nginx
  type: LoadBalancer

Use the nodes where pods reside as backend servers

Annotation: service.beta.kubernetes.io/alibaba-cloud-loadbalancer-scheduler

By default, externalTrafficPolicy is set to Cluster. In this mode, all nodes in the cluster are mounted to the backend server group. If you set externalTrafficPolicy to Local, only the nodes where the pods reside are used as backend servers.

Description

Supported CCM version

The scheduling algorithm. Valid values:

wrr: weighted round-robin. Backend servers with higher weights receive more requests.
In Local mode, you must set the scheduling algorithm to weighted round-robin (wrr).
Note
For CCM v1.9.3.164-g2105d2e-aliyun and later, if the external traffic policy of a Service is set to Local mode, the weights of nodes are automatically set based on the number of pods on the nodes. For more information about how node weights are calculated, see How are node weights automatically set in Local mode?.
rr: round-robin. Requests are distributed to backend servers in sequence.

Default value: rr

v1.9.3 and later

apiVersion: v1
kind: Service
metadata:
  annotations:
    service.beta.kubernetes.io/alibaba-cloud-loadbalancer-scheduler: "wrr"
  name: nginx
  namespace: default
spec:
  externalTrafficPolicy: Local
  ports:
  - port: 80
    protocol: TCP
    targetPort: 80
  selector:
    run: nginx
  type: LoadBalancer

Remove unschedulable nodes from the backend server group of the CLB instance

Annotation: service.beta.kubernetes.io/alibaba-cloud-loadbalancer-remove-unscheduled-backend

Description

Supported CCM version

Remove SchedulingDisabled nodes from the backend server group of the CLB instance. Valid values:

on: removes unschedulable nodes from the backend server group of the CLB instance.
off: The kubectl cordon and kubectl drain commands set a node to the unschedulable state. In this case, the unschedulable node is not removed from the backend server group of the CLB instance.

Default value: off

v1.9.3.164-g2105d2e-aliyun and later

apiVersion: v1
kind: Service
metadata:
  annotations:
    service.beta.kubernetes.io/alibaba-cloud-loadbalancer-remove-unscheduled-backend: "on"
  name: nginx
spec:
  externalTrafficPolicy: Local
  ports:
  - name: http
    port: 30080
    protocol: TCP
    targetPort: 80
  selector:
    app: nginx
  type: LoadBalancer

Directly attach pod ENIs to the backend server group of the CLB instance

Annotation: service.beta.kubernetes.io/backend-type

Description

Default value

Supported CCM version

The type of backend server for the CLB instance. Valid values:

eni: Pods are attached to the backend server group of the CLB instance. This improves network forwarding performance. This value is valid only in Terway network mode.
You can also manually edit service.beta.kubernetes.io/backend-type: "eni", changing eni to ecs to attach ECS instances to the CLB backend.
ecs: ECS instances are attached to the backend server group of the CLB instance.

Flannel network mode: ecs
Terway network mode:
- Terway clusters created before August 10, 2020: ecs
- Terway clusters created after August 10, 2020: eni

v1.9.3.164-g2105d2e-aliyun and later

apiVersion: v1
kind: Service
metadata:
  annotations:
    service.beta.kubernetes.io/backend-type: "eni"
  name: nginx
spec:
  ports:
  - name: http
    port: 30080
    protocol: TCP
    targetPort: 80
  selector:
    app: nginx
  type: LoadBalancer

Reuse an existing vServer group

Annotation: service.beta.kubernetes.io/alibaba-cloud-loadbalancer-vgroup-port

You can reuse an existing vServer group. This feature is valid only when you reuse an existing CLB instance. For more information about usage examples, see Deploy services across clusters by reusing an SLB instance.

Set the weight of a Service to receive traffic

Annotation: service.beta.kubernetes.io/alibaba-cloud-loadbalancer-weight

In scenarios where multiple Services reuse the same CLB instance, you can use this annotation to set the weight of the current Service to receive traffic. This annotation is valid only when you reuse an existing vServer group. For more information about usage examples, see Deploy services across clusters by reusing an SLB instance.

Ignore backend server weight updates

Annotation: service.beta.kubernetes.io/alibaba-cloud-loadbalancer-ignore-weight-update

Description

Supported CCM version

During Service synchronization, the update of backend server weights in the vServer group is skipped. This configuration is suitable for scenarios where you need to manually manage backend server weights using a mechanism other than CCM. Valid values:

on
off

Default value: off

v2.11.1 and later

apiVersion: v1
kind: Service
metadata:
  annotations:
    service.beta.kubernetes.io/alibaba-cloud-loadbalancer-ignore-weight-update: "on"
  name: nginx
  namespace: default
spec:
  ports:
  - port: 443
    protocol: TCP
    targetPort: 443
  selector:
    run: nginx
  type: LoadBalancer