All Products
Search
Document Center

Container Service for Kubernetes:Add annotations to the YAML file of a Service to configure a CLB instance

Last Updated:Jan 15, 2024

To use different load balancing features, you can add annotations to the YAML file of a Service to configure Classic Load Balancer (CLB) instances, listeners, and backend server groups.

Table of contents

Category

Hyperlink

Annotation usage notes

Common operations to manage CLB instances

Common operations to manage listeners

Common operations to manage backend server groups

References

Annotation usage notes

  • The values of annotations are case-sensitive.

  • Starting from September 11, 2019, the keyword alicloud in annotations is changed to alibaba-cloud.

    Example:

    Before the change: service.beta.kubernetes.io/alicloud-loadbalancer-id

    After the change: service.beta.kubernetes.io/alibaba-cloud-loadbalancer-id

    The system still supports annotations that use the alicloud keyword. You do not need to modify the existing annotations.

Common operations to manage CLB instances

Create an Internet-facing CLB instance

apiVersion: v1
kind: Service
metadata:
  name: nginx
  namespace: default
spec:
  ports:
  - port: 80
    protocol: TCP
    targetPort: 80
  selector:
    run: nginx
  type: LoadBalancer

Create an internal-facing CLB instance

Annotation: service.beta.kubernetes.io/alibaba-cloud-loadbalancer-address-type

Description

Default

Supported CCM version

To create an internal-facing CLB instance, set the value to intranet. Valid values:

  • internet: Use an Internet-facing CLB instance to expose the Service. This is the default value. The address type of the CLB instance must be set to Internet.

  • intranet: Use an internal-facing CLB instance to expose the Service. The address type of the CLB instance must be set to intranet.

internet

CCM 1.9.3 and later

apiVersion: v1
kind: Service
metadata:
  annotations:
    service.beta.kubernetes.io/alibaba-cloud-loadbalancer-address-type: "intranet"
  name: nginx
  namespace: default
spec:
  ports:
  - port: 80
    protocol: TCP
    targetPort: 80
  selector:
    run: nginx
  type: LoadBalancer

Create a CLB instance that uses the specified specifications

The following annotations are used.

Annotation

Description

Default

Supported CCM version

service.beta.kubernetes.io/alibaba-cloud-loadbalancer-charge-type

The metering method of the CLB instance. Valid values:

  • paybytraffic

  • paybybandwidth

paybytraffic

CCM 1.9.3 and later

service.beta.kubernetes.io/alibaba-cloud-loadbalancer-spec

The specifications of the CLB instance. For more information, see CreateLoadBalancer. You can use this annotation to create a CLB instance that uses the specified specifications or update the specifications of a CLB instance.

Important

If you use the CLB console to update the specifications of a CLB instance, the Cloud Controller Manager (CCM) may restore the CLB instance to the original specifications. Proceed with caution. You can use the CLB console to modify the specifications of only CLB instances that use the pay-by-specification metering method.

slb.s1.small

CCM 1.9.3 and later

apiVersion: v1
kind: Service
metadata:
  annotations:
    service.beta.kubernetes.io/alibaba-cloud-loadbalancer-instance-charge-type: "PayBySpec"
    service.beta.kubernetes.io/alibaba-cloud-loadbalancer-spec: "slb.s1.small"
  name: nginx
  namespace: default
spec:
  ports:
  - port: 443
    protocol: TCP
    targetPort: 443
  selector:
    run: nginx
  type: LoadBalancer

Use an existing CLB instance

Annotation: service.beta.kubernetes.io/alibaba-cloud-loadbalancer-id

Description

Default

Supported CCM version

The ID of the CLB instance. You can use this annotation to specify an existing CLB instance.

  • By default, if you use an existing CLB instance, the listeners of the CLB instance are not overwritten. To overwrite the existing listeners, set service.beta.kubernetes.io/alibaba-cloud-loadbalancer-force-override-listeners to "true".

    Note

    The following list explains why the CCM does not overwrite the listeners of an existing CLB instance:

    • If the listeners of the CLB instance are associated with applications, service interruptions may occur after the listeners are overwritten.

    • The CCM supports limited backend configurations and cannot handle complex configurations. To use complex configurations, you can create listeners in the console. The listeners do not overwrite the existing ones.

    Therefore, we recommend that you do not overwrite the listeners of an existing CLB instance. You can forcibly overwrite the listeners if the ports on which these listeners listen are no longer in use.

  • You cannot add additional tags (annotation: service.beta.kubernetes.io/alibaba-cloud-loadbalancer-additional-resource-tags) if you use an existing CLB instance.

None

CCM 1.9.3.81-gca19cd4-aliyun and later

apiVersion: v1
kind: Service
metadata:
  annotations:
    service.beta.kubernetes.io/alibaba-cloud-loadbalancer-id: "${YOUR_LOADBALANCER_ID}" # You can configure only one access control list (ACL). 
  name: nginx
  namespace: default
spec:
  ports:
  - port: 443
    protocol: TCP
    targetPort: 443
  selector:
    run: nginx
  type: LoadBalancer

Use an existing CLB instance and forcefully overwrite the listeners of the CLB instance

The following annotations are used. Use the following annotations to forcefully overwrite the listeners of a CLB instance. If a listener port conflict occurs, the original listener is deleted.

Annotation

Description

Default

Supported CCM version

service.beta.kubernetes.io/alibaba-cloud-loadbalancer-id

The ID of the CLB instance. You can use this annotation to specify an existing CLB instance.

  • By default, if you use an existing CLB instance, the listeners of the CLB instance are not overwritten. To overwrite the existing listeners, set service.beta.kubernetes.io/alibaba-cloud-loadbalancer-force-override-listeners to "true".

    Note

    The following list explains why the CCM does not overwrite the listeners of an existing CLB instance:

    • If the listeners of the CLB instance are associated with applications, service interruptions may occur after the listeners are overwritten.

    • The CCM supports limited backend configurations and cannot handle complex configurations. To use complex configurations, you can create listeners in the console. The listeners do not overwrite the existing ones.

    Therefore, we recommend that you do not overwrite the listeners of an existing CLB instance. You can forcibly overwrite the listeners if the ports on which these listeners listen are no longer in use.

  • You cannot add additional tags (annotation: service.beta.kubernetes.io/alibaba-cloud-loadbalancer-additional-resource-tags) if you use an existing CLB instance.

service.beta.kubernetes.io/alibaba-cloud-loadbalancer-force-override-listeners

Specifies whether to overwrite the listeners of an existing CLB instance. Valid values:

  • "true": Overwrite the listeners.

  • false: Do not overwrite the listeners.

Important

If you use an existing CLB instance and set forceoverride to true, do not associate a listener of the CLB instance with multiple Services. Otherwise, a configuration conflict occurs when these Services overwrite the listener.

"false"

CCM 1.9.3.81-gca19cd4-aliyun and later

apiVersion: v1
kind: Service
metadata:
  annotations:
    service.beta.kubernetes.io/alibaba-cloud-loadbalancer-id: "${YOUR_LOADBALANCER_ID}"
    service.beta.kubernetes.io/alibaba-cloud-loadbalancer-force-override-listeners: "true"
  name: nginx
  namespace: default
spec:
  ports:
  - port: 443
    protocol: TCP
    targetPort: 443
  selector:
    run: nginx
  type: LoadBalancer

Specify the primary zone and secondary zone when you create a CLB instance

The following annotations are used. After you specify the primary zone and secondary zone for a CLB instance, the zones cannot be changed.

CLB instances in some regions do not support the primary and secondary zones. The actual information on the CLB instance creation page shall prevail.

Annotation

Description

Default

Supported CCM version

service.beta.kubernetes.io/alibaba-cloud-loadbalancer-master-zoneid

The ID of the zone for the primary backend server.

None

CCM 1.9.3.10-gfb99107-aliyun and later

service.beta.kubernetes.io/alibaba-cloud-loadbalancer-slave-zoneid

The ID of the zone for the secondary backend server.

None

CCM 1.9.3.10-gfb99107-aliyun and later

apiVersion: v1
kind: Service
metadata:
  annotations:
    service.beta.kubernetes.io/alibaba-cloud-loadbalancer-master-zoneid: "cn-hangzhou-k"
    service.beta.kubernetes.io/alibaba-cloud-loadbalancer-slave-zoneid: "cn-hangzhou-j"
  name: nginx
  namespace: default
spec:
  ports:
  - port: 80
    protocol: TCP
    targetPort: 80
  selector:
    run: nginx
  type: LoadBalancer

Create a pay-by-bandwidth CLB instance

The following annotations are used. Both annotations are required.

Annotation

Description

Default

Supported CCM version

service.beta.kubernetes.io/alibaba-cloud-loadbalancer-charge-type

The metering method of the CLB instance. Valid values:

  • paybytraffic

  • paybybandwidth

paybytraffic

CCM 1.9.3 and later

service.beta.kubernetes.io/alibaba-cloud-loadbalancer-bandwidth

The maximum bandwidth of the CLB instance. This annotation applies only to Internet-facing CLB instances. For more information about the limits on metering methods for Internet-facing CLB instances, see Change the metering method of an Internet-facing CLB instance.

50

CCM 1.9.3.10-gfb99107-aliyun and later

apiVersion: v1
kind: Service
metadata:
  annotations:
    service.beta.kubernetes.io/alibaba-cloud-loadbalancer-charge-type: "paybybandwidth"
    service.beta.kubernetes.io/alibaba-cloud-loadbalancer-bandwidth: "2"
  name: nginx
  namespace: default
spec:
  ports:
  - port: 443
    protocol: TCP
    targetPort: 443
  selector:
    run: nginx
  type: LoadBalancer

Specify a vSwitch for a CLB instance

The following annotations are used. Both annotations are required.

Annotation

Description

Default

Supported CCM version

service.beta.kubernetes.io/alibaba-cloud-loadbalancer-address-type

To create an internal-facing CLB instance, set the value to intranet. Valid values:

  • internet: Use an Internet-facing CLB instance to expose the Service. This is the default value. The address type of the CLB instance must be set to Internet.

  • intranet: Use an internal-facing CLB instance to expose the Service. The address type of the CLB instance must be set to intranet.

internet

CCM 1.9.3 and later

service.beta.kubernetes.io/alibaba-cloud-loadbalancer-vswitch-id

The ID of the vSwitch in which the CLB instance resides. The vSwitch and the cluster must be deployed in the same virtual private cloud (VPC).

When you use this annotation, you must set service.beta.kubernetes.io/alibaba-cloud-loadbalancer-address-type to "intranet".

You can log on to the VPC console to query vSwitch IDs.

None

CCM 1.9.3 and later

apiVersion: v1
kind: Service
metadata:
  annotations:
   service.beta.kubernetes.io/alibaba-cloud-loadbalancer-address-type: "intranet"
   service.beta.kubernetes.io/alibaba-cloud-loadbalancer-vswitch-id: "${YOUR_VSWITCH_ID}"
  name: nginx
  namespace: default
spec:
  ports:
  - port: 443
    protocol: TCP
    targetPort: 443
  selector:
    run: nginx
  type: LoadBalancer

Specify an IP address for an internal-facing CLB instance

The following annotations are used. These annotations are required.

Annotation

Description

Default

Supported CCM version

service.beta.kubernetes.io/alibaba-cloud-loadbalancer-address-type

To create an internal-facing CLB instance, set the value to intranet. Valid values:

  • internet: Use an Internet-facing CLB instance to expose the Service. This is the default value. The address type of the CLB instance must be set to Internet.

  • intranet: Use an internal-facing CLB instance to expose the Service. The address type of the CLB instance must be set to intranet.

internet

CCM 1.9.3 and later

service.beta.kubernetes.io/alibaba-cloud-loadbalancer-vswitch-id

The ID of the vSwitch in which the CLB instance resides. The vSwitch and the cluster must be deployed in the same VPC.

When you use this annotation, you must set service.beta.kubernetes.io/alibaba-cloud-loadbalancer-address-type to intranet.

You can log on to the VPC console to query vSwitch IDs.

None

CCM 1.9.3 and later

service.beta.kubernetes.io/alibaba-cloud-loadbalancer-ip

The IP address of the internal-facing CLB instance.

  • The IP address must fall within the CIDR block of the vSwitch. Only IPv4 addresses are supported. This annotation must be used together with service.beta.kubernetes.io/alibaba-cloud-loadbalancer-vswitch-id.

  • The IP address cannot be changed after it is specified.

None

CCM 2.7.0 and later

apiVersion: v1
kind: Service
metadata:
 annotations:
  service.beta.kubernetes.io/alibaba-cloud-loadbalancer-address-type: "intranet"
  service.beta.kubernetes.io/alibaba-cloud-loadbalancer-vswitch-id: "${YOUR_VSWITCH_ID}"
  service.beta.kubernetes.io/alibaba-cloud-loadbalancer-ip: "192.168.x.x"
 name: nginx
 namespace: default
spec:
 type: LoadBalancer
 ports:
 - port: 80
   targetPort: 80
   name: http
 selector:
   app: nginx

Add additional tags to a CLB instance

Annotation: service.beta.kubernetes.io/alibaba-cloud-loadbalancer-additional-resource-tags

Description

Default

Supported CCM version

The tags to be added. Separate multiple tags with commas (,), such as "k1=v1,k2=v2".

None

CCM 1.9.3 and later

apiVersion: v1
kind: Service
metadata:
  annotations:
    service.beta.kubernetes.io/alibaba-cloud-loadbalancer-additional-resource-tags: "Key1=Value1,Key2=Value2" 
  name: nginx
  namespace: default
spec:
  ports:
  - port: 80
    protocol: TCP
    targetPort: 80
  selector:
    run: nginx
  type: LoadBalancer

Create an IPv6 CLB instance

Annotation: service.beta.kubernetes.io/alibaba-cloud-loadbalancer-ip-version

Description

Default

Supported CCM version

The IP version of the CLB instance. The IP version cannot be changed after it is specified. To use this annotation, make sure that the kube-proxy mode of the cluster is set to IPVS. Valid values:

  • ipv4

  • ipv6 The IPv6 address can be used only in an IPv6 network.

ipv4

CCM 1.9.3.220-g24b1885-aliyun and later

apiVersion: v1
kind: Service
metadata:
  annotations:
    service.beta.kubernetes.io/alibaba-cloud-loadbalancer-ip-version: "ipv6"
  name: nginx
spec:
  ports:
  - port: 80
    protocol: TCP
    targetPort: 80
  selector:
    app: nginx
  type: LoadBalancer

Enable deletion protection for a CLB instance

Annotation: service.beta.kubernetes.io/alibaba-cloud-loadbalancer-delete-protection

Description

Default

Supported CCM version

Specify whether to enable deletion protection for the CLB instance. Valid values:

  • on

  • off

Important

If you manually enable deletion protection in the CLB console for a CLB instance that is created for a LoadBalancer Service, you can run the kubectl delete svc {your-svc-name} command to delete the CLB instance.

on

CCM 1.9.3.313-g748f81e-aliyun and later

apiVersion: v1
kind: Service
metadata:
  annotations:
    service.beta.kubernetes.io/alibaba-cloud-loadbalancer-delete-protection: "on"
  name: nginx
spec:
  externalTrafficPolicy: Local
  ports:
  - port: 80
    protocol: TCP
    targetPort: 80
  selector:
    app: nginx
  type: LoadBalancer

Enable the configuration read-only mode for a CLB instance

Annotation: service.beta.kubernetes.io/alibaba-cloud-loadbalancer-modification-protection

Description

Default

Supported CCM version

Specify whether to enable configuration modification protection for the CLB instance. Valid values:

  • ConsoleProtection

  • NonProtection

ConsoleProtection

CCM 1.9.3.313-g748f81e-aliyun and later

apiVersion: v1
kind: Service
metadata:
  annotations:
    service.beta.kubernetes.io/alibaba-cloud-loadbalancer-modification-protection: "ConsoleProtection"
  name: nginx
spec:
  externalTrafficPolicy: Local
  ports:
  - port: 80
    protocol: TCP
    targetPort: 80
  selector:
    app: nginx
  type: LoadBalancer

Specify the name of a CLB instance

Annotation: service.beta.kubernetes.io/alibaba-cloud-loadbalancer-name

Description

Default

Supported CCM version

The name of the CLB instance. It must be 2 to 128 characters in length, and can contain letters, digits, periods (.), underscores (_), and hyphens (-). It must start with a letter.

None

CCM 1.9.3.313-g748f81e-aliyun and later

apiVersion: v1
kind: Service
metadata:
  annotations:
    service.beta.kubernetes.io/alibaba-cloud-loadbalancer-name: "your-svc-name"
  name: nginx
spec:
  externalTrafficPolicy: Local
  ports:
  - port: 80
    protocol: TCP
    targetPort: 80
  selector:
    app: nginx
  type: LoadBalancer

Specify a resource group to which a CLB instance belongs

Annotation: service.beta.kubernetes.io/alibaba-cloud-loadbalancer-resource-group-id

Description

Default

Supported CCM version

The ID of the resource group to which the CLB instance belongs. The resource group cannot be changed after it is specified. You can log on to the Resource Management console to query resource group IDs.

None

CCM 1.9.3.313-g748f81e-aliyun and later

apiVersion: v1
kind: Service
metadata:
  annotations:
    service.beta.kubernetes.io/alibaba-cloud-loadbalancer-resource-group-id: "rg-xxxx"
  name: nginx
spec:
  externalTrafficPolicy: Local
  ports:
  - port: 80
    protocol: TCP
    targetPort: 80
  selector:
    app: nginx
  type: LoadBalancer

Specify a hostname for a Service

The following annotations are used.

Annotation

Description

Default

Supported CCM version

service.beta.kubernetes.io/alibaba-cloud-loadbalancer-protocol-port

Separate multiple values with commas (,), such as https:443,http:80.

None

CCM 1.9.3 and later

service.beta.kubernetes.io/alibaba-cloud-loadbalancer-hostname

The hostname of the Service. The hostname must comply with the naming conventions of domain names.

Take note of the following items:

  • After you add this annotation, the EXTERNAL-IP of the Service changes from the default CLB IP address to your_service_name. If you access the CLB IP address from within the cluster, requests are first forwarded to the corresponding CLB instance.

  • After you add this annotation, if the listener protocol is TCP or UDP, a loop occurs when requests from within the cluster are forwarded to the CLB IP address. For more information, see Why does the client fail to access a CLB instance?.

  • This annotation does not automatically associate a domain name with the CLB instance. You can log on to the Domain Names console, purchase a domain name, and then associate the domain name with the CLB instance. For more information about how to purchase domain names, see Purchase domain names.

None

CCM 2.3.0 and later

apiVersion: v1
kind: Service
metadata:
  annotations:
    service.beta.kubernetes.io/alibaba-cloud-loadbalancer-protocol-port: "http:80"
    service.beta.kubernetes.io/alibaba-cloud-loadbalancer-hostname: "${your_service_hostname}"
  name: nginx-svc
  namespace: default
spec:
  ports:
  - name: http
    port: 80
    protocol: TCP
    targetPort: 80
  selector:
    app: nginx
  type: LoadBalancer

Expected output:

NAME         TYPE           CLUSTER-IP       EXTERNAL-IP            PORT(S)                      AGE
nginx-svc    loadBalancer   47.100.XX.XX     www.example.com        80:30248/TCP,443:32670/TCP   10s

Create a pay-by-LCU CLB instance

Annotation: service.beta.kubernetes.io/alibaba-cloud-loadbalancer-instance-charge-type

Description

Default

Supported CCM version

The metering method of the CLB instance. Valid values:

  • PayBySpec: The CLB instance is billed based on its specification. This is the default value.

  • PayByCLCU: The CLB instance is billed based on Loadbalancer Capacity Unit (LCUs).

Important

You cannot change the specification of a pay-by-LCU CLB instance. You cannot specify PayByCLCU and add the annotation service.beta.kubernetes.io/alibaba-cloud-loadbalancer-spec at the same time.

PayBySpec

CCM 2.4.0 and later

apiVersion: v1
kind: Service
metadata:
  annotations:
    service.beta.kubernetes.io/alibaba-cloud-loadbalancer-instance-charge-type: "PayByCLCU"
  name: nginx
  namespace: default
spec:
  ports:
  - port: 80
    protocol: TCP
    targetPort: 80
  selector:
    run: nginx
  type: LoadBalancer

Common operations to manage listeners

Specify the session persistence period for a TCP-based CLB instance

Annotation: service.beta.kubernetes.io/alibaba-cloud-loadbalancer-persistence-timeout

Description

Default

Supported CCM version

The session persistence period. This annotation applies only to TCP listeners. If a CLB instance has multiple TCP listeners, the configuration applies to all the TCP listeners.

Unit: seconds. Valid values: 0 to 3600. Default value: 0. By default, session persistence is disabled. For more information, see CreateLoadBalancerTCPListener.

0

CCM 1.9.3 and later

apiVersion: v1
kind: Service
metadata:
  annotations:
    service.beta.kubernetes.io/alibaba-cloud-loadbalancer-persistence-timeout: "1800"
  name: nginx
  namespace: default
spec:
  ports:
  - port: 443
    protocol: TCP
    targetPort: 443
  selector:
    run: nginx
  type: LoadBalancer

Enable session persistence for an HTTP or HTTPS listener by inserting a cookie

The following annotations are used. The following annotations are required if you insert a cookie to enable session persistence.

  • The following annotations apply only to HTTP-based and HTTPS-based CLB instances.

  • If a CLB instance has multiple HTTP or HTTPS listeners, the configuration applies to all the HTTP or HTTPS listeners.

Annotation

Description

Default

Supported CCM version

service.beta.kubernetes.io/alibaba-cloud-loadbalancer-sticky-session

Specify whether to enable session persistence. This annotation applies only to HTTP and HTTPS listeners. Valid values:

  • on

  • off

For more information, see CreateLoadBalancerHTTPListener and CreateLoadBalancerHTTPSListener.

off

CCM 1.9.3 and later

service.beta.kubernetes.io/alibaba-cloud-loadbalancer-sticky-session-type

The method that is used to process cookies. This annotation applies only to HTTP and HTTPS listeners. This annotation is required when service.beta.kubernetes.io/alibaba-cloud-loadbalancer-sticky-session is set to on. Valid values:

  • insert: Insert a cookie.

  • server: Rewrite a cookie.

For more information, see CreateLoadBalancerHTTPListener and CreateLoadBalancerHTTPSListener.

None

CCM 1.9.3 and later

service.beta.kubernetes.io/alibaba-cloud-loadbalancer-cookie-timeout

The timeout period of the cookie. This annotation is required when service.beta.kubernetes.io/alibaba-cloud-loadbalancer-sticky-session is set to on and service.beta.kubernetes.io/alibaba-cloud-loadbalancer-sticky-session-type is set to insert. Unit: seconds. Valid values: 1 to 86400.

For more information, see CreateLoadBalancerHTTPListener and CreateLoadBalancerHTTPSListener.

None

CCM 1.9.3 and later

service.beta.kubernetes.io/alibaba-cloud-loadbalancer-protocol-port

Separate multiple values with commas (,), such as https:443,http:80.

None

CCM 1.9.3 and later

apiVersion: v1
kind: Service
metadata:
  annotations:
    service.beta.kubernetes.io/alibaba-cloud-loadbalancer-sticky-session: "on"
    service.beta.kubernetes.io/alibaba-cloud-loadbalancer-sticky-session-type: "insert"
    service.beta.kubernetes.io/alibaba-cloud-loadbalancer-cookie-timeout: "1800"
    service.beta.kubernetes.io/alibaba-cloud-loadbalancer-protocol-port: "http:80"
  name: nginx
  namespace: default
spec:
  ports:
  - port: 80
    protocol: TCP
    targetPort: 80
  selector:
    run: nginx
  type: LoadBalancer

Configure an ACL for a CLB instance

The following annotations are used. These annotations are required.

Before you use this annotation to create a CLB instance that is configured with access control, you need to create an ACL in the CLB console and record the ID of the ACL.

Annotation

Description

Default

Supported CCM version

service.beta.kubernetes.io/alibaba-cloud-loadbalancer-acl-status

Specify whether to enable access control. Valid values:

  • on

  • off

off

CCM 1.9.3.164-g2105d2e-aliyun and later

service.beta.kubernetes.io/alibaba-cloud-loadbalancer-acl-id

The ID of the ACL that is associated with a listener. This annotation is required when service.beta.kubernetes.io/alibaba-cloud-loadbalancer-acl-status is set to "on".

None

CCM 1.9.3.164-g2105d2e-aliyun and later

service.beta.kubernetes.io/alibaba-cloud-loadbalancer-acl-type

The type of the ACL. Valid values:

  • white: a whitelist. Only requests from the IP addresses or CIDR blocks in the ACL are forwarded. You can use an IP address whitelist in scenarios in which you want to allow access from only specified IP addresses to an application. Risks may arise if a whitelist is improperly configured. After a whitelist is set, the CLB instance forwards only requests from the IP addresses in the whitelist. If you enable a whitelist but do not add an IP address to the ACL, the listener forwards all requests.

  • black: a blacklist. All requests from the IP addresses or CIDR blocks in the ACL are not forwarded. You can use an IP address blacklist in scenarios in which you want to deny access from only specified IP addresses to an application. If you enable an IP address blacklist but do not add an IP address to the blacklist, the listener forwards all requests. This annotation is required when the annotation service.beta.kubernetes.io/alibaba-cloud-loadbalancer-acl-status is set to on.

None

CCM 1.9.3.164-g2105d2e-aliyun and later

apiVersion: v1
kind: Service
metadata:
  annotations:
    service.beta.kubernetes.io/alibaba-cloud-loadbalancer-acl-status: "on"
    service.beta.kubernetes.io/alibaba-cloud-loadbalancer-acl-id: "${YOUR_ACL_ID}"
    service.beta.kubernetes.io/alibaba-cloud-loadbalancer-acl-type: "white"
  name: nginx
  namespace: default
spec:
  ports:
  - port: 443
    protocol: TCP
    targetPort: 443
  selector:
    run: nginx
  type: LoadBalancer

Configure port forwarding for a CLB instance

Port forwarding allows a CLB instance to forward requests from an HTTP port to an HTTPS port.

The following annotations are used. These annotations are required.

Annotation

Description

Default

Supported CCM version

service.beta.kubernetes.io/alibaba-cloud-loadbalancer-protocol-port

Separate multiple values with commas (,), such as https:443,http:80.

None

CCM 1.9.3 and later

service.beta.kubernetes.io/alibaba-cloud-loadbalancer-cert-id

The ID of a certificate for the CLB instance.

You can log on to the CLB console and view certificate IDs on the Certificates page.

Note

For more information about how to create a certificate, see Use a certificate from Alibaba Cloud SSL Certificates Service.

None

CCM 1.9.3.164-g2105d2e-aliyun and later

service.beta.kubernetes.io/alibaba-cloud-loadbalancer-forward-port

Forward requests from an HTTP port to an HTTPS port, such as 80:443.

None

CCM 1.9.3.164-g2105d2e-aliyun and later

apiVersion: v1
kind: Service
metadata:
  annotations:
    service.beta.kubernetes.io/alibaba-cloud-loadbalancer-protocol-port: "https:443,http:80"
    service.beta.kubernetes.io/alibaba-cloud-loadbalancer-cert-id: "${YOUR_CERT_ID}"
    service.beta.kubernetes.io/alibaba-cloud-loadbalancer-forward-port: "80:443"
  name: nginx
  namespace: default
spec:
  ports:
  - name: https
    port: 443
    protocol: TCP
    targetPort: 80
  - name: http
    port: 80
    protocol: TCP
    targetPort: 80
  selector:
    run: nginx
  type: LoadBalancer

Set the scheduling algorithm for a CLB instance

Annotation: service.beta.kubernetes.io/alibaba-cloud-loadbalancer-scheduler

Description

Default

Supported CCM version

The scheduling algorithm. Valid values:

  • wrr: Backend servers with higher weights receive more requests than backend servers with lower weights.

  • rr: The round-robin algorithm is used. Requests are forwarded to backend servers in sequence.

rr

CCM 1.9.3 and later

apiVersion: v1
kind: Service
metadata:
  annotations:
    service.beta.kubernetes.io/alibaba-cloud-loadbalancer-scheduler: "wrr"
  name: nginx
  namespace: default
spec:
  ports:
  - port: 443
    protocol: TCP
    targetPort: 443
  selector:
    run: nginx
  type: LoadBalancer

Create a UDP listener

apiVersion: v1
kind: Service
metadata:
  name: nginx
  namespace: default
spec:
  ports:
  - port: 80
    protocol: UDP
    targetPort: 80
  selector:
    run: nginx
  type: LoadBalancer

Create an HTTP listener

Annotation: service.beta.kubernetes.io/alibaba-cloud-loadbalancer-protocol-port

Description

Default

Supported CCM version

Separate multiple values with commas (,), such as https:443,http:80.

None

CCM 1.9.3 and later

apiVersion: v1
kind: Service
metadata:
  annotations:
    service.beta.kubernetes.io/alibaba-cloud-loadbalancer-protocol-port: "http:80"
  name: nginx
  namespace: default
spec:
  ports:
  - port: 80
    protocol: TCP
    targetPort: 80
  selector:
    run: nginx
  type: LoadBalancer

Create an HTTPS listener

The following annotations are used.

The HTTPS listeners of the CLB instance decrypt HTTPS requests into HTTP requests and forward the requests to pods on the backend servers.

Annotation

Description

Default

Supported CCM version

service.beta.kubernetes.io/alibaba-cloud-loadbalancer-protocol-port

Separate multiple values with commas (,), such as https:443,http:80.

None

CCM 1.9.3 and later

service.beta.kubernetes.io/alibaba-cloud-loadbalancer-cert-id

The ID of a certificate for the CLB instance.

You can log on to the CLB console and view certificate IDs on the Certificates page.

Note

For more information about how to create a certificate, see Use a certificate from Alibaba Cloud SSL Certificates Service.

None

CCM 1.9.3.164-g2105d2e-aliyun and later

apiVersion: v1
kind: Service
metadata:
  annotations:
    service.beta.kubernetes.io/alibaba-cloud-loadbalancer-protocol-port: "https:443"
    service.beta.kubernetes.io/alibaba-cloud-loadbalancer-cert-id: "${YOUR_CERT_ID}"
  name: nginx
  namespace: default
spec:
  ports:
  - port: 443
    protocol: TCP
    targetPort: 80
  selector:
    run: nginx
  type: LoadBalancer

Create a listener that has health checks enabled

Enable TCP health checks

The following annotations are used. These annotations are required.

By default, health checks are enabled for TCP ports.

Annotation

Description

Default

Supported CCM version

service.beta.kubernetes.io/alibaba-cloud-loadbalancer-health-check-switch

Specify whether to enable health checks for a TCP or UDP listener. Valid values:

  • on

  • off

on

V2.6.0 and later

service.beta.kubernetes.io/alibaba-cloud-loadbalancer-health-check-type

The type of the health check. Valid values:

  • tcp

  • http

For more information, see CreateLoadBalancerTCPListener.

tcp

CCM 1.9.3 and later

service.beta.kubernetes.io/alibaba-cloud-loadbalancer-health-check-connect-timeout

The timeout period of a health check. This annotation applies to TCP health checks. If a backend server, such as an Elastic Compute Service (ECS) instance, does not return a health check response within the specified timeout period, the server fails the health check. Unit: seconds. Valid values: 1 to 300.

If the value of service.beta.kubernetes.io/alibaba-cloud-loadbalancer-health-check-connect-timeout is smaller than that of service.beta.kubernetes.io/alibaba-cloud-loadbalancer-health-check-interval, the first annotation does not take effect. The timeout period is set to the value of the second annotation. For more information, see CreateLoadBalancerTCPListener.

5

CCM 1.9.3 and later

service.beta.kubernetes.io/alibaba-cloud-loadbalancer-healthy-threshold

The number of consecutive successful health checks that must occur before an unhealthy backend server is declared healthy.

Valid values: 2 to 10. For more information, see CreateLoadBalancerTCPListener.

3

CCM 1.9.3 and later

service.beta.kubernetes.io/alibaba-cloud-loadbalancer-unhealthy-threshold

The number of consecutive failed health checks that must occur before a healthy backend server is declared unhealthy. Valid values: 2 to 10. For more information, see CreateLoadBalancerTCPListener.

3

CCM 1.9.3 and later

service.beta.kubernetes.io/alibaba-cloud-loadbalancer-health-check-interval

The interval at which health checks are performed. Unit: seconds. Valid values: 1 to 50. For more information, see CreateLoadBalancerTCPListener.

2

CCM 1.9.3 and later

apiVersion: v1
kind: Service
metadata:
  annotations:
    service.beta.kubernetes.io/alibaba-cloud-loadbalancer-health-check-switch: "on"
    service.beta.kubernetes.io/alibaba-cloud-loadbalancer-health-check-type: "tcp"
    service.beta.kubernetes.io/alibaba-cloud-loadbalancer-health-check-connect-timeout: "8"
    service.beta.kubernetes.io/alibaba-cloud-loadbalancer-healthy-threshold: "4"
    service.beta.kubernetes.io/alibaba-cloud-loadbalancer-unhealthy-threshold: "4"
    service.beta.kubernetes.io/alibaba-cloud-loadbalancer-health-check-interval: "3"
  name: nginx
  namespace: default
spec:
  ports:
  - port: 80
    protocol: TCP
    targetPort: 80
  selector:
    run: nginx
  type: LoadBalancer

Enable UDP health checks

The following annotations are used. These annotations are required.

By default, health checks are enabled for UDP ports.

Annotation

Description

Default

Supported CCM version

service.beta.kubernetes.io/alibaba-cloud-loadbalancer-health-check-switch

Specify whether to enable health checks for a TCP or UDP listener. Valid values:

  • on

  • off

on

V2.6.0 and later

service.beta.kubernetes.io/alibaba-cloud-loadbalancer-health-check-connect-timeout

The timeout period of a health check. This annotation applies to TCP health checks. If a backend server, such as an ECS instance, does not return a health check response within the specified timeout period, the server fails the health check. Unit: seconds. Valid values: 1 to 300.

If the value of service.beta.kubernetes.io/alibaba-cloud-loadbalancer-health-check-connect-timeout is smaller than that of service.beta.kubernetes.io/alibaba-cloud-loadbalancer-health-check-interval, the first annotation does not take effect. The timeout period is set to the value of the second annotation. For more information, see CreateLoadBalancerTCPListener.

5

CCM 1.9.3 and later

service.beta.kubernetes.io/alibaba-cloud-loadbalancer-healthy-threshold

The number of consecutive successful health checks that must occur before an unhealthy backend server is declared healthy.

Valid values: 2 to 10. For more information, see CreateLoadBalancerTCPListener.

3

CCM 1.9.3 and later

service.beta.kubernetes.io/alibaba-cloud-loadbalancer-unhealthy-threshold

The number of consecutive failed health checks that must occur before a healthy backend server is declared unhealthy. Valid values: 2 to 10. For more information, see CreateLoadBalancerTCPListener.

3

CCM 1.9.3 and later

service.beta.kubernetes.io/alibaba-cloud-loadbalancer-health-check-interval

The interval at which health checks are performed. Unit: seconds. Valid values: 1 to 50. For more information, see CreateLoadBalancerTCPListener.

2

CCM 1.9.3 and later

apiVersion: v1
kind: Service
metadata:
  annotations:
    service.beta.kubernetes.io/alibaba-cloud-loadbalancer-health-check-switch: "on"
    service.beta.kubernetes.io/alibaba-cloud-loadbalancer-health-check-interval: "5"
    service.beta.kubernetes.io/alibaba-cloud-loadbalancer-health-check-connect-timeout: "10"
    service.beta.kubernetes.io/alibaba-cloud-loadbalancer-healthy-threshold: "3"
    service.beta.kubernetes.io/alibaba-cloud-loadbalancer-unhealthy-threshold: "3"
  name: nginx
  namespace: default
spec:
  ports:
  - port: 80
    protocol: UDP
    targetPort: 80
  selector:
    run: nginx
  type: LoadBalancer

Disable health checks for a TCP or UDP listener

Annotation: service.beta.kubernetes.io/alibaba-cloud-loadbalancer-protocol-port

Description

Default

Supported CCM version

Specify whether to enable health checks for a TCP or UDP listener. Valid values:

  • on

  • off

on

V2.6.0 and later

apiVersion: v1
kind: Service
metadata:
  annotations:
    service.beta.kubernetes.io/alibaba-cloud-loadbalancer-health-check-switch: "off" #Disable health checks.
  name: nginx
  namespace: default
spec:
  ports:
  - port: 443
    protocol: TCP
    targetPort: 443
  selector:
    run: nginx
  type: LoadBalancer

Enable HTTP health checks

The following annotations are used.

Annotation

Description

Default

Supported CCM version

service.beta.kubernetes.io/alibaba-cloud-loadbalancer-health-check-flag

Valid values:

  • on: The value is set to on for TCP listeners by default and cannot be changed. This annotation is not required if you use TCP listeners.

  • off: The value is set to off for HTTP listeners by default.

off

CCM 1.9.3 and later

service.beta.kubernetes.io/alibaba-cloud-loadbalancer-health-check-type

The type of the health check. Valid values:

  • tcp

  • http

For more information, see CreateLoadBalancerTCPListener.

tcp

CCM 1.9.3 and later

service.beta.kubernetes.io/alibaba-cloud-loadbalancer-health-check-uri

The Uniform Resource Identifier (URI) that you want to use for health checks. This annotation is not required when the type of the health check is set to TCP. For more information, see CreateLoadBalancerTCPListener.

None

CCM 1.9.3 and later

service.beta.kubernetes.io/alibaba-cloud-loadbalancer-health-check-connect-timeout

The timeout period of a health check. This annotation applies to TCP health checks. If a backend server, such as an ECS instance, does not return a health check response within the specified timeout period, the server fails the health check. Unit: seconds. Valid values: 1 to 300.

If the value of service.beta.kubernetes.io/alibaba-cloud-loadbalancer-health-check-connect-timeout is smaller than that of service.beta.kubernetes.io/alibaba-cloud-loadbalancer-health-check-interval, the first annotation does not take effect. The timeout period is set to the value of the second annotation. For more information, see CreateLoadBalancerTCPListener.

5

CCM 1.9.3 and later

service.beta.kubernetes.io/alibaba-cloud-loadbalancer-healthy-threshold

The number of consecutive successful health checks that must occur before an unhealthy backend server is declared healthy.

Valid values: 2 to 10. For more information, see CreateLoadBalancerTCPListener.

3

CCM 1.9.3 and later

service.beta.kubernetes.io/alibaba-cloud-loadbalancer-unhealthy-threshold

The number of consecutive failed health checks that must occur before a healthy backend server is declared unhealthy. Valid values: 2 to 10. For more information, see CreateLoadBalancerTCPListener.

3

CCM 1.9.3 and later

service.beta.kubernetes.io/alibaba-cloud-loadbalancer-health-check-interval

The interval at which health checks are performed. Unit: seconds. Valid values: 1 to 50. For more information, see CreateLoadBalancerTCPListener.

2

CCM 1.9.3 and later

service.beta.kubernetes.io/alibaba-cloud-loadbalancer-protocol-port

Separate multiple values with commas (,), such as https:443,http:80.

None

CCM 1.9.3 and later

service.beta.kubernetes.io/alibaba-cloud-loadbalancer-health-check-method

The health check method used by HTTP listeners. Valid values:

  • head

  • get

None

CCM 2.3.0 and later

apiVersion: v1
kind: Service
metadata:
  annotations:
    service.beta.kubernetes.io/alibaba-cloud-loadbalancer-health-check-flag: "on"
    service.beta.kubernetes.io/alibaba-cloud-loadbalancer-health-check-type: "http"
    service.beta.kubernetes.io/alibaba-cloud-loadbalancer-health-check-uri: "/test/index.html"
    service.beta.kubernetes.io/alibaba-cloud-loadbalancer-healthy-threshold: "4"
    service.beta.kubernetes.io/alibaba-cloud-loadbalancer-unhealthy-threshold: "4"
    service.beta.kubernetes.io/alibaba-cloud-loadbalancer-health-check-timeout: "10"
    service.beta.kubernetes.io/alibaba-cloud-loadbalancer-health-check-interval: "3"
    service.beta.kubernetes.io/alibaba-cloud-loadbalancer-protocol-port: "http:80"
    # Specify the health check method. This annotation is optional.
    service.beta.kubernetes.io/alibaba-cloud-loadbalancer-health-check-method: "head"
  name: nginx
  namespace: default
spec:
  ports:
  - port: 80
    protocol: TCP
    targetPort: 80
  selector:
    run: nginx
  type: LoadBalancer

Enable connection draining for a listener.

The following annotations are used. These annotations are required.

Only TCP and UDP are supported.

Annotation

Description

Default

Supported CCM version

service.beta.kubernetes.io/alibaba-cloud-loadbalancer-connection-drain

Specify whether to enable connection draining. Valid values:

  • on

  • off

None

CCM 2.0.1 and later

service.beta.kubernetes.io/alibaba-cloud-loadbalancer-connection-drain-timeout

The timeout period of connection draining. Unit: seconds. Valid values: 10 to 900.

None

CCM 2.0.1 and later

apiVersion: v1
kind: Service
metadata:
  annotations:
    service.beta.kubernetes.io/alibaba-cloud-loadbalancer-connection-drain: "on"
    service.beta.kubernetes.io/alibaba-cloud-loadbalancer-connection-drain-timeout: "30"
  name: nginx
  namespace: default
spec:
  ports:
  - port: 80
    protocol: TCP
    targetPort: 80
  selector:
    run: nginx
  type: LoadBalancer

Use the X-Forwarded-Proto header to identify the listener protocol of a CLB instance.

The following annotations are used.

Only HTTP and HTTPS are supported.

Annotation

Description

Default

Supported CCM version

service.beta.kubernetes.io/alibaba-cloud-loadbalancer-protocol-port

Separate multiple values with commas (,), such as https:443,http:80.

None

CCM 1.9.3 and later

service.beta.kubernetes.io/alibaba-cloud-loadbalancer-xforwardedfor-proto

Specifies whether to use the X-Forwarded-Proto header to obtain the listener protocol of a CLB instance. Valid values:

  • on

  • off

off

CCM 2.1.0 and later

apiVersion: v1
kind: Service
metadata:
  annotations:
    service.beta.kubernetes.io/alibaba-cloud-loadbalancer-protocol-port: "http:80"
    service.beta.kubernetes.io/alibaba-cloud-loadbalancer-xforwardedfor-proto: "on"
  name: nginx
  namespace: default
spec:
  ports:
  - port: 80
    protocol: TCP
    targetPort: 80
  selector:
    run: nginx
  type: LoadBalancer

Specify the timeout period of idle connections for a listener

The following annotations are used.

Only HTTP and HTTPS are supported.

Annotation

Description

Default

Supported CCM version

service.beta.kubernetes.io/alibaba-cloud-loadbalancer-protocol-port

Separate multiple values with commas (,), such as https:443,http:80.

None

CCM 1.9.3 and later

service.beta.kubernetes.io/alibaba-cloud-loadbalancer-idle-timeout

Specify the timeout period of idle connections for a listener. Unit: seconds. Valid values: 1 to 60.

15

CCM 2.1.0 and later

apiVersion: v1
kind: Service
metadata:
  annotations:
    service.beta.kubernetes.io/alibaba-cloud-loadbalancer-protocol-port: "http:80"
    service.beta.kubernetes.io/alibaba-cloud-loadbalancer-idle-timeout: "30"
  name: nginx
  namespace: default
spec:
  ports:
  - port: 80
    protocol: TCP
    targetPort: 80
  selector:
    run: nginx
  type: LoadBalancer

Disable HTTP/2 for a listener.

The following annotations are used.

Only HTTPS is supported.

Annotation

Description

Default

Supported CCM version

service.beta.kubernetes.io/alibaba-cloud-loadbalancer-protocol-port

Separate multiple values with commas (,), such as https:443,http:80.

None

CCM 1.9.3 and later

service.beta.kubernetes.io/alibaba-cloud-loadbalancer-cert-id

The ID of a certificate for the CLB instance.

You can log on to the CLB console and view certificate IDs on the Certificates page.

Note

For more information about how to create a certificate, see Use a certificate from Alibaba Cloud SSL Certificates Service.

None

CCM 1.9.3.164-g2105d2e-aliyun and later

service.beta.kubernetes.io/alibaba-cloud-loadbalancer-http2-enabled

Specify whether to enable HTTP/2. Valid values:

  • on

  • off

on

CCM 2.1.0 and later

apiVersion: v1
kind: Service
metadata:
  annotations:
    service.beta.kubernetes.io/alibaba-cloud-loadbalancer-protocol-port: "https:443"
    service.beta.kubernetes.io/alibaba-cloud-loadbalancer-cert-id: "${YOUR_CERT_ID}"
    service.beta.kubernetes.io/alibaba-cloud-loadbalancer-http2-enabled: "off"
  name: nginx
  namespace: default
spec:
  ports:
  - port: 443
    protocol: TCP
    targetPort: 80
  selector:
    run: nginx
  type: LoadBalancer

Specify the timeout period of requests for a listener.

The following annotations are used.

Only HTTP and HTTPS are supported.

Annotation

Description

Default

Supported CCM version

service.beta.kubernetes.io/alibaba-cloud-loadbalancer-protocol-port

Separate multiple values with commas (,), such as https:443,http:80.

None

CCM 1.9.3 and later

service.beta.kubernetes.io/alibaba-cloud-loadbalancer-request-timeout

The timeout period of requests. Unit: seconds. Valid values: 1 to 180.

If the backend servers do not respond within the timeout period, the CLB instance stops waiting and returns an HTTP 504 error to the client.

60

CCM 2.3.0 and later

apiVersion: v1
kind: Service
metadata:
  annotations:
    service.beta.kubernetes.io/alibaba-cloud-loadbalancer-protocol-port: "http:80"
    service.beta.kubernetes.io/alibaba-cloud-loadbalancer-request-timeout: "60"
  name: nginx
  namespace: default
spec:
  ports:
  - port: 80
    protocol: TCP
    targetPort: 80
  selector:
    run: nginx
  type: LoadBalancer

Specify the timeout period of connections for a listener.

Annotation: service.beta.kubernetes.io/alibaba-cloud-loadbalancer-established-timeout

Only TCP is supported.

Description

Default

Supported CCM version

The timeout period of connections. Unit: seconds. Valid values: 10 to 900. For more information, see CreateLoadBalancerTCPListener.

None

CCM 2.3.0 and later

apiVersion: v1
kind: Service
metadata:
  annotations:
    service.beta.kubernetes.io/alibaba-cloud-loadbalancer-established-timeout: "60"
  name: nginx
  namespace: default
spec:
  ports:
  - port: 80
    protocol: TCP
    targetPort: 80
  selector:
    run: nginx
  type: LoadBalancer

Configure security policies for a listener

The following annotations are used.

Only HTTPS is supported.

Annotation

Description

Default

Supported CCM version

service.beta.kubernetes.io/alibaba-cloud-loadbalancer-protocol-port

Separate multiple values with commas (,), such as https:443,http:80.

None

CCM 1.9.3 and later

service.beta.kubernetes.io/alibaba-cloud-loadbalancer-cert-id

The ID of a certificate for the CLB instance.

You can log on to the CLB console and view certificate IDs on the Certificates page.

Note

For more information about how to create a certificate, see Use a certificate from Alibaba Cloud SSL Certificates Service.

None

CCM 1.9.3.164-g2105d2e-aliyun and later

service.beta.kubernetes.io/alibaba-cloud-loadbalancer-tls-cipher-policy

The TLS security policy. Each security policy contains TLS protocol versions and cipher suites available for HTTPS. For more information, see CreateLoadBalancerHTTPSListener. Valid values:

  • tls_cipher_policy_1_0

  • tls_cipher_policy_1_1

  • tls_cipher_policy_1_2

  • tls_cipher_policy_1_2_strict

  • tls_cipher_policy_1_2_strict_with_1_3

tls_cipher_policy_1_0

CCM 2.4.0 and later

apiVersion: v1
kind: Service
metadata:
  annotations:
    service.beta.kubernetes.io/alibaba-cloud-loadbalancer-protocol-port: "https:443,http:80"
    service.beta.kubernetes.io/alibaba-cloud-loadbalancer-cert-id: "${YOUR_CERT_ID}"
    service.beta.kubernetes.io/alibaba-cloud-loadbalancer-tls-cipher-policy: "tls_cipher_policy_1_2"
  name: nginx
  namespace: default
spec:
  ports:
  - name: https
    port: 443
    protocol: TCP
    targetPort: 443
  - name: http
    port: 80
    protocol: TCP
    targetPort: 80
  selector:
    run: nginx
  type: LoadBalancer

Configure both TCP and UDP for a listener

Note

Only clusters whose Kubernetes version is 1.24 and later support this feature. For more information about how to update a cluster, see Update the Kubernetes version of an ACK cluster.

apiVersion: v1
kind: Service
metadata:
  name: nginx
  namespace: default
spec:
  ports:
  - name: tcp
    port: 80
    protocol: TCP
    targetPort: 80
  - name: udp
    port: 80
    protocol: UDP
    targetPort: 81
  selector:
    app: nginx
  sessionAffinity: None
  type: LoadBalancer

Configure Proxy Protocol for TCP and UDP listeners

Annotation: service.beta.kubernetes.io/alibaba-cloud-loadbalancer-proxy-protocol

Description

Default

Supported CCM version

Specify whether to enable Proxy Protocol for a TCP or UDP listener. After Proxy Protocol is configured, client IP addresses can be passed to backend servers. Valid values:

  • on

  • off

Important

Enabling this feature results in service interruptions. You must stop your applications before you enable Proxy Protocol. Proceed with caution.

off

V2.6.0 and later

apiVersion: v1
kind: Service
metadata:
  annotations:
    service.beta.kubernetes.io/alibaba-cloud-loadbalancer-proxy-protocol: "on"
  name: nginx
  namespace: default
spec:
  ports:
  - port: 443
    protocol: TCP
    targetPort: 443
  selector:
    run: nginx
  type: LoadBalancer

Common operations to manage backend server groups

Add worker nodes that have specified labels as the backend servers of a CLB instance

Annotation: service.beta.kubernetes.io/alibaba-cloud-loadbalancer-backend-label

Description

Default

Supported CCM version

Specify the labels of worker nodes that you want to add as the backend servers of the CLB instance. Separate multiple labels with commas (,), such as "k1=v1,k2=v2". The logical relationship among multiple labels is AND.

None

CCM 1.9.3 and later

apiVersion: v1
kind: Service
metadata:
  annotations:
    service.beta.kubernetes.io/alibaba-cloud-loadbalancer-backend-label: "failure-domain.beta.kubernetes.io/zone=ap-southeast-5a"
  name: nginx
  namespace: default
spec:
  ports:
  - port: 443
    protocol: TCP
    targetPort: 443
  selector:
    run: nginx
  type: LoadBalancer

Add the nodes on which pods run as the backend servers of a CLB instance

Annotation: service.beta.kubernetes.io/alibaba-cloud-loadbalancer-scheduler

By default, externalTrafficPolicy is set to Cluster for a Service. In Cluster mode, all nodes in the cluster are added as the backend servers of the CLB instance. In Local mode, only nodes where the pods are deployed are added as the backend servers of the CLB instance.

Description

Default

Supported CCM version

The scheduling algorithm. Valid values:

  • wrr: Backend servers with higher weights receive more requests than backend servers with lower weights.

    In Local mode, you must set the scheduling algorithm to weighted round-robin (WRR).

    Note

    For CCM V1.9.3.164-g2105d2e-aliyun and later, node weights are calculated based on the number of pods that run on each node for Services whose externalTrafficPolicy is set to Local. For more information about node weight calculation, see How does the CCM calculate node weights in Local mode?

  • rr: The round-robin algorithm is used. Requests are forwarded to backend servers in sequence.

rr

CCM 1.9.3 and later

apiVersion: v1
kind: Service
metadata:
  annotations:
    service.beta.kubernetes.io/alibaba-cloud-loadbalancer-scheduler: "wrr"
  name: nginx
  namespace: default
spec:
  externalTrafficPolicy: Local
  ports:
  - port: 80
    protocol: TCP
    targetPort: 80
  selector:
    run: nginx
  type: LoadBalancer

Remove unschedulable nodes from the backend server group of a CLB instance

Annotation: service.beta.kubernetes.io/alibaba-cloud-loadbalancer-scheduler

Description

Default

Supported CCM version

Specify whether to remove backend servers in the Unschedulable state from a CLB instance. Valid values:

  • on: Remove nodes in the unschedulable state from the backend server group of the CLB instance.

  • off: The kubectl cordon and kubectl drain commands will set nodes to the unschedulable state. In this scenario, nodes in the unschedulable state are not removed from the backend server group of the CLB instance.

off

CCM 1.9.3.164-g2105d2e-aliyun and later

apiVersion: v1
kind: Service
metadata:
  annotations:
    service.beta.kubernetes.io/alibaba-cloud-loadbalancer-remove-unscheduled-backend: "on"
  name: nginx
spec:
  externalTrafficPolicy: Local
  ports:
  - name: http
    port: 30080
    protocol: TCP
    targetPort: 80
  selector:
    app: nginx
  type: LoadBalancer

Add pods that are assigned ENIs as the backend servers of a CLB instance

Annotation: service.beta.kubernetes.io/backend-type

Description

Default

Supported CCM version

The type of resources that are added as the backend servers of the CLB instance. Valid values:

  • eni: Add pods as the backend servers of the CLB instance. This mode improves the network forwarding performance. This annotation takes effect only in Terway mode.

    You can also manually change service.beta.kubernetes.io/backend-type: "eni" from eni to ecs and then add ECS instances as the backend servers of the CLB instance.

  • ecs: Add ECS instances as the backend servers of the CLB instance.

  • Flannel mode: ecs.

  • When the Terway network plug-in is used:

    • Terway clusters that are created before August 10, 2020: ecs

    • Terway clusters that are created after August 10, 2020: eni

CCM 1.9.3.164-g2105d2e-aliyun and later

  apiVersion: v1
  kind: Service
  metadata:
    annotations:
      service.beta.kubernetes.io/backend-type: "eni"
    name: nginx
  spec:
    ports:
    - name: http
      port: 30080
      protocol: TCP
      targetPort: 80
    selector:
      app: nginx
    type: LoadBalancer

Reuse an existing vServer group

Annotation: service.beta.kubernetes.io/alibaba-cloud-loadbalancer-vgroup-port This annotation can be used to reuse a vServer group. This annotation takes effect only when you use an existing CLB instance. For more information, see Use the CCM to deploy services across clusters.

Specify weights for Services to enable weighted round robin

Annotation: service.beta.kubernetes.io/alibaba-cloud-loadbalancer-weight

When multiple Services use the same CLB instance, you can use this annotation to specify the percentage of traffic distributed to the current Service. This annotation takes effect only when an existing vServer group is reused. For more information, see Use the CCM to deploy services across clusters.