Annotations that you can use to configure an NLB instance for an ACK cluster - Container Service for Kubernetes

When you use a Network Load Balancer (NLB) instance for a Service, you can use annotations in the YAML file to configure advanced load balancing features, such as specifying the network type of the NLB instance, enabling configuration read-only mode, and configuring mutual authentication. This topic describes the annotations that you can use in a Service YAML file to perform common configurations for NLB instances, listeners, and server groups.

Index

Category	Feature category	Configuration link
Prerequisites
Common operations on NLB instances	Create a load balancer	Create a public-facing NLB instance Create a private NLB instance Create a dual-stack NLB instance Specify the network type of an IPv6 address Reuse an existing load balancer
	Instance configuration	Specify the load balancer name Specify the resource group of the load balancer Add additional tags to the load balancer Associate an Internet Shared Bandwidth instance
	Security configuration	Retain the automatically created load balancer when the Service is deleted
Common operations on listeners	Create a listener	Create a TCP listener Create a UDP listener Create a TCP/SSL listener Configure a listener port range
	Listener configuration	Configure TCP and UDP protocols for a listener at the same time Set the maximum number of new connections per second for a listener Configure Proxy Protocol Carry additional information in Proxy Protocol Set the idle connection timeout period for a listener Configure an ALPN policy
	Security configuration	Enable mutual authentication Configure a TLS security policy Configure a security group for a listener
Common operations on server groups	Server configuration	Configure a scheduling algorithm Configure connection draining Enable client IP address persistence Configure a TCP health check Configure an HTTP health check Configure the server group type Reuse an existing server group Set the weight of the Service to receive traffic Ignore backend server weight updates

Considerations

The cluster version is 1.24 or later and the Cloud Controller Manager (CCM) version is 2.5.0 or later. To upgrade the cluster, see Manually upgrade an ACK cluster. To upgrade components, see Manage components.
In the Service definition, set spec.loadBalancerClass to alibabacloud.com/nlb. If you do not specify this parameter, a Classic Load Balancer (CLB) instance is created by default.
After a Service is created, you cannot change the value of spec.loadBalancerClass. You cannot convert CLB resources to NLB resources, or vice versa.
You cannot manage NLB instances in the ACK console. You must use kubectl commands.

Common operations on NLB instances

Create a public-facing NLB instance

Annotation: service.beta.kubernetes.io/alibaba-cloud-loadbalancer-zone-maps

Description

Supported CCM versions

Specifies the zones of the NLB instance.

You can log on to the NLB console to view the regions and zones that NLB supports. At least two zones are required. Separate multiple zones with commas (,), for example, cn-hangzhou-k:vsw-i123456,cn-hangzhou-j:vsw-j654321.

v2.5.0 and later

apiVersion: v1
kind: Service
metadata:
  annotations:
    service.beta.kubernetes.io/alibaba-cloud-loadbalancer-zone-maps: "${zone-A}:${vsw-A},${zone-B}:${vsw-B}" # For example, cn-hangzhou-k:vsw-i123456,cn-hangzhou-j:vsw-j654321.
  name: nginx
  namespace: default
spec:
  externalTrafficPolicy: Local
  ports:
  - name: tcp
    port: 80
    protocol: TCP
    targetPort: 80
  - name: https
    port: 443
    protocol: TCP
    targetPort: 443
  selector:
    app: nginx
  loadBalancerClass: "alibabacloud.com/nlb"
  type: LoadBalancer

Create a private NLB instance

Annotation: service.beta.kubernetes.io/alibaba-cloud-loadbalancer-address-type

Description

Supported CCM versions

Specifies the NLB instance as a private NLB instance. You can change the value of this annotation to switch between a public-facing NLB instance and a private NLB instance. Valid values:

internet: a public-facing NLB instance.
intranet: a private NLB instance.

Default value: internet

v2.5.0 and later

apiVersion: v1
kind: Service
metadata:
  annotations:
    service.beta.kubernetes.io/alibaba-cloud-loadbalancer-zone-maps: "${zone-A}:${vsw-A},${zone-B}:${vsw-B}" # For example, cn-hangzhou-k:vsw-i123456,cn-hangzhou-j:vsw-j654321.
    service.beta.kubernetes.io/alibaba-cloud-loadbalancer-address-type: "intranet"
  name: nginx
  namespace: default
spec:
  externalTrafficPolicy: Local
  ports:
  - name: tcp
    port: 80
    protocol: TCP
    targetPort: 80
  - name: https
    port: 443
    protocol: TCP
    targetPort: 443
  selector:
    app: nginx
  loadBalancerClass: "alibabacloud.com/nlb"
  type: LoadBalancer

Specify the load balancer name

Annotation: service.beta.kubernetes.io/alibaba-cloud-loadbalancer-name

Description	Supported CCM versions
Specifies the name of the load balancer instance. The name must be 2 to 128 characters in length, start with a letter or a Chinese character, and can contain digits, periods (.), underscores (_), and hyphens (-).	v2.5.0 and later

apiVersion: v1
kind: Service
metadata:
  annotations:
    service.beta.kubernetes.io/alibaba-cloud-loadbalancer-zone-maps: "${zone-A}:${vsw-A},${zone-B}:${vsw-B}" # For example, cn-hangzhou-k:vsw-i123456,cn-hangzhou-j:vsw-j654321.
    service.beta.kubernetes.io/alibaba-cloud-loadbalancer-name: "${your-nlb-name}" # The name of the NLB instance.
  name: nginx
  namespace: default
spec:
  externalTrafficPolicy: Local
  ports:
  - name: tcp
    port: 80
    protocol: TCP
    targetPort: 80
  - name: https
    port: 443
    protocol: TCP
    targetPort: 443
  selector:
    app: nginx
  loadBalancerClass: "alibabacloud.com/nlb"
  type: LoadBalancer

Specify the resource group of the load balancer

Annotation: service.beta.kubernetes.io/alibaba-cloud-loadbalancer-resource-group-id

Description

Supported CCM versions

Specifies the resource group to which the load balancer belongs. The resource group ID cannot be changed after it is specified.

You can log on to the Alibaba Cloud Resource Management platform to query the resource group ID.

v2.5.0 and later

apiVersion: v1
kind: Service
metadata:
  annotations:
    service.beta.kubernetes.io/alibaba-cloud-loadbalancer-zone-maps: "${zone-A}:${vsw-A},${zone-B}:${vsw-B}" # For example, cn-hangzhou-k:vsw-i123456,cn-hangzhou-j:vsw-j654321.
    service.beta.kubernetes.io/alibaba-cloud-loadbalancer-resource-group-id:  "${your-resource-group-id}" # The ID of the resource group.
  name: nginx
  namespace: default
spec:
  externalTrafficPolicy: Local
  ports:
  - name: tcp
    port: 80
    protocol: TCP
    targetPort: 80
  - name: https
    port: 443
    protocol: TCP
    targetPort: 443
  selector:
    app: nginx
  loadBalancerClass: "alibabacloud.com/nlb"
  type: LoadBalancer

Create a dual-stack NLB instance

Annotation: service.beta.kubernetes.io/alibaba-cloud-loadbalancer-ip-version

Description

Supported CCM versions

Specifies the IP version of the NLB instance. The IP version cannot be changed after the NLB instance is created. The kube-proxy of the cluster must work in IPVS mode. Valid values:

ipv4: IPv4.
DualStack: dual-stack. Both IPv4 and IPv6 are supported.
- The two vSwitches specified in service.beta.kubernetes.io/alibaba-cloud-loadbalancer-zone-maps must have IPv6 enabled.
- The generated IPv6 address can be accessed only in an environment that supports IPv6.

Default value: ipv4

v2.5.0 and later

apiVersion: v1
kind: Service
metadata:
  annotations:
    service.beta.kubernetes.io/alibaba-cloud-loadbalancer-zone-maps: "${zone-A}:${vsw-A},${zone-B}:${vsw-B}" # For example, cn-hangzhou-k:vsw-i123456,cn-hangzhou-j:vsw-j654321.
    service.beta.kubernetes.io/alibaba-cloud-loadbalancer-ip-version: "DualStack"
  name: nginx
  namespace: default
spec:
  externalTrafficPolicy: Local
  ports:
  - name: tcp
    port: 80
    protocol: TCP
    targetPort: 80
  - name: https
    port: 443
    protocol: TCP
    targetPort: 443
  selector:
    app: nginx
  sessionAffinity: None
  loadBalancerClass: "alibabacloud.com/nlb"
  type: LoadBalancer

Add additional tags to the load balancer

Annotation: service.beta.kubernetes.io/alibaba-cloud-loadbalancer-additional-resource-tags

Description	Supported CCM versions
Allows you to add additional tags. Separate multiple tags with commas (,), for example, `k1=v1,k2=v2`. For CCM v2.10.0 and later, you can modify the tags of created instances and reused instances. Important If you add this annotation to a Service to specify additional tags, the modifications made to the tags of the corresponding load balancer instance in the console may be overwritten.	v2.5.0 and later

apiVersion: v1
kind: Service
metadata:
  annotations:
    service.beta.kubernetes.io/alibaba-cloud-loadbalancer-zone-maps: "${zone-A}:${vsw-A},${zone-B}:${vsw-B}" # For example, cn-hangzhou-k:vsw-i123456,cn-hangzhou-j:vsw-j654321.
    service.beta.kubernetes.io/alibaba-cloud-loadbalancer-additional-resource-tags: "Key1=Value1,Key2=Value2"
  name: nginx
  namespace: default
spec:
  externalTrafficPolicy: Local
  ports:
  - name: tcp
    port: 80
    protocol: TCP
    targetPort: 80
  - name: https
    port: 443
    protocol: TCP
    targetPort: 443
  selector:
    app: nginx
  sessionAffinity: None
  loadBalancerClass: "alibabacloud.com/nlb"
  type: LoadBalancer

Reuse an existing load balancer

Annotation: Multiple annotations are available, as shown in the following table.

Annotation

Description

Supported CCM versions

service.beta.kubernetes.io/alibaba-cloud-loadbalancer-id

The ID of an existing load balancer.

v2.5.0 and later

service.beta.kubernetes.io/alibaba-cloud-loadbalancer-force-override-listeners

Specifies whether to synchronize the NLB listener configuration based on the Service. Valid values:

true: CCM creates, updates, and deletes NLB listeners based on the Service configuration.
false: CCM does not perform any operations on NLB listeners.

Default value: false

apiVersion: v1
kind: Service
metadata:
  annotations:
    service.beta.kubernetes.io/alibaba-cloud-loadbalancer-id: "${your-nlb-id}" # The ID of the NLB instance.
    service.beta.kubernetes.io/alibaba-cloud-loadbalancer-force-override-listeners: "true"
  name: nginx
  namespace: default
spec:
  externalTrafficPolicy: Local
  ports:
  - name: tcp
    port: 80
    protocol: TCP
    targetPort: 80
  - name: https
    port: 443
    protocol: TCP
    targetPort: 443
  selector:
    app: nginx
  sessionAffinity: None
  loadBalancerClass: "alibabacloud.com/nlb"
  type: LoadBalancer

Associate an Internet Shared Bandwidth instance

Annotation: service.beta.kubernetes.io/alibaba-cloud-loadbalancer-bandwidth-package-id

Description

Supported CCM versions

The ID of the Internet Shared Bandwidth instance to associate.

You can log on to the VPC console to query the ID of the Internet Shared Bandwidth instance.

v2.9.1 and later

apiVersion: v1
kind: Service
metadata:
  annotations:
    service.beta.kubernetes.io/alibaba-cloud-loadbalancer-zone-maps: "${zone-A}:${vsw-A},${zone-B}:${vsw-B}" # For example, cn-hangzhou-k:vsw-i123456,cn-hangzhou-j:vsw-j654321.
    service.beta.kubernetes.io/alibaba-cloud-loadbalancer-bandwidth-package-id: "cbwp-xxxxxxxxxx" 
  name: nginx
  namespace: default
spec:
  externalTrafficPolicy: Local
  ports:
  - name: tcp
    port: 80
    protocol: TCP
    targetPort: 80
  - name: https
    port: 443
    protocol: TCP
    targetPort: 443
  selector:
    app: nginx
  sessionAffinity: None
  loadBalancerClass: "alibabacloud.com/nlb"
  type: LoadBalancer

Specify the network type of an IPv6 address

Annotation: Multiple annotations are available, as shown in the following table.

Annotation

Description

Supported CCM versions

service.beta.kubernetes.io/alibaba-cloud-loadbalancer-ip-version

Specifies the IP version of the NLB instance. The IP version cannot be changed after the NLB instance is created. The kube-proxy of the cluster must work in IPVS mode. Valid values:

ipv4: IPv4.
DualStack: dual-stack. Both IPv4 and IPv6 are supported.
- The two vSwitches specified in service.beta.kubernetes.io/alibaba-cloud-loadbalancer-zone-maps must have IPv6 enabled.
- The generated IPv6 address can be accessed only in an environment that supports IPv6.

Default value: ipv4

v2.5.0 and later

service.beta.kubernetes.io/alibaba-cloud-loadbalancer-ipv6-address-type

Specifies the network type of the IPv6 address of the NLB instance. Valid values:

intranet: private IPv6
internet: public IPv6

Default value: intranet

Note

When you enable public IPv6, an IPv6 gateway must exist in the VPC where the NLB instance is deployed. For more information, see Create and manage an IPv6 gateway.

v2.9.1 and later

apiVersion: v1
kind: Service
metadata:
  annotations:
    service.beta.kubernetes.io/alibaba-cloud-loadbalancer-zone-maps: "${zone-A}:${vsw-A},${zone-B}:${vsw-B}" # For example, cn-hangzhou-k:vsw-i123456,cn-hangzhou-j:vsw-j654321.
    service.beta.kubernetes.io/alibaba-cloud-loadbalancer-ip-version: "DualStack"
    service.beta.kubernetes.io/alibaba-cloud-loadbalancer-ipv6-address-type: internet # Specify the network type of the IPv6 address as public.
  name: nginx
  namespace: default
spec:
  externalTrafficPolicy: Local
  ports:
  - name: tcp
    port: 80
    protocol: TCP
    targetPort: 80
  - name: https
    port: 443
    protocol: TCP
    targetPort: 443
  selector:
    app: nginx
  sessionAffinity: None
  loadBalancerClass: "alibabacloud.com/nlb"
  type: LoadBalancer

Retain the automatically created load balancer when the Service is deleted

Annotation: service.beta.kubernetes.io/alibaba-cloud-loadbalancer-preserve-lb-on-delete

Description

Supported CCM versions

When you delete a LoadBalancer Service, the NLB instance created by the Service is retained, and the kubernetes.do.not.delete and ack.aliyun.com tags are removed from the NLB instance and server group. The existing servers in the server group are retained.

When this feature is enabled, a Warning event of the PreservedOnDelete type is generated during Service synchronization. After you configure this annotation, check whether this event exists to confirm that the feature is enabled.

Valid values:

Not empty: enables the retention feature.
Empty or not set: disables the retention feature.

Important

Perform this operation by deleting the Service instead of modifying the Service type. Otherwise, the Service may be incorrectly re-associated with the previously retained NLB instance.

v2.10.0 and later

apiVersion: v1
kind: Service
metadata:
  annotations:
    service.beta.kubernetes.io/alibaba-cloud-loadbalancer-zone-maps: "${zone-A}:${vsw-A},${zone-B}:${vsw-B}" # For example, cn-hangzhou-k:vsw-i123456,cn-hangzhou-j:vsw-j654321.
    service.beta.kubernetes.io/alibaba-cloud-loadbalancer-preserve-lb-on-delete: "true"
  name: nginx
  namespace: default
spec:
  externalTrafficPolicy: Local
  ports:
  - name: tcp
    port: 80
    protocol: TCP
    targetPort: 80
  - name: https
    port: 443
    protocol: TCP
    targetPort: 443
  selector:
    app: nginx
  loadBalancerClass: "alibabacloud.com/nlb"
  type: LoadBalancer

Common operations on listeners

Configure a security group for a listener

Annotation: service.beta.kubernetes.io/alibaba-cloud-loadbalancer-security-group-ids

Description	Supported CCM versions
Configures a security group for the listener. Separate multiple values with commas (,), for example, `sg-aaaaa,sg-bbbbb`.	v2.6.0 and later

apiVersion: v1
kind: Service
metadata:
  annotations:
    service.beta.kubernetes.io/alibaba-cloud-loadbalancer-zone-maps: "${zone-A}:${vsw-A},${zone-B}:${vsw-B}" # For example, cn-hangzhou-k:vsw-i123456,cn-hangzhou-j:vsw-j654321.
    service.beta.kubernetes.io/alibaba-cloud-loadbalancer-security-group-ids: "sg-aaaaa,sg-bbbbb" # Separate multiple security groups with commas (,).
  name: nginx
  namespace: default
spec:
  externalTrafficPolicy: Local
  ports:
  - name: tcp
    port: 80
    protocol: TCP
    targetPort: 80
  - name: https
    port: 443
    protocol: TCP
    targetPort: 443
  selector:
    app: nginx
  loadBalancerClass: "alibabacloud.com/nlb"
  type: LoadBalancer

Configure TCP and UDP protocols for a listener

Description	Supported CCM versions
This feature requires that the Kubernetes cluster version is 1.24 or later. For more information about how to upgrade the cluster version, see Upgrade an ACK cluster.	Not applicable

apiVersion: v1
kind: Service
metadata:
  annotations:
      service.beta.kubernetes.io/alibaba-cloud-loadbalancer-zone-maps: "${zone-A}:${vsw-A},${zone-B}:${vsw-B}" # For example, cn-hangzhou-k:vsw-i123456,cn-hangzhou-j:vsw-j654321.
  name: nginx
  namespace: default
spec:
  externalTrafficPolicy: Local
  ports:
  - name: tcp
    port: 80
    protocol: TCP
    targetPort: 80
  - name: udp
    port: 80
    protocol: UDP
    targetPort: 81
  selector:
    app: nginx
  sessionAffinity: None
  loadBalancerClass: "alibabacloud.com/nlb"
  type: LoadBalancer

Create a TCP listener

apiVersion: v1
kind: Service
metadata:
  annotations:
    service.beta.kubernetes.io/alibaba-cloud-loadbalancer-zone-maps: "${zone-A}:${vsw-A},${zone-B}:${vsw-B}" # For example, cn-hangzhou-k:vsw-i123456,cn-hangzhou-j:vsw-j654321.
  name: nginx
  namespace: default
spec:
  externalTrafficPolicy: Local
  ports:
  - name: tcp
    port: 80
    protocol: TCP
    targetPort: 80
  - name: https
    port: 443
    protocol: TCP
    targetPort: 443
  selector:
    app: nginx
  sessionAffinity: None
  loadBalancerClass: "alibabacloud.com/nlb"
  type: LoadBalancer

Create a UDP listener

apiVersion: v1
kind: Service
metadata:
  annotations:
    service.beta.kubernetes.io/alibaba-cloud-loadbalancer-zone-maps: "${zone-A}:${vsw-A},${zone-B}:${vsw-B}" # For example, cn-hangzhou-k:vsw-i123456,cn-hangzhou-j:vsw-j654321.
  name: nginx
  namespace: default
spec:
  externalTrafficPolicy: Local
  ports:
  - name: udp
    port: 80
    protocol: UDP
    targetPort: 80
  selector:
    app: nginx
  sessionAffinity: None
  loadBalancerClass: "alibabacloud.com/nlb"
  type: LoadBalancer

Create a TCP/SSL listener

Annotation: Multiple annotations are available, as shown in the following table.

Annotation	Description	Supported CCM versions
service.beta.kubernetes.io/alibaba-cloud-loadbalancer-protocol-port	Sets the protocol type of the listener. Separate multiple values with commas (,), for example, `TCP:80,TCPSSL:443`.	v2.5.0 and later
service.beta.kubernetes.io/alibaba-cloud-loadbalancer-cert-id	The ID of the server certificate. You can log on to Digital Certificate Management Service console and create and view the certificate on the SSL Certificates page. The following figure shows an example:	v2.5.0 and later

Important

TCP/SSL listeners cannot be associated with server groups that have client IP address persistence enabled. This means you cannot configure the service.beta.kubernetes.io/alibaba-cloud-loadbalancer-protocol-port: "tcpssl:${port}" and service.beta.kubernetes.io/alibaba-cloud-loadbalancer-preserve-client-ip: "on" annotations at the same time. For more information, see Obtain the originating IP addresses of clients using an NLB instance.

apiVersion: v1
kind: Service
metadata:
  annotations:
    service.beta.kubernetes.io/alibaba-cloud-loadbalancer-zone-maps: "${zone-A}:${vsw-A},${zone-B}:${vsw-B}" # For example, cn-hangzhou-k:vsw-i123456,cn-hangzhou-j:vsw-j654321.
    service.beta.kubernetes.io/alibaba-cloud-loadbalancer-protocol-port: "tcpssl:443"
    service.beta.kubernetes.io/alibaba-cloud-loadbalancer-cert-id: "${CertIdentifier}"
  name: nginx
  namespace: default
spec:
  externalTrafficPolicy: Local
  ports:
  - name: tcp
    port: 80
    protocol: TCP
    targetPort: 80
  - name: https
    port: 443
    protocol: TCP
    targetPort: 80
  selector:
    app: nginx
  loadBalancerClass: "alibabacloud.com/nlb"
  type: LoadBalancer

Configure a listener port range

Annotation: service.beta.kubernetes.io/alibaba-cloud-loadbalancer-listener-port-range

Description

Supported CCM versions

This feature is supported only for the Terway network plugin.

By configuring the listener address range for all-port forwarding of NLB, you can listen to a specified port range and forward traffic to the corresponding ports of backend servers.

The format is Port range:Service port. Separate multiple values with commas (,), for example, 80-100:80,400-500:443. Multiple ports and port ranges of the same protocol cannot overlap.

targetPort is the health check port of the backend server. It must be an integer from 1 to 65535.

v2.11.4 and later

apiVersion: v1
kind: Service
metadata:
  annotations:
    service.beta.kubernetes.io/alibaba-cloud-loadbalancer-zone-maps: "${zone-A}:${vsw-A},${zone-B}:${vsw-B}" # For example, cn-hangzhou-k:vsw-i123456,cn-hangzhou-j:vsw-j654321.
    # Configure listeners for the 80-100 and 400-500 port ranges.
    service.beta.kubernetes.io/alibaba-cloud-loadbalancer-listener-port-range: "80-100:80,400-500:443"
  name: nginx
  namespace: default
spec:
  externalTrafficPolicy: Local
  ports:
  - name: tcp
    port: 80
    protocol: TCP
    # targetPort is the health check port of the backend server and must be an integer.
    targetPort: 80
  - name: https
    port: 443
    protocol: TCP
    targetPort: 80
  selector:
    app: nginx
  loadBalancerClass: "alibabacloud.com/nlb"
  type: LoadBalancer

Enable mutual authentication

Annotation: Multiple annotations are available, as shown in the following table.

Annotation	Description	Supported CCM versions
service.beta.kubernetes.io/alibaba-cloud-loadbalancer-protocol-port	Sets the protocol type of the listener. Separate multiple values with commas (,), for example, `TCP:80,TCPSSL:443`.	v2.5.0 and later
service.beta.kubernetes.io/alibaba-cloud-loadbalancer-cert-id	The ID of the server certificate. You can log on to Digital Certificate Management Service console and create and view the certificate on the SSL Certificates page. The following figure shows an example:
service.beta.kubernetes.io/alibaba-cloud-loadbalancer-cacert-id	The ID of the CA certificate. You can log on to Digital Certificate Management Service console and view the certificate details on the PCA Certificate Management page.
service.beta.kubernetes.io/alibaba-cloud-loadbalancer-cacert	Specifies whether to enable mutual authentication. Valid values: `on`: enables the feature. `off`: disables the feature. Default value: `off`

apiVersion: v1
kind: Service
metadata:
  annotations:
    service.beta.kubernetes.io/alibaba-cloud-loadbalancer-zone-maps: "${zone-A}:${vsw-A},${zone-B}:${vsw-B}" # For example, cn-hangzhou-k:vsw-i123456,cn-hangzhou-j:vsw-j654321.
    service.beta.kubernetes.io/alibaba-cloud-loadbalancer-protocol-port: "tcpssl:443"   
    service.beta.kubernetes.io/alibaba-cloud-loadbalancer-cert-id: "${CertIdentifier}" 
    service.beta.kubernetes.io/alibaba-cloud-loadbalancer-cacert-id: "${your-cacert-id}" 
    service.beta.kubernetes.io/alibaba-cloud-loadbalancer-cacert: "on"
  name: nginx
  namespace: default
spec:
  externalTrafficPolicy: Local
  ports:
  - name: tcp
    port: 80
    protocol: TCP
    targetPort: 80
  - name: https
    port: 443
    protocol: TCP
    targetPort: 80
  selector:
    app: nginx
  loadBalancerClass: "alibabacloud.com/nlb"
  type: LoadBalancer

Configure a TLS security policy

Annotation: Multiple annotations are available, as shown in the following table.

Annotation	Description	Supported CCM versions
service.beta.kubernetes.io/alibaba-cloud-loadbalancer-protocol-port	Sets the protocol type of the listener. Separate multiple values with commas (,), for example, `TCP:80,TCPSSL:443`.	v2.5.0 and later
service.beta.kubernetes.io/alibaba-cloud-loadbalancer-cert-id	The ID of the server certificate. You can log on to Digital Certificate Management Service console and create and view the certificate on the SSL Certificates page. The following figure shows an example:
service.beta.kubernetes.io/alibaba-cloud-loadbalancer-tls-cipher-policy	The ID of the security policy. System security policies and custom security policies are supported. Valid values: `tls_cipher_policy_1_0` `tls_cipher_policy_1_1` `tls_cipher_policy_1_2` `tls_cipher_policy_1_2_strict` `tls_cipher_policy_1_2_strict_with_1_3` Default value: `tls_cipher_policy_1_0`

apiVersion: v1
kind: Service
metadata:
  annotations:
    service.beta.kubernetes.io/alibaba-cloud-loadbalancer-zone-maps: "${zone-A}:${vsw-A},${zone-B}:${vsw-B}" # For example, cn-hangzhou-k:vsw-i123456,cn-hangzhou-j:vsw-j654321.
    service.beta.kubernetes.io/alibaba-cloud-loadbalancer-protocol-port: "tcpssl:443"
    service.beta.kubernetes.io/alibaba-cloud-loadbalancer-cert-id: "${CertIdentifier}" 
    service.beta.kubernetes.io/alibaba-cloud-loadbalancer-tls-cipher-policy: "tls_cipher_policy_1_0"
  name: nginx
  namespace: default
spec:
  externalTrafficPolicy: Local
  ports:
  - name: tcp
    port: 80
    protocol: TCP
    targetPort: 80
  - name: https
    port: 443
    protocol: TCP
    targetPort: 80
  selector:
    app: nginx
  loadBalancerClass: "alibabacloud.com/nlb"
  type: LoadBalancer

Configure Proxy Protocol

Annotation: service.beta.kubernetes.io/alibaba-cloud-loadbalancer-proxy-protocol

Description

Supported CCM versions

Specifies whether to use the Proxy Protocol to pass client IP addresses to backend servers. Valid values:

on: enables the feature.
off: disables the feature.

Default value: off

Important

Before you enable Proxy Protocol, check whether Proxy Protocol v2 is enabled for the backend service. If it is not enabled, access may fail. Configure this parameter with caution.

v2.5.0 and later

apiVersion: v1
kind: Service
metadata:
  annotations:
    service.beta.kubernetes.io/alibaba-cloud-loadbalancer-zone-maps: "${zone-A}:${vsw-A},${zone-B}:${vsw-B}" # For example, cn-hangzhou-k:vsw-i123456,cn-hangzhou-j:vsw-j654321.
    service.beta.kubernetes.io/alibaba-cloud-loadbalancer-proxy-protocol: "on"
  name: nginx
  namespace: default
spec:
  externalTrafficPolicy: Local
  ports:
  - name: tcp
    port: 80
    protocol: TCP
    targetPort: 80
  - name: https
    port: 443
    protocol: TCP
    targetPort: 443
  selector:
    app: nginx
  loadBalancerClass: "alibabacloud.com/nlb"
  type: LoadBalancer

Carry additional information in Proxy Protocol

Annotation: Multiple annotations are available, as shown in the following table.

Annotation	Description	Supported CCM versions
service.beta.kubernetes.io/alibaba-cloud-loadbalancer-proxy-protocol	Specifies whether to use the Proxy Protocol to pass client IP addresses to backend servers. Valid values: `on`: enables the feature. `off`: disables the feature. Default value: `off` Important Before you enable Proxy Protocol, check whether Proxy Protocol v2 is enabled for the backend service. If it is not enabled, access may fail. Configure this parameter with caution.	v2.5.0 and later
service.beta.kubernetes.io/alibaba-cloud-loadbalancer-ppv2-pvl-vpc-id-enabled	Specifies whether to use the Proxy Protocol to pass the VpcId to backend servers. Valid values: `on`: enables the feature. `off`: disables the feature. Default value: `off`	v2.9.1 and later
service.beta.kubernetes.io/alibaba-cloud-loadbalancer-ppv2-pvl-ep-id-enabled	Specifies whether to use the Proxy Protocol to pass the PrivateLinkEpId to backend servers. Valid values: `on`: enables the feature. `off`: disables the feature. Default value: `off`	v2.9.1 and later
service.beta.kubernetes.io/alibaba-cloud-loadbalancer-ppv2-pvl-eps-id-enabled	Specifies whether to use the Proxy Protocol to pass the PrivateLinkEpsId to backend servers. Valid values: `on`: enables the feature. `off`: disables the feature. Default value: `off`	v2.9.1 and later

apiVersion: v1
kind: Service
metadata:
  annotations:
    service.beta.kubernetes.io/alibaba-cloud-loadbalancer-zone-maps: "${zone-A}:${vsw-A},${zone-B}:${vsw-B}" # For example, cn-hangzhou-k:vsw-i123456,cn-hangzhou-j:vsw-j654321.
    service.beta.kubernetes.io/alibaba-cloud-loadbalancer-proxy-protocol: "on"
    service.beta.kubernetes.io/alibaba-cloud-loadbalancer-ppv2-pvl-ep-id-enabled: "on"
    service.beta.kubernetes.io/alibaba-cloud-loadbalancer-ppv2-pvl-eps-id-enabled: "on"
    service.beta.kubernetes.io/alibaba-cloud-loadbalancer-ppv2-pvl-vpc-id-enabled: "on"
  name: nginx
  namespace: default
spec:
  externalTrafficPolicy: Local
  ports:
  - name: tcp
    port: 80
    protocol: TCP
    targetPort: 80
  - name: https
    port: 443
    protocol: TCP
    targetPort: 443
  selector:
    app: nginx
  loadBalancerClass: "alibabacloud.com/nlb"
  type: LoadBalancer

Set the maximum number of new connections per second for a listener

Annotation: service.beta.kubernetes.io/alibaba-cloud-loadbalancer-cps

Description	Supported CCM versions
The maximum number of new connections per second for the NLB instance. The value must be in the range of 0 to 1,000,000. A value of 0 indicates no limit.	v2.5.0 and later

apiVersion: v1
kind: Service
metadata:
  annotations:
    service.beta.kubernetes.io/alibaba-cloud-loadbalancer-zone-maps: "${zone-A}:${vsw-A},${zone-B}:${vsw-B}" # For example, cn-hangzhou-k:vsw-i123456,cn-hangzhou-j:vsw-j654321.
    service.beta.kubernetes.io/alibaba-cloud-loadbalancer-cps: "100"
  name: nginx
  namespace: default
spec:
  externalTrafficPolicy: Local
  ports:
  - name: tcp
    port: 80
    protocol: TCP
    targetPort: 80
  - name: https
    port: 443
    protocol: TCP
    targetPort: 443
  selector:
    app: nginx
  loadBalancerClass: "alibabacloud.com/nlb"
  type: LoadBalancer

Set the idle connection timeout period for a listener

Annotation: service.beta.kubernetes.io/alibaba-cloud-loadbalancer-idle-timeout

Description

Supported CCM versions

The idle connection timeout period. Unit: seconds. The value must be in the range of 10 to 900.

Default value: 900

v2.5.0 and later

apiVersion: v1
kind: Service
metadata:
  annotations:
    service.beta.kubernetes.io/alibaba-cloud-loadbalancer-zone-maps: "${zone-A}:${vsw-A},${zone-B}:${vsw-B}" # For example, cn-hangzhou-k:vsw-i123456,cn-hangzhou-j:vsw-j654321.
    service.beta.kubernetes.io/alibaba-cloud-loadbalancer-idle-timeout: "60"
  name: nginx
  namespace: default
spec:
  externalTrafficPolicy: Local
  ports:
  - name: tcp
    port: 80
    protocol: TCP
    targetPort: 80
  - name: https
    port: 443
    protocol: TCP
    targetPort: 443
  selector:
    app: nginx
  loadBalancerClass: "alibabacloud.com/nlb"
  type: LoadBalancer

Configure an ALPN policy

Annotation: Multiple annotations are available, as shown in the following table.

Annotation	Description	Supported CCM versions
service.beta.kubernetes.io/alibaba-cloud-loadbalancer-protocol-port	Sets the protocol type of the listener. Separate multiple values with commas (,), for example, `TCP:80,TCPSSL:443`.	v2.5.0 and later
service.beta.kubernetes.io/alibaba-cloud-loadbalancer-cert-id	The ID of the server certificate. You can log on to Digital Certificate Management Service console and create and view the certificate on the SSL Certificates page. The following figure shows an example:	v2.5.0 and later
service.beta.kubernetes.io/alibaba-cloud-loadbalancer-alpn	Specifies whether to enable Application-Layer Protocol Negotiation (ALPN). Valid values: `on`: enables the feature. `off`: disables the feature. Default value: `off`	v2.10.0 and later
service.beta.kubernetes.io/alibaba-cloud-loadbalancer-alpn-policy	The ALPN policy. Valid values: `HTTP1Only`: negotiates only HTTP/1.x protocols. The priority is HTTP/1.1 > HTTP/1.0. `HTTP2Only`: negotiates only the HTTP/2.0 protocol. `HTTP2Optional`: prioritizes HTTP/1.x protocols but also accepts the HTTP/2.0 protocol. The priority is HTTP/1.1 > HTTP/1.0 > HTTP/2.0. `HTTP2Preferred`: prioritizes the HTTP/2 protocol but also accepts HTTP/1.x protocols. The priority is HTTP/2.0 > HTTP/1.1 > HTTP/1.0. For more information about the valid values of this option, see the `AlpnPolicy` field in the CreateListener API reference.	v2.10.0 and later

apiVersion: v1
kind: Service
metadata:
  annotations:
    service.beta.kubernetes.io/alibaba-cloud-loadbalancer-zone-maps: "${zone-A}:${vsw-A},${zone-B}:${vsw-B}" # For example, cn-hangzhou-k:vsw-i123456,cn-hangzhou-j:vsw-j654321.
    service.beta.kubernetes.io/alibaba-cloud-loadbalancer-protocol-port: "tcpssl:443"
    service.beta.kubernetes.io/alibaba-cloud-loadbalancer-cert-id: "${CertIdentifier}" 
    service.beta.kubernetes.io/alibaba-cloud-loadbalancer-alpn: "on"
    service.beta.kubernetes.io/alibaba-cloud-loadbalancer-alpn-policy: "HTTP1Only" 
  name: nginx
  namespace: default
spec:
  externalTrafficPolicy: Local
  ports:
  - name: tcp
    port: 80
    protocol: TCP
    targetPort: 80
  - name: https
    port: 443
    protocol: TCP
    targetPort: 80
  selector:
    app: nginx
  loadBalancerClass: "alibabacloud.com/nlb"
  type: LoadBalancer

Common operations on server groups

Configure a scheduling algorithm

Annotation: service.beta.kubernetes.io/alibaba-cloud-loadbalancer-scheduler

Description

Supported CCM versions

The scheduling algorithm. Valid values:

wrr: weighted round-robin. Servers with higher weights receive more requests than servers with lower weights.
rr: round-robin. Requests are distributed to backend servers in sequence.
sch: source IP hash. Requests from the same source IP address are distributed to the same backend server.
tch: four-element hash. Requests are distributed to backend servers based on a hash of four factors: source IP address, destination IP address, source port, and destination port. This ensures that requests from the same stream are distributed to the same backend server.
wlc: weighted least connections. In addition to the weight of each backend server, the number of connections to each backend server is also considered. If backend servers have the same weight, the backend server that has the smallest number of connections is polled.

Default value: wrr

For more information about the valid values of this option, see the Scheduler field in the CreateServerGroup API reference.

v2.5.0 and later

apiVersion: v1
kind: Service
metadata:
  annotations:
    service.beta.kubernetes.io/alibaba-cloud-loadbalancer-zone-maps: "${zone-A}:${vsw-A},${zone-B}:${vsw-B}" # For example, cn-hangzhou-k:vsw-i123456,cn-hangzhou-j:vsw-j654321.
    service.beta.kubernetes.io/alibaba-cloud-loadbalancer-scheduler: "sch"
  name: nginx
  namespace: default
spec:
  externalTrafficPolicy: Local
  ports:
  - name: tcp
    port: 80
    protocol: TCP
    targetPort: 80
  - name: https
    port: 443
    protocol: TCP
    targetPort: 443
  selector:
    app: nginx
  loadBalancerClass: "alibabacloud.com/nlb"
  type: LoadBalancer

Configure connection draining

Annotation: Multiple annotations are available, as shown in the following table.

Annotation

Description

Supported CCM versions

service.beta.kubernetes.io/alibaba-cloud-loadbalancer-connection-drain

Specifies whether to enable connection draining. After you enable this feature, the following behaviors are performed when a backend server is removed or fails a health check:

on: Existing connections are allowed to be transmitted normally within a certain period of time. After the timeout period elapses, the connections are actively disconnected to ensure smooth business offline.
off: Existing connections are not actively interrupted. They are interrupted only when the client actively disconnects or the persistent connection session expires.

Default value: off

v2.5.0 and later

service.beta.kubernetes.io/alibaba-cloud-loadbalancer-connection-drain-timeout

Sets the connection draining timeout period.

Unit: seconds.
The value must be in the range of 0 to 900. A value of 0 indicates that the connection is immediately interrupted.

apiVersion: v1
kind: Service
metadata:
  annotations:
    service.beta.kubernetes.io/alibaba-cloud-loadbalancer-zone-maps: "${zone-A}:${vsw-A},${zone-B}:${vsw-B}" # For example, cn-hangzhou-k:vsw-i123456,cn-hangzhou-j:vsw-j654321.
    service.beta.kubernetes.io/alibaba-cloud-loadbalancer-connection-drain: "on"
    service.beta.kubernetes.io/alibaba-cloud-loadbalancer-connection-drain-timeout: "30"
  name: nginx
  namespace: default
spec:
  externalTrafficPolicy: Local
  ports:
  - name: tcp
    port: 80
    protocol: TCP
    targetPort: 80
  - name: https
    port: 443
    protocol: TCP
    targetPort: 443
  selector:
    app: nginx
  loadBalancerClass: "alibabacloud.com/nlb"
  type: LoadBalancer

Enable client IP address persistence

Annotation: service.beta.kubernetes.io/alibaba-cloud-loadbalancer-preserve-client-ip

Description

Supported CCM versions

Specifies whether to enable client IP address persistence. Valid values:

on: enables the feature.
off: disables the feature.

Default value: on

v2.5.0 and later

apiVersion: v1
kind: Service
metadata:
  annotations:
    service.beta.kubernetes.io/alibaba-cloud-loadbalancer-zone-maps: "${zone-A}:${vsw-A},${zone-B}:${vsw-B}" # For example, cn-hangzhou-k:vsw-i123456,cn-hangzhou-j:vsw-j654321.
    service.beta.kubernetes.io/alibaba-cloud-loadbalancer-preserve-client-ip: "on"
  name: nginx
  namespace: default
spec:
  externalTrafficPolicy: Local
  ports:
  - name: tcp
    port: 80
    protocol: TCP
    targetPort: 80
  - name: https
    port: 443
    protocol: TCP
    targetPort: 443
  selector:
    app: nginx
  loadBalancerClass: "alibabacloud.com/nlb"
  type: LoadBalancer

Configure a TCP health check

Annotation: Multiple annotations are available, as shown in the following table. To configure a TCP health check, all the following annotations are required. Health checks are enabled by default for TCP ports.

Annotation	Description
service.beta.kubernetes.io/alibaba-cloud-loadbalancer-health-check-flag	Specifies whether to enable health checks. Valid values: `on`: Enabled. `off`: disables health checks. Default value: `on`
service.beta.kubernetes.io/alibaba-cloud-loadbalancer-health-check-type	The health check protocol. Valid values: `tcp` `http` Default value: `tcp`
service.beta.kubernetes.io/alibaba-cloud-loadbalancer-health-check-connect-port	The port on the server that is used for health checks. The value must be in the range of 0 to 65535. A value of 0 indicates that the port of the server is used for health checks.
service.beta.kubernetes.io/alibaba-cloud-loadbalancer-health-check-connect-timeout	The maximum timeout period for a health check response. Unit: seconds. The value must be in the range of 1 to 300. Default value: 5
service.beta.kubernetes.io/alibaba-cloud-loadbalancer-healthy-threshold	The number of consecutive successful health checks required before the health check status of a server is changed from failed to successful. The value must be in the range of 2 to 10. Default value: 2
service.beta.kubernetes.io/alibaba-cloud-loadbalancer-unhealthy-threshold	The number of consecutive failed health checks required before the health check status of a server is changed from successful to failed. The value must be in the range of 2 to 10. Default value: 2
service.beta.kubernetes.io/alibaba-cloud-loadbalancer-health-check-interval	The interval at which health checks are performed. Unit: seconds. The value must be in the range of 1 to 50. Default value: 10

apiVersion: v1
kind: Service
metadata:
  annotations:
    service.beta.kubernetes.io/alibaba-cloud-loadbalancer-zone-maps: "${zone-A}:${vsw-A},${zone-B}:${vsw-B}" # For example, cn-hangzhou-k:vsw-i123456,cn-hangzhou-j:vsw-j654321.
    service.beta.kubernetes.io/alibaba-cloud-loadbalancer-health-check-flag: "on"
    service.beta.kubernetes.io/alibaba-cloud-loadbalancer-health-check-type: "tcp"
    service.beta.kubernetes.io/alibaba-cloud-loadbalancer-health-check-connect-timeout: "8"
    service.beta.kubernetes.io/alibaba-cloud-loadbalancer-healthy-threshold: "4"
    service.beta.kubernetes.io/alibaba-cloud-loadbalancer-unhealthy-threshold: "4"
    service.beta.kubernetes.io/alibaba-cloud-loadbalancer-health-check-interval: "5"
  name: nginx
  namespace: default
spec:
  externalTrafficPolicy: Local
  ports:
  - name: tcp
    port: 80
    protocol: TCP
    targetPort: 80
  - name: https
    port: 443
    protocol: TCP
    targetPort: 443
  selector:
    app: nginx
  loadBalancerClass: "alibabacloud.com/nlb"
  type: LoadBalancer

Configure an HTTP health check

Annotation: Multiple annotations are available, as shown in the following table. To configure an HTTP health check, all the following annotations are required. By default, health checks are enabled for TCP ports.

Annotation	Description	Supported CCM versions
service.beta.kubernetes.io/alibaba-cloud-loadbalancer-health-check-flag	Specifies whether to enable health checks. Valid values: `on`: Enabled. `off`: Shutdown. Default value: `on`	v2.5.0 and later
service.beta.kubernetes.io/alibaba-cloud-loadbalancer-health-check-type	The health check protocol. Valid values: `tcp` `http` Default value: `tcp`
service.beta.kubernetes.io/alibaba-cloud-loadbalancer-health-check-uri	The path for health checks. The path must be 1 to 80 characters in length and can contain only letters, digits, and special characters. It must start with a forward slash (/). For more information, see CreateServerGroup. Note This parameter takes effect only when `service.beta.kubernetes.io/alibaba-cloud-loadbalancer-health-check-type` is set to `HTTP`.
service.beta.kubernetes.io/alibaba-cloud-loadbalancer-health-check-domain	The domain name used for health checks. Valid values: $SERVER_IP: uses the private IP address of the backend server domain: specifies a domain name. The domain name must be 1 to 80 characters in length and can contain only lowercase letters, digits, hyphens (-), and periods (.). Note This parameter takes effect only when `service.beta.kubernetes.io/alibaba-cloud-loadbalancer-health-check-type` is set to `HTTP`.
service.beta.kubernetes.io/alibaba-cloud-loadbalancer-health-check-connect-port	The port on the server that is used for health checks. The value must be in the range of 0 to 65535. A value of 0 indicates that the port of the server is used for health checks.
service.beta.kubernetes.io/alibaba-cloud-loadbalancer-health-check-connect-timeout	The maximum timeout period for a health check response. Unit: seconds. The value must be in the range of 1 to 300. Default value: 5
service.beta.kubernetes.io/alibaba-cloud-loadbalancer-healthy-threshold	The number of consecutive successful health checks required before the health check status of a server is changed from failed to successful. The value must be in the range of 2 to 10. Default value: 2
service.beta.kubernetes.io/alibaba-cloud-loadbalancer-unhealthy-threshold	The number of consecutive failed health checks required before the health check status of a server is changed from successful to failed. The value must be in the range of 2 to 10. Default value: 2
service.beta.kubernetes.io/alibaba-cloud-loadbalancer-health-check-interval	The interval at which health checks are performed. Unit: seconds. The value must be in the range of 1 to 50. Default value: 10
service.beta.kubernetes.io/alibaba-cloud-loadbalancer-health-check-method	The health check method. Valid values: `GET` `HEAD` Note This parameter takes effect only when `service.beta.kubernetes.io/alibaba-cloud-loadbalancer-health-check-type` is set to `HTTP`.

apiVersion: v1
kind: Service
metadata:
  annotations:
    service.beta.kubernetes.io/alibaba-cloud-loadbalancer-zone-maps: "${zone-A}:${vsw-A},${zone-B}:${vsw-B}" # For example, cn-hangzhou-k:vsw-i123456,cn-hangzhou-j:vsw-j654321.
    service.beta.kubernetes.io/alibaba-cloud-loadbalancer-health-check-flag: "on"
    service.beta.kubernetes.io/alibaba-cloud-loadbalancer-health-check-type: "http"
    service.beta.kubernetes.io/alibaba-cloud-loadbalancer-health-check-uri: "/test/index.html"
    service.beta.kubernetes.io/alibaba-cloud-loadbalancer-health-check-domain: "www.test.com"
    service.beta.kubernetes.io/alibaba-cloud-loadbalancer-healthy-threshold: "4"
    service.beta.kubernetes.io/alibaba-cloud-loadbalancer-unhealthy-threshold: "4"
    service.beta.kubernetes.io/alibaba-cloud-loadbalancer-health-check-connect-timeout: "10"
    service.beta.kubernetes.io/alibaba-cloud-loadbalancer-health-check-interval: "5"
    # Configure the health check method. This annotation is optional.
    service.beta.kubernetes.io/alibaba-cloud-loadbalancer-health-check-method: "head"
  name: nginx
  namespace: default
spec:
  externalTrafficPolicy: Local
  ports:
  - name: tcp
    port: 80
    protocol: TCP
    targetPort: 80
  - name: https
    port: 443
    protocol: TCP
    targetPort: 443
  selector:
    app: nginx
  loadBalancerClass: "alibabacloud.com/nlb"
  type: LoadBalancer

Configure the server group type

Annotation: service.beta.kubernetes.io/alibaba-cloud-loadbalancer-server-group-type

Description

Supported CCM versions

Configures the server group type. Valid values:

Ip: IP address type. You can directly add backend servers of the IP address type.
Instance (default): server type. You can add instances of the ECS and ENI types.

Default value: Instance

For more information about NLB server group types and descriptions, see NLB server groups.

v2.8.0 and later

apiVersion: v1
kind: Service
metadata:
  annotations:
    service.beta.kubernetes.io/alibaba-cloud-loadbalancer-zone-maps: "${zone-A}:${vsw-A},${zone-B}:${vsw-B}" # For example, cn-hangzhou-k:vsw-i123456,cn-hangzhou-j:vsw-j654321.
    service.beta.kubernetes.io/alibaba-cloud-loadbalancer-server-group-type: "Ip"
  name: nginx
  namespace: default
spec:
  externalTrafficPolicy: Local
  ports:
  - name: tcp
    port: 80
    protocol: TCP
    targetPort: 80
  - name: https
    port: 443
    protocol: TCP
    targetPort: 443
  selector:
    app: nginx
  loadBalancerClass: "alibabacloud.com/nlb"
  type: LoadBalancer

Reuse an existing server group

Annotation: service.beta.kubernetes.io/alibaba-cloud-loadbalancer-vgroup-port

You can reuse an existing server group. This feature works only when you reuse an existing NLB instance. For more information, see Deploy services across clusters by reusing an existing load balancer.

Set the weight of the Service to receive traffic

Annotation: service.beta.kubernetes.io/alibaba-cloud-loadbalancer-weight

When multiple Services reuse the same NLB instance, you can use this annotation to set the traffic weight for the current Service. This annotation works only when you reuse an existing server group. For more information, see Deploy services across clusters by reusing an existing load balancer.

Ignore backend server weight updates

Annotation: service.beta.kubernetes.io/alibaba-cloud-loadbalancer-ignore-weight-update

Description

Supported CCM versions

During Service synchronization, the update of backend server weights in the server group is skipped. This configuration is applicable to scenarios where you need to manually manage backend server weights through a mechanism other than CCM. Valid values:

on
off

Default value: off

v2.11.1 and later

apiVersion: v1
kind: Service
metadata:
  annotations:
    service.beta.kubernetes.io/alibaba-cloud-loadbalancer-zone-maps: "${zone-A}:${vsw-A},${zone-B}:${vsw-B}" # For example, cn-hangzhou-k:vsw-i123456,cn-hangzhou-j:vsw-j654321.
    service.beta.kubernetes.io/alibaba-cloud-loadbalancer-ignore-weight-update: "on"
  name: nginx
  namespace: default
spec:
  externalTrafficPolicy: Local
  ports:
  - name: tcp
    port: 80
    protocol: TCP
    targetPort: 80
  - name: https
    port: 443
    protocol: TCP
    targetPort: 443
  selector:
    app: nginx
  loadBalancerClass: "alibabacloud.com/nlb"
  type: LoadBalancer