Which annotations does the ACK cluster support to configure Network Load Balancer (NLB)? - Container Service for Kubernetes

When you use a Network Load Balancer (NLB) for a Kubernetes Service, you can use annotations in the YAML file to enable additional load balancing features. For example, you can specify the NLB network type, enable configuration read-only mode, or configure mutual authentication. This topic describes common NLB configuration operations using annotations in Service YAML files. These operations are grouped into three resource categories: NLB, listeners, and vServer groups.

Index

Category	Feature category	Configuration link
Important notes
Typical NLB operations	Create a load balancer	Create an Internet-facing NLB Create an internal NLB Create a dual-stack NLB Specify IPv6 address types Use an existing load balancer
	Instance configuration	Specify a load balancer name Specify a resource group for the load balancer Add tags to the load balancer Attach an Internet Shared Bandwidth package Enable cross-zone forwarding
	Security configuration	Retain automatically created load balancer instances when deleting a Service
Typical listener operations	Create a listener	Create a TCP listener Create a UDP listener Create a TCP/SSL listener Configure a listener port range
	Listener configuration	Configure both TCP and UDP protocols for a listener Set the new connection rate limit per second for a listener Enable Proxy Protocol Carry additional information with Proxy Protocol Set the idle timeout for listener connections Set the ALPN policy Configure extended certificates
	Security configuration	Enable mutual authentication Set the TLS security policy Set a security group for the listener
Typical vServer group operations	Server configuration	Set the scheduling algorithm Enable connection draining Preserve client IP addresses Configure TCP health checks Configure HTTP health checks Set the vServer group type Reuse an existing vServer group Set the traffic weight for the Service Ignore backend server weight updates

Important notes

Your ACK cluster version must be 1.24 or later, and your Cloud Controller Manager (CCM) version must be 2.5.0 or later. To upgrade your cluster, see Manually upgrade a cluster. To upgrade components, see Manage components.
In the Service specification, set spec.loadBalancerClass to alibabacloud.com/nlb. If you do not specify this field, Alibaba Cloud creates a Classic Load Balancer (CLB) instance by default. For more information, see Classic Load Balancer (CLB).
You cannot change spec.loadBalancerClass after you create a Service. You cannot convert between CLB and NLB resources.
You cannot manage NLBs in the Container Service console. Use the kubectl command instead.

Typical NLB operations

Create an Internet-facing NLB

Annotation: service.beta.kubernetes.io/alibaba-cloud-loadbalancer-zone-maps

Description

Supported CCM versions

Specify the zone, private IP, and EIP information for the NLB, in the following format:

{zone}:{vSwitch ID}:{private IP address}:{EIP instance ID}. You can separate multiple configurations with commas.

Zone and vSwitch ID: Required. Specify at least two zones. Example: cn-hangzhou-k:vsw-i123456,cn-hangzhou-j:vsw-j654321.
To view supported regions and zones, go to the NLB console.
Private IP address: Optional.
EIP ID: Optional.

Zone and vSwitch ID: Version 2.5.0 or later

Private IP address and EIP: Version 2.12.1 or later

apiVersion: v1
kind: Service
metadata:
  annotations:
    service.beta.kubernetes.io/alibaba-cloud-loadbalancer-zone-maps: "${zone-A}:${vsw-A},${zone-B}:${vsw-B}" # For example, cn-hangzhou-k:vsw-i123456,cn-hangzhou-j:vsw-j654321
  name: nginx
  namespace: default
spec:
  externalTrafficPolicy: Local
  ports:
  - name: tcp
    port: 80
    protocol: TCP
    targetPort: 80
  - name: https
    port: 443
    protocol: TCP
    targetPort: 443
  selector:
    app: nginx
  loadBalancerClass: "alibabacloud.com/nlb"
  type: LoadBalancer

Create an internal NLB

Annotation: service.beta.kubernetes.io/alibaba-cloud-loadbalancer-address-type

Description

Supported CCM versions

Specify that the NLB is internal. You can change this annotation value to switch between Internet-facing and internal NLBs. Valid values:

internet: Internet-facing NLB.
intranet: Internal NLB.

Default value: internet

To view supported regions and zones, go to the NLB console. You must specify at least two zones. Separate zones with commas. Example: cn-hangzhou-k:vsw-i123456,cn-hangzhou-j:vsw-j654321.

Version 2.5.0 or later

apiVersion: v1
kind: Service
metadata:
  annotations:
    service.beta.kubernetes.io/alibaba-cloud-loadbalancer-zone-maps: "${zone-A}:${vsw-A},${zone-B}:${vsw-B}" # For example, cn-hangzhou-k:vsw-i123456,cn-hangzhou-j:vsw-j654321
    service.beta.kubernetes.io/alibaba-cloud-loadbalancer-address-type: "intranet"
  name: nginx
  namespace: default
spec:
  externalTrafficPolicy: Local
  ports:
  - name: tcp
    port: 80
    protocol: TCP
    targetPort: 80
  - name: https
    port: 443
    protocol: TCP
    targetPort: 443
  selector:
    app: nginx
  loadBalancerClass: "alibabacloud.com/nlb"
  type: LoadBalancer

Specify a private IP address for the load balancer

Annotation: service.beta.kubernetes.io/alibaba-cloud-loadbalancer-zone-maps

Description

Supported CCM versions

Specify the zone, private IP, and EIP for the NLB in the following format:

{zone}:{vSwitchID}:{private IP address}:{EIP instance ID}. Separate multiple configurations with commas.

Zone and vSwitch ID: Required. Specify at least two zones. Example: cn-hangzhou-k:vsw-i123456,cn-hangzhou-j:vsw-j654321.
To view supported regions and zones, go to the NLB console.
Private IP address: Optional.
EIP ID: Optional.

Zone and vSwitch ID: Version 2.5.0 or later
Private IP address and EIP: Version 2.12.1 or later

apiVersion: v1
kind: Service
metadata:
  annotations:
    # For example, cn-hangzhou-k:vsw-i123456:10.1.0.1,cn-hangzhou-j:vsw-j654321:10.2.0.1
    service.beta.kubernetes.io/alibaba-cloud-loadbalancer-zone-maps: "${zone-A}:${vsw-A}:${private-ipv4-A},${zone-B}:${vsw-B}:${private-ipv4-B}"
  name: nginx
  namespace: default
spec:
  externalTrafficPolicy: Local
  ports:
  - name: tcp
    port: 80
    protocol: TCP
    targetPort: 80
  - name: https
    port: 443
    protocol: TCP
    targetPort: 443
  selector:
    app: nginx
  loadBalancerClass: "alibabacloud.com/nlb"
  type: LoadBalancer

Specify an EIP ID for the load balancer

Annotation: service.beta.kubernetes.io/alibaba-cloud-loadbalancer-zone-maps

Description

Supported CCM versions

Specify the zone, private IP, and EIP information for the NLB in the following format:

{zone}:{vSwitchID}:{private IP address}:{EIP instance ID}. You can separate multiple configurations with commas.

Zone and vSwitch ID: Required. Specify at least two zones. Example: cn-hangzhou-k:vsw-i123456,cn-hangzhou-j:vsw-j654321.
To view supported regions and zones, go to the NLB console.
Private IP address: Optional.
EIP ID: Optional.

Zone and vSwitch ID: Version 2.5.0 or later

Private IP address and EIP: Version 2.12.1 or later

apiVersion: v1
kind: Service
metadata:
  annotations:
    # Leave the private IP address blank if not needed.
    # For example, cn-hangzhou-k:vsw-i123456::eip-12345,cn-hangzhou-j:vsw-j654321::eip-54321
    service.beta.kubernetes.io/alibaba-cloud-loadbalancer-zone-maps: "${zone-A}:${vsw-A}::${eip-A},${zone-B}:${vsw-B}::${eip-B}"
  name: nginx
  namespace: default
spec:
  externalTrafficPolicy: Local
  ports:
  - name: tcp
    port: 80
    protocol: TCP
    targetPort: 80
  - name: https
    port: 443
    protocol: TCP
    targetPort: 443
  selector:
    app: nginx
  loadBalancerClass: "alibabacloud.com/nlb"
  type: LoadBalancer

Specify a load balancer name

Annotation: service.beta.kubernetes.io/alibaba-cloud-loadbalancer-name

Description	Supported CCM versions
Specify a name for the load balancer instance. The name must be 2 to 128 characters in length. It must start with a letter or Chinese character. It can contain letters, digits, periods (.), underscores (_), and hyphens (-).	Version 2.5.0 or later

apiVersion: v1
kind: Service
metadata:
  annotations:
    service.beta.kubernetes.io/alibaba-cloud-loadbalancer-zone-maps: "${zone-A}:${vsw-A},${zone-B}:${vsw-B}" # For example, cn-hangzhou-k:vsw-i123456,cn-hangzhou-j:vsw-j654321
    service.beta.kubernetes.io/alibaba-cloud-loadbalancer-name: "${your-nlb-name}" # NLB name.
  name: nginx
  namespace: default
spec:
  externalTrafficPolicy: Local
  ports:
  - name: tcp
    port: 80
    protocol: TCP
    targetPort: 80
  - name: https
    port: 443
    protocol: TCP
    targetPort: 443
  selector:
    app: nginx
  loadBalancerClass: "alibabacloud.com/nlb"
  type: LoadBalancer

Specify a resource group for the load balancer

Annotation: service.beta.kubernetes.io/alibaba-cloud-loadbalancer-resource-group-id

Description

Supported CCM versions

Specify a resource group for the load balancer. After you specify the resource group ID, you cannot modify it.

To query the resource group ID, go to the Alibaba Cloud Resource Management console.

Version 2.5.0 or later

apiVersion: v1
kind: Service
metadata:
  annotations:
    service.beta.kubernetes.io/alibaba-cloud-loadbalancer-zone-maps: "${zone-A}:${vsw-A},${zone-B}:${vsw-B}" # For example, cn-hangzhou-k:vsw-i123456,cn-hangzhou-j:vsw-j654321
    service.beta.kubernetes.io/alibaba-cloud-loadbalancer-resource-group-id:  "${your-resource-group-id}" # Resource group ID.
  name: nginx
  namespace: default
spec:
  externalTrafficPolicy: Local
  ports:
  - name: tcp
    port: 80
    protocol: TCP
    targetPort: 80
  - name: https
    port: 443
    protocol: TCP
    targetPort: 443
  selector:
    app: nginx
  loadBalancerClass: "alibabacloud.com/nlb"
  type: LoadBalancer

Create a dual-stack NLB

Annotation: service.beta.kubernetes.io/alibaba-cloud-loadbalancer-ip-version

Description

Supported CCM versions

Specifies the protocol version of the Network Load Balancer (NLB). The IP address type cannot be modified after the NLB is created. To use this feature, the kube-proxy proxy mode of the cluster must be IPVS. Valid values:

ipv4: Specifies the IPv4 address type.
DualStack: Dual-stack. Supports both IPv4 and IPv6.
- The two vSwitches specified in service.beta.kubernetes.io/alibaba-cloud-loadbalancer-zone-maps must have IPv6 enabled.
- The generated IPv6 address is accessible only in IPv6-enabled environments.

Default value: ipv4

Version 2.5.0 or later

apiVersion: v1
kind: Service
metadata:
  annotations:
    service.beta.kubernetes.io/alibaba-cloud-loadbalancer-zone-maps: "${zone-A}:${vsw-A},${zone-B}:${vsw-B}" # For example, cn-hangzhou-k:vsw-i123456,cn-hangzhou-j:vsw-j654321
    service.beta.kubernetes.io/alibaba-cloud-loadbalancer-ip-version: "DualStack"
  name: nginx
  namespace: default
spec:
  externalTrafficPolicy: Local
  ports:
  - name: tcp
    port: 80
    protocol: TCP
    targetPort: 80
  - name: https
    port: 443
    protocol: TCP
    targetPort: 443
  selector:
    app: nginx
  sessionAffinity: None
  loadBalancerClass: "alibabacloud.com/nlb"
  type: LoadBalancer

Add tags to the load balancer

Annotation: service.beta.kubernetes.io/alibaba-cloud-loadbalancer-additional-resource-tags

Description	Supported CCM versions
Add custom tags. Separate multiple tags with commas (,). Example: `k1=v1,k2=v2`. Starting from version 2.10.0, you can modify tags for existing and reused instances. Important If you add this annotation to a Service, any manual tag changes made in the console for the corresponding load balancer instance may be overwritten.	Version 2.5.0 or later

apiVersion: v1
kind: Service
metadata:
  annotations:
    service.beta.kubernetes.io/alibaba-cloud-loadbalancer-zone-maps: "${zone-A}:${vsw-A},${zone-B}:${vsw-B}" # For example, cn-hangzhou-k:vsw-i123456,cn-hangzhou-j:vsw-j654321
    service.beta.kubernetes.io/alibaba-cloud-loadbalancer-additional-resource-tags: "Key1=Value1,Key2=Value2"
  name: nginx
  namespace: default
spec:
  externalTrafficPolicy: Local
  ports:
  - name: tcp
    port: 80
    protocol: TCP
    targetPort: 80
  - name: https
    port: 443
    protocol: TCP
    targetPort: 443
  selector:
    app: nginx
  sessionAffinity: None
  loadBalancerClass: "alibabacloud.com/nlb"
  type: LoadBalancer

Use an existing load balancer

Annotation: Multiple annotations, as shown in the following table.

Annotation

Description

Supported CCM versions

service.beta.kubernetes.io/alibaba-cloud-loadbalancer-id

Specify the ID of an existing load balancer.

Version 2.5.0 or later

service.beta.kubernetes.io/alibaba-cloud-loadbalancer-force-override-listeners

Specify whether to synchronize listener configurations based on the Service. Valid values:

true: CCM creates, updates, or deletes NLB listeners based on the Service configuration.
false: CCM does not process NLB listeners.

Default value: false

apiVersion: v1
kind: Service
metadata:
  annotations:
    service.beta.kubernetes.io/alibaba-cloud-loadbalancer-id: "${your-nlb-id}" # NLB ID.
    service.beta.kubernetes.io/alibaba-cloud-loadbalancer-force-override-listeners: "true"
  name: nginx
  namespace: default
spec:
  externalTrafficPolicy: Local
  ports:
  - name: tcp
    port: 80
    protocol: TCP
    targetPort: 80
  - name: https
    port: 443
    protocol: TCP
    targetPort: 443
  selector:
    app: nginx
  sessionAffinity: None
  loadBalancerClass: "alibabacloud.com/nlb"
  type: LoadBalancer

Attach an Internet Shared Bandwidth package

Annotation: service.beta.kubernetes.io/alibaba-cloud-loadbalancer-bandwidth-package-id

Description

Supported CCM versions

Specify the ID of an Internet Shared Bandwidth package.

To query the ID, go to the VPC console.

Version 2.9.1 or later

apiVersion: v1
kind: Service
metadata:
  annotations:
    service.beta.kubernetes.io/alibaba-cloud-loadbalancer-zone-maps: "${zone-A}:${vsw-A},${zone-B}:${vsw-B}" # For example, cn-hangzhou-k:vsw-i123456,cn-hangzhou-j:vsw-j654321
    service.beta.kubernetes.io/alibaba-cloud-loadbalancer-bandwidth-package-id: "cbwp-xxxxxxxxxx" 
  name: nginx
  namespace: default
spec:
  externalTrafficPolicy: Local
  ports:
  - name: tcp
    port: 80
    protocol: TCP
    targetPort: 80
  - name: https
    port: 443
    protocol: TCP
    targetPort: 443
  selector:
    app: nginx
  sessionAffinity: None
  loadBalancerClass: "alibabacloud.com/nlb"
  type: LoadBalancer

Enable cross-zone forwarding

Annotation: service.beta.kubernetes.io/alibaba-cloud-loadbalancer-cross-zone-enabled

Description

Supported CCM versions

Enable or disable cross-zone forwarding. Valid values:

on
off

Default value: on

Version 2.13.0 or later

apiVersion: v1
kind: Service
metadata:
  annotations:
    service.beta.kubernetes.io/alibaba-cloud-loadbalancer-zone-maps: "${zone-A}:${vsw-A},${zone-B}:${vsw-B}" # For example, cn-hangzhou-k:vsw-i123456,cn-hangzhou-j:vsw-j654321
    service.beta.kubernetes.io/alibaba-cloud-loadbalancer-cross-zone-enabled: "off" 
  name: nginx
  namespace: default
spec:
  externalTrafficPolicy: Local
  ports:
  - name: tcp
    port: 80
    protocol: TCP
    targetPort: 80
  - name: https
    port: 443
    protocol: TCP
    targetPort: 443
  selector:
    app: nginx
  sessionAffinity: None
  loadBalancerClass: "alibabacloud.com/nlb"
  type: LoadBalancer

Specify the IPv6 Public or Private Network Type

Annotation: Multiple annotations, as shown in the following table.

Annotation

Description

Supported CCM versions

service.beta.kubernetes.io/alibaba-cloud-loadbalancer-ip-version

Specifies the protocol version for the NLB. You cannot change the IP type after you create the NLB. The kube-proxy proxy mode for the cluster must be IPVS. Valid values:

ipv4: Specifies the IPv4 type.
DualStack: Dual-stack. Supports both IPv4 and IPv6.
- The two vSwitches specified in service.beta.kubernetes.io/alibaba-cloud-loadbalancer-zone-maps must have IPv6 enabled.
- The generated IPv6 address is accessible only in IPv6-enabled environments.

Default value: ipv4

Version 2.5.0 or later

service.beta.kubernetes.io/alibaba-cloud-loadbalancer-ipv6-address-type

Specify the IPv6 network type for the NLB. Valid values:

intranet: Internal IPv6
internet: Internet-facing IPv6

Default value: intranet

Note

To enable Internet-facing IPv6, your VPC must contain an IPv6 gateway. For more information, see Create and manage an IPv6 gateway.

Version 2.9.1 or later

apiVersion: v1
kind: Service
metadata:
  annotations:
    service.beta.kubernetes.io/alibaba-cloud-loadbalancer-zone-maps: "${zone-A}:${vsw-A},${zone-B}:${vsw-B}" # For example, cn-hangzhou-k:vsw-i123456,cn-hangzhou-j:vsw-j654321
    service.beta.kubernetes.io/alibaba-cloud-loadbalancer-ip-version: "DualStack"
    service.beta.kubernetes.io/alibaba-cloud-loadbalancer-ipv6-address-type: internet # Set the IPv6 network type to Internet-facing.
  name: nginx
  namespace: default
spec:
  externalTrafficPolicy: Local
  ports:
  - name: tcp
    port: 80
    protocol: TCP
    targetPort: 80
  - name: https
    port: 443
    protocol: TCP
    targetPort: 443
  selector:
    app: nginx
  sessionAffinity: None
  loadBalancerClass: "alibabacloud.com/nlb"
  type: LoadBalancer

Retain automatically created load balancer instances when deleting a Service

Annotation: service.beta.kubernetes.io/alibaba-cloud-loadbalancer-preserve-lb-on-delete

Description

Supported CCM versions

When you delete a LoadBalancer-type Service, retain the NLB instance created by the Service. Remove the kubernetes.do.not.delete and ack.aliyun.com tags from the NLB instance and its vServer group. Keep existing servers in the vServer group.

When this feature is enabled, a Warning event of type PreservedOnDelete appears during Service synchronization. After you configure this annotation, check for this event to confirm that the feature is active.

Valid values:

A non-empty value: Enable the feature.
An empty value or no value: Disable the feature.

Important

Delete the Service instead of changing its type. Otherwise, the Service might be incorrectly re-associated with a previously retained NLB.

Version 2.10.0 or later

apiVersion: v1
kind: Service
metadata:
  annotations:
    service.beta.kubernetes.io/alibaba-cloud-loadbalancer-zone-maps: "${zone-A}:${vsw-A},${zone-B}:${vsw-B}" # For example, cn-hangzhou-k:vsw-i123456,cn-hangzhou-j:vsw-j654321
    service.beta.kubernetes.io/alibaba-cloud-loadbalancer-preserve-lb-on-delete: "true"
  name: nginx
  namespace: default
spec:
  externalTrafficPolicy: Local
  ports:
  - name: tcp
    port: 80
    protocol: TCP
    targetPort: 80
  - name: https
    port: 443
    protocol: TCP
    targetPort: 443
  selector:
    app: nginx
  loadBalancerClass: "alibabacloud.com/nlb"
  type: LoadBalancer

Typical listener operations

Set a security group for the listener

Annotation: service.beta.kubernetes.io/alibaba-cloud-loadbalancer-security-group-ids

Description	Supported CCM versions
Specify one or more security groups for the listener. Separate multiple values with commas (,). Example: `sg-aaaaa,sg-bbbbb`.	Version 2.6.0 or later

apiVersion: v1
kind: Service
metadata:
  annotations:
    service.beta.kubernetes.io/alibaba-cloud-loadbalancer-zone-maps: "${zone-A}:${vsw-A},${zone-B}:${vsw-B}" # For example, cn-hangzhou-k:vsw-i123456,cn-hangzhou-j:vsw-j654321
    service.beta.kubernetes.io/alibaba-cloud-loadbalancer-security-group-ids: "sg-aaaaa,sg-bbbbb" # Separate multiple security groups with commas.
  name: nginx
  namespace: default
spec:
  externalTrafficPolicy: Local
  ports:
  - name: tcp
    port: 80
    protocol: TCP
    targetPort: 80
  - name: https
    port: 443
    protocol: TCP
    targetPort: 443
  selector:
    app: nginx
  loadBalancerClass: "alibabacloud.com/nlb"
  type: LoadBalancer

Configure Both TCP and UDP Protocols for a Listener

Description	Supported CCM versions
Your Kubernetes cluster version must be 1.24 or later. To upgrade your cluster, see Upgrade the Kubernetes version of an ACK cluster.	Not applicable

apiVersion: v1
kind: Service
metadata:
  annotations:
      service.beta.kubernetes.io/alibaba-cloud-loadbalancer-zone-maps: "${zone-A}:${vsw-A},${zone-B}:${vsw-B}" # For example, cn-hangzhou-k:vsw-i123456,cn-hangzhou-j:vsw-j654321
  name: nginx
  namespace: default
spec:
  externalTrafficPolicy: Local
  ports:
  - name: tcp
    port: 80
    protocol: TCP
    targetPort: 80
  - name: udp
    port: 80
    protocol: UDP
    targetPort: 81
  selector:
    app: nginx
  sessionAffinity: None
  loadBalancerClass: "alibabacloud.com/nlb"
  type: LoadBalancer

Create a TCP listener

apiVersion: v1
kind: Service
metadata:
  annotations:
    service.beta.kubernetes.io/alibaba-cloud-loadbalancer-zone-maps: "${zone-A}:${vsw-A},${zone-B}:${vsw-B}" # For example, cn-hangzhou-k:vsw-i123456,cn-hangzhou-j:vsw-j654321
  name: nginx
  namespace: default
spec:
  externalTrafficPolicy: Local
  ports:
  - name: tcp
    port: 80
    protocol: TCP
    targetPort: 80
  - name: https
    port: 443
    protocol: TCP
    targetPort: 443
  selector:
    app: nginx
  sessionAffinity: None
  loadBalancerClass: "alibabacloud.com/nlb"
  type: LoadBalancer

Create a UDP listener

Note

If you create a UDP listener without specifying a health check, the system enables a TCP health check for the vServer group by default. When you use a UDP listener, we recommend that you explicitly configure a UDP health check or disable health checks.

apiVersion: v1
kind: Service
metadata:
  annotations:
    service.beta.kubernetes.io/alibaba-cloud-loadbalancer-zone-maps: "${zone-A}:${vsw-A},${zone-B}:${vsw-B}" # For example, cn-hangzhou-k:vsw-i123456,cn-hangzhou-j:vsw-j654321
  name: nginx
  namespace: default
spec:
  externalTrafficPolicy: Local
  ports:
  - name: udp
    port: 80
    protocol: UDP
    targetPort: 80
  selector:
    app: nginx
  sessionAffinity: None
  loadBalancerClass: "alibabacloud.com/nlb"
  type: LoadBalancer

Create a TCP/SSL listener

Annotation: Multiple annotations, as shown in the following table.

Annotation	Description	Supported CCM versions
service.beta.kubernetes.io/alibaba-cloud-loadbalancer-protocol-port	Specify the protocol for the listener. Separate multiple values with commas (,). Example: `TCP:80,TCPSSL:443`.	Version 2.5.0 or later
service.beta.kubernetes.io/alibaba-cloud-loadbalancer-cert-id	Specify the server certificate ID. To create and view a certificate, go to the Certificate Management Service console, and then click SSL Certificates. The following figure shows an example.	Version 2.5.0 or later

Important

Currently, TCP/SSL listeners do not support attaching to server groups that have the client address preservation feature enabled—that is, you cannot configure both the service.beta.kubernetes.io/alibaba-cloud-loadbalancer-protocol-port: "tcpssl:${port}" and service.beta.kubernetes.io/alibaba-cloud-loadbalancer-preserve-client-ip: "on" annotations simultaneously. To obtain the originating IP address through NLB, see Obtaining the Originating IP Address Through NLB.

apiVersion: v1
kind: Service
metadata:
  annotations:
    service.beta.kubernetes.io/alibaba-cloud-loadbalancer-zone-maps: "${zone-A}:${vsw-A},${zone-B}:${vsw-B}" # For example, cn-hangzhou-k:vsw-i123456,cn-hangzhou-j:vsw-j654321
    service.beta.kubernetes.io/alibaba-cloud-loadbalancer-protocol-port: "tcpssl:443"
    service.beta.kubernetes.io/alibaba-cloud-loadbalancer-cert-id: "${CertIdentifier}"
  name: nginx
  namespace: default
spec:
  externalTrafficPolicy: Local
  ports:
  - name: tcp
    port: 80
    protocol: TCP
    targetPort: 80
  - name: https
    port: 443
    protocol: TCP
    targetPort: 80
  selector:
    app: nginx
  loadBalancerClass: "alibabacloud.com/nlb"
  type: LoadBalancer

Configure a listener port range

Annotation: service.beta.kubernetes.io/alibaba-cloud-loadbalancer-listener-port-range

Description

Supported CCM versions

This annotation is supported only when you use the Terway network plug-in.

Configure a port range for full-port forwarding to listen on specific ports and forward traffic to the corresponding ports on backend servers.

Format: port range:Service port. Separate multiple entries with commas (,). Example: 80-100:80,400-500:443. Port ranges for the same protocol must not overlap.

The targetPort is the health check port on the backend server. It must be an integer between 1 and 65535.

Version 2.11.4 or later

apiVersion: v1
kind: Service
metadata:
  annotations:
    service.beta.kubernetes.io/alibaba-cloud-loadbalancer-zone-maps: "${zone-A}:${vsw-A},${zone-B}:${vsw-B}" # For example, cn-hangzhou-k:vsw-i123456,cn-hangzhou-j:vsw-j654321
    # Configure listeners for port ranges 80-100 and 400-500.
    service.beta.kubernetes.io/alibaba-cloud-loadbalancer-listener-port-range: "80-100:80,400-500:443"
  name: nginx
  namespace: default
spec:
  externalTrafficPolicy: Local
  ports:
  - name: tcp
    port: 80
    protocol: TCP
    # targetPort is the health check port on the backend server. It must be an integer.
    targetPort: 80
  - name: https
    port: 443
    protocol: TCP
    targetPort: 80
  selector:
    app: nginx
  loadBalancerClass: "alibabacloud.com/nlb"
  type: LoadBalancer

Enable mutual authentication

Annotation: Multiple annotations, as shown in the following table.

Annotation	Description	Supported CCM versions
service.beta.kubernetes.io/alibaba-cloud-loadbalancer-protocol-port	Specify the protocol for the listener. Separate multiple values with commas (,). Example: `TCP:80,TCPSSL:443`.	Version 2.5.0 or later
service.beta.kubernetes.io/alibaba-cloud-loadbalancer-cert-id	Specify the server certificate ID. To create and view a certificate, go to the Certificate Management Service console, and then click SSL Certificates. The following figure shows an example.
service.beta.kubernetes.io/alibaba-cloud-loadbalancer-cacert-id	Specify the CA certificate ID. To view the ID, go to the Certificate Management Service console, and then click PCA Certificates.
service.beta.kubernetes.io/alibaba-cloud-loadbalancer-cacert	Enable or disable mutual authentication. Valid values: `on`: Enable. `off`: Disable. Default value: `off`

apiVersion: v1
kind: Service
metadata:
  annotations:
    service.beta.kubernetes.io/alibaba-cloud-loadbalancer-zone-maps: "${zone-A}:${vsw-A},${zone-B}:${vsw-B}" # For example, cn-hangzhou-k:vsw-i123456,cn-hangzhou-j:vsw-j654321
    service.beta.kubernetes.io/alibaba-cloud-loadbalancer-protocol-port: "tcpssl:443"   
    service.beta.kubernetes.io/alibaba-cloud-loadbalancer-cert-id: "${CertIdentifier}" 
    service.beta.kubernetes.io/alibaba-cloud-loadbalancer-cacert-id: "${your-cacert-id}" 
    service.beta.kubernetes.io/alibaba-cloud-loadbalancer-cacert: "on"
  name: nginx
  namespace: default
spec:
  externalTrafficPolicy: Local
  ports:
  - name: tcp
    port: 80
    protocol: TCP
    targetPort: 80
  - name: https
    port: 443
    protocol: TCP
    targetPort: 80
  selector:
    app: nginx
  loadBalancerClass: "alibabacloud.com/nlb"
  type: LoadBalancer

Set the TLS security policy

Annotation: Multiple annotations, as shown in the following table.

Annotation	Description	Supported CCM versions
service.beta.kubernetes.io/alibaba-cloud-loadbalancer-protocol-port	Specify the protocol for the listener. Separate multiple values with commas (,). Example: `TCP:80,TCPSSL:443`.	Version 2.5.0 or later
service.beta.kubernetes.io/alibaba-cloud-loadbalancer-cert-id	Specify the server certificate ID. To create and view a certificate, go to the Certificate Management Service console, and then click SSL Certificates. The following figure shows an example.
service.beta.kubernetes.io/alibaba-cloud-loadbalancer-tls-cipher-policy	Specify the security policy ID. Supported policies include system-defined and custom policies. Valid values: `tls_cipher_policy_1_0` `tls_cipher_policy_1_1` `tls_cipher_policy_1_2` `tls_cipher_policy_1_2_strict` `tls_cipher_policy_1_2_strict_with_1_3` Default value: `tls_cipher_policy_1_0`

apiVersion: v1
kind: Service
metadata:
  annotations:
    service.beta.kubernetes.io/alibaba-cloud-loadbalancer-zone-maps: "${zone-A}:${vsw-A},${zone-B}:${vsw-B}" # For example, cn-hangzhou-k:vsw-i123456,cn-hangzhou-j:vsw-j654321
    service.beta.kubernetes.io/alibaba-cloud-loadbalancer-protocol-port: "tcpssl:443"
    service.beta.kubernetes.io/alibaba-cloud-loadbalancer-cert-id: "${CertIdentifier}" 
    service.beta.kubernetes.io/alibaba-cloud-loadbalancer-tls-cipher-policy: "tls_cipher_policy_1_0"
  name: nginx
  namespace: default
spec:
  externalTrafficPolicy: Local
  ports:
  - name: tcp
    port: 80
    protocol: TCP
    targetPort: 80
  - name: https
    port: 443
    protocol: TCP
    targetPort: 80
  selector:
    app: nginx
  loadBalancerClass: "alibabacloud.com/nlb"
  type: LoadBalancer

Enable Proxy Protocol

Annotation: service.beta.kubernetes.io/alibaba-cloud-loadbalancer-proxy-protocol

Description

Supported CCM versions

Enable or disable Proxy Protocol to carry the source IP address of clients to backend servers. Valid values:

on: Enable.
off: Disable.

Default value: off

Important

Before enabling Proxy Protocol, verify that your backend services support Proxy Protocol v2. Otherwise, requests fail.

Version 2.5.0 or later

apiVersion: v1
kind: Service
metadata:
  annotations:
    service.beta.kubernetes.io/alibaba-cloud-loadbalancer-zone-maps: "${zone-A}:${vsw-A},${zone-B}:${vsw-B}" # For example, cn-hangzhou-k:vsw-i123456,cn-hangzhou-j:vsw-j654321
    service.beta.kubernetes.io/alibaba-cloud-loadbalancer-proxy-protocol: "on"
  name: nginx
  namespace: default
spec:
  externalTrafficPolicy: Local
  ports:
  - name: tcp
    port: 80
    protocol: TCP
    targetPort: 80
  - name: https
    port: 443
    protocol: TCP
    targetPort: 443
  selector:
    app: nginx
  loadBalancerClass: "alibabacloud.com/nlb"
  type: LoadBalancer

Carry additional information with Proxy Protocol

Annotation: Multiple annotations, as shown in the following table.

Annotation	Description	Supported CCM versions
service.beta.kubernetes.io/alibaba-cloud-loadbalancer-proxy-protocol	Enable or disable Proxy Protocol to carry the source IP address of clients to backend servers. Valid values: `on`: Enable. `off`: Disable. Default value: `off` Important Before enabling Proxy Protocol, verify that your backend services support Proxy Protocol v2. Otherwise, requests fail.	Version 2.5.0 or later
service.beta.kubernetes.io/alibaba-cloud-loadbalancer-ppv2-pvl-vpc-id-enabled	Enable or disable carrying the VPC ID to backend servers using Proxy Protocol. Valid values: `on`: Enable. `off`: Disable. Default value: `off`	Version 2.9.1 or later
service.beta.kubernetes.io/alibaba-cloud-loadbalancer-ppv2-pvl-ep-id-enabled	Enable or disable carrying the PrivateLink endpoint ID to backend servers using Proxy Protocol. Valid values: `on`: Enable. `off`: Disable. Default value: `off`	Version 2.9.1 or later
service.beta.kubernetes.io/alibaba-cloud-loadbalancer-ppv2-pvl-eps-id-enabled	Enable or disable carrying the PrivateLink endpoint service ID to backend servers using Proxy Protocol. Valid values: `on`: Enable. `off`: Disable. Default value: `off`	Version 2.9.1 or later

apiVersion: v1
kind: Service
metadata:
  annotations:
    service.beta.kubernetes.io/alibaba-cloud-loadbalancer-zone-maps: "${zone-A}:${vsw-A},${zone-B}:${vsw-B}" # For example, cn-hangzhou-k:vsw-i123456,cn-hangzhou-j:vsw-j654321
    service.beta.kubernetes.io/alibaba-cloud-loadbalancer-proxy-protocol: "on"
    service.beta.kubernetes.io/alibaba-cloud-loadbalancer-ppv2-pvl-ep-id-enabled: "on"
    service.beta.kubernetes.io/alibaba-cloud-loadbalancer-ppv2-pvl-eps-id-enabled: "on"
    service.beta.kubernetes.io/alibaba-cloud-loadbalancer-ppv2-pvl-vpc-id-enabled: "on"
  name: nginx
  namespace: default
spec:
  externalTrafficPolicy: Local
  ports:
  - name: tcp
    port: 80
    protocol: TCP
    targetPort: 80
  - name: https
    port: 443
    protocol: TCP
    targetPort: 443
  selector:
    app: nginx
  loadBalancerClass: "alibabacloud.com/nlb"
  type: LoadBalancer

Set the new connection rate limit per second for a listener

Annotation: service.beta.kubernetes.io/alibaba-cloud-loadbalancer-cps

Description	Supported CCM versions
Set the maximum number of new connections per second for the NLB instance. Valid values: 0 to 1000000. A value of 0 means no limit.	Version 2.5.0 or later

apiVersion: v1
kind: Service
metadata:
  annotations:
    service.beta.kubernetes.io/alibaba-cloud-loadbalancer-zone-maps: "${zone-A}:${vsw-A},${zone-B}:${vsw-B}" # For example, cn-hangzhou-k:vsw-i123456,cn-hangzhou-j:vsw-j654321
    service.beta.kubernetes.io/alibaba-cloud-loadbalancer-cps: "100"
  name: nginx
  namespace: default
spec:
  externalTrafficPolicy: Local
  ports:
  - name: tcp
    port: 80
    protocol: TCP
    targetPort: 80
  - name: https
    port: 443
    protocol: TCP
    targetPort: 443
  selector:
    app: nginx
  loadBalancerClass: "alibabacloud.com/nlb"
  type: LoadBalancer

Set the idle timeout for listener connections

Annotation: service.beta.kubernetes.io/alibaba-cloud-loadbalancer-idle-timeout

Description

Supported CCM versions

Set the idle timeout for connections. Unit: seconds. Valid values: 10 to 900.

Default value: 900

Version 2.5.0 or later

apiVersion: v1
kind: Service
metadata:
  annotations:
    service.beta.kubernetes.io/alibaba-cloud-loadbalancer-zone-maps: "${zone-A}:${vsw-A},${zone-B}:${vsw-B}" # For example, cn-hangzhou-k:vsw-i123456,cn-hangzhou-j:vsw-j654321
    service.beta.kubernetes.io/alibaba-cloud-loadbalancer-idle-timeout: "60"
  name: nginx
  namespace: default
spec:
  externalTrafficPolicy: Local
  ports:
  - name: tcp
    port: 80
    protocol: TCP
    targetPort: 80
  - name: https
    port: 443
    protocol: TCP
    targetPort: 443
  selector:
    app: nginx
  loadBalancerClass: "alibabacloud.com/nlb"
  type: LoadBalancer

Set the ALPN policy

Annotation: Multiple annotations, as shown in the following table.

Annotation	Description	Supported CCM versions
service.beta.kubernetes.io/alibaba-cloud-loadbalancer-protocol-port	Specify the protocol for the listener. Separate multiple values with commas (,). Example: `TCP:80,TCPSSL:443`.	Version 2.5.0 or later
service.beta.kubernetes.io/alibaba-cloud-loadbalancer-cert-id	Specify the server certificate ID. To create and view a certificate, go to the Certificate Management Service console, and then click SSL Certificates. The following figure shows an example.	Version 2.5.0 or later
service.beta.kubernetes.io/alibaba-cloud-loadbalancer-alpn	Enable or disable ALPN. Valid values: `on`: Enable. `off`: Disable. Default value: `off`	Version 2.10.0 or later
service.beta.kubernetes.io/alibaba-cloud-loadbalancer-alpn-policy	Specify the ALPN policy. Valid values: `HTTP1Only`: Negotiate only HTTP/1.x. Priority order: HTTP/1.1 > HTTP/1.0. `HTTP2Only`: Negotiate only HTTP/2.0. `HTTP2Optional`: Prefer HTTP/1.x but accept HTTP/2.0. Priority order: HTTP/1.1 > HTTP/1.0 > HTTP/2.0. `HTTP2Preferred`: Prefer HTTP/2 but accept HTTP/1.x. Priority order: HTTP/2.0 > HTTP/1.1 > HTTP/1.0. For more valid values, see the `AlpnPolicy` field in the CreateListener API.	Version 2.10.0 or later

apiVersion: v1
kind: Service
metadata:
  annotations:
    service.beta.kubernetes.io/alibaba-cloud-loadbalancer-zone-maps: "${zone-A}:${vsw-A},${zone-B}:${vsw-B}" # For example, cn-hangzhou-k:vsw-i123456,cn-hangzhou-j:vsw-j654321
    service.beta.kubernetes.io/alibaba-cloud-loadbalancer-protocol-port: "tcpssl:443"
    service.beta.kubernetes.io/alibaba-cloud-loadbalancer-cert-id: "${CertIdentifier}" 
    service.beta.kubernetes.io/alibaba-cloud-loadbalancer-alpn: "on"
    service.beta.kubernetes.io/alibaba-cloud-loadbalancer-alpn-policy: "HTTP1Only" 
  name: nginx
  namespace: default
spec:
  externalTrafficPolicy: Local
  ports:
  - name: tcp
    port: 80
    protocol: TCP
    targetPort: 80
  - name: https
    port: 443
    protocol: TCP
    targetPort: 80
  selector:
    app: nginx
  loadBalancerClass: "alibabacloud.com/nlb"
  type: LoadBalancer

Configure extended certificates

Annotation	Description	Supported CCM versions
service.beta.kubernetes.io/alibaba-cloud-loadbalancer-protocol-port	Specify the protocol for the listener. Separate multiple values with commas (,). Example: `TCP:80,TCPSSL:443`.	Version 2.5.0 or later
service.beta.kubernetes.io/alibaba-cloud-loadbalancer-cert-id	Specify the server certificate ID. To create and view a certificate, go to the Certificate Management Service console, and then click SSL Certificates. The following figure shows an example.	Version 2.5.0 or later
service.beta.kubernetes.io/alibaba-cloud-loadbalancer-additional-cert-ids	Specify one or more extended certificate IDs. Separate multiple IDs with commas (,). To create and view certificates, go to the Certificate Management Service console, and then click SSL Certificates. For example, see the following graph:

apiVersion: v1
kind: Service
metadata:
  annotations:
    service.beta.kubernetes.io/alibaba-cloud-loadbalancer-zone-maps: "${zone-A}:${vsw-A},${zone-B}:${vsw-B}" # For example, cn-hangzhou-k:vsw-i123456,cn-hangzhou-j:vsw-j654321
    service.beta.kubernetes.io/alibaba-cloud-loadbalancer-protocol-port: "tcpssl:443"
    service.beta.kubernetes.io/alibaba-cloud-loadbalancer-cert-id: "${CertIdentifier}" 
    service.beta.kubernetes.io/alibaba-cloud-loadbalancer-additional-cert-ids: "${CertIdentifier-1},${CertIdentifier-2}" # Separate multiple extended certificates with commas.
  name: nginx
  namespace: default
spec:
  externalTrafficPolicy: Local
  ports:
  - name: tcp
    port: 80
    protocol: TCP
    targetPort: 80
  - name: https
    port: 443
    protocol: TCP
    targetPort: 80
  selector:
    app: nginx
  loadBalancerClass: "alibabacloud.com/nlb"
  type: LoadBalancer

Typical vServer group operations

Set the scheduling algorithm

Annotation: service.beta.kubernetes.io/alibaba-cloud-loadbalancer-scheduler

Description

Supported CCM versions

Specify the scheduling algorithm. Valid values:

wrr: Weighted round-robin. Servers with higher weights receive more requests.
rr: Round-robin. Requests are distributed to servers in sequence.
sch: Source IP hash. Requests from the same source IP address are routed to the same server.
tch: Four-tuple hash. Based on source IP, destination IP, source port, and destination port. Traffic from the same flow is routed to the same server.
wlc: Weighted least connections. Considers both server weights and current connection counts. When weights are equal, servers with fewer connections receive more requests.

Default value: wrr

For more valid values, see the Scheduler field in the CreateVServerGroup API.

Version 2.5.0 or later

apiVersion: v1
kind: Service
metadata:
  annotations:
    service.beta.kubernetes.io/alibaba-cloud-loadbalancer-zone-maps: "${zone-A}:${vsw-A},${zone-B}:${vsw-B}" # For example, cn-hangzhou-k:vsw-i123456,cn-hangzhou-j:vsw-j654321
    service.beta.kubernetes.io/alibaba-cloud-loadbalancer-scheduler: "sch"
  name: nginx
  namespace: default
spec:
  externalTrafficPolicy: Local
  ports:
  - name: tcp
    port: 80
    protocol: TCP
    targetPort: 80
  - name: https
    port: 443
    protocol: TCP
    targetPort: 443
  selector:
    app: nginx
  loadBalancerClass: "alibabacloud.com/nlb"
  type: LoadBalancer

Enable connection draining

Annotation: Multiple annotations, as shown in the following table.

Annotation

Description

Supported CCM versions

service.beta.kubernetes.io/alibaba-cloud-loadbalancer-connection-drain

Enable or disable connection draining. When enabled, the following actions occur if a backend server is removed or fails a health check:

on: Existing connections continue normally for a period. After the timeout, connections close gracefully to ensure smooth service termination.
off: Existing connections remain open until clients disconnect or long-lived sessions expire.

Default value: off

Version 2.5.0 or later

service.beta.kubernetes.io/alibaba-cloud-loadbalancer-connection-drain-timeout

Set the connection draining timeout.

Unit: seconds.
Valid values: 0 to 900. A value of 0 closes connections immediately.

apiVersion: v1
kind: Service
metadata:
  annotations:
    service.beta.kubernetes.io/alibaba-cloud-loadbalancer-zone-maps: "${zone-A}:${vsw-A},${zone-B}:${vsw-B}" # For example, cn-hangzhou-k:vsw-i123456,cn-hangzhou-j:vsw-j654321
    service.beta.kubernetes.io/alibaba-cloud-loadbalancer-connection-drain: "on"
    service.beta.kubernetes.io/alibaba-cloud-loadbalancer-connection-drain-timeout: "30"
  name: nginx
  namespace: default
spec:
  externalTrafficPolicy: Local
  ports:
  - name: tcp
    port: 80
    protocol: TCP
    targetPort: 80
  - name: https
    port: 443
    protocol: TCP
    targetPort: 443
  selector:
    app: nginx
  loadBalancerClass: "alibabacloud.com/nlb"
  type: LoadBalancer

Preserve client IP addresses

Annotation: service.beta.kubernetes.io/alibaba-cloud-loadbalancer-preserve-client-ip

Description

Supported CCM versions

Enable or disable client IP address preservation. Valid values:

on: Enable.
off: Disable.

Default value: on

Version 2.5.0 or later

apiVersion: v1
kind: Service
metadata:
  annotations:
    service.beta.kubernetes.io/alibaba-cloud-loadbalancer-zone-maps: "${zone-A}:${vsw-A},${zone-B}:${vsw-B}" # For example, cn-hangzhou-k:vsw-i123456,cn-hangzhou-j:vsw-j654321
    service.beta.kubernetes.io/alibaba-cloud-loadbalancer-preserve-client-ip: "on"
  name: nginx
  namespace: default
spec:
  externalTrafficPolicy: Local
  ports:
  - name: tcp
    port: 80
    protocol: TCP
    targetPort: 80
  - name: https
    port: 443
    protocol: TCP
    targetPort: 443
  selector:
    app: nginx
  loadBalancerClass: "alibabacloud.com/nlb"
  type: LoadBalancer

Configure TCP health checks

Annotation: Multiple annotations, as shown in the following table. To configure TCP health checks, you must specify all these annotations. TCP health checks are enabled by default.

Annotation	Description
service.beta.kubernetes.io/alibaba-cloud-loadbalancer-health-check-flag	Enable or disable health checks. Valid values: `on`: Enable. `off`: Disable. Default value: `on`
service.beta.kubernetes.io/alibaba-cloud-loadbalancer-health-check-type	Specify the health check protocol. Valid values: `tcp` `http` Default value: `tcp`
service.beta.kubernetes.io/alibaba-cloud-loadbalancer-health-check-connect-port	Specify the server port for health checks. Valid values: 0 to 65535. Default value: 0, which means use the server port.
service.beta.kubernetes.io/alibaba-cloud-loadbalancer-health-check-connect-timeout	Set the maximum timeout for health check responses. Unit: seconds. Valid values: 1 to 300. Default value: 5
service.beta.kubernetes.io/alibaba-cloud-loadbalancer-healthy-threshold	Set the number of consecutive successful health checks required to mark a server as healthy. Valid values: 2 to 10. Default value: 2
service.beta.kubernetes.io/alibaba-cloud-loadbalancer-unhealthy-threshold	Set the number of consecutive failed health checks required to mark a server as unhealthy. Valid values: 2 to 10. Default value: 2
service.beta.kubernetes.io/alibaba-cloud-loadbalancer-health-check-interval	Set the interval between health checks. Unit: seconds. Valid values: 1 to 50. Default value: 10

apiVersion: v1
kind: Service
metadata:
  annotations:
    service.beta.kubernetes.io/alibaba-cloud-loadbalancer-zone-maps: "${zone-A}:${vsw-A},${zone-B}:${vsw-B}" # For example, cn-hangzhou-k:vsw-i123456,cn-hangzhou-j:vsw-j654321
    service.beta.kubernetes.io/alibaba-cloud-loadbalancer-health-check-flag: "on"
    service.beta.kubernetes.io/alibaba-cloud-loadbalancer-health-check-type: "tcp"
    service.beta.kubernetes.io/alibaba-cloud-loadbalancer-health-check-connect-timeout: "8"
    service.beta.kubernetes.io/alibaba-cloud-loadbalancer-healthy-threshold: "4"
    service.beta.kubernetes.io/alibaba-cloud-loadbalancer-unhealthy-threshold: "4"
    service.beta.kubernetes.io/alibaba-cloud-loadbalancer-health-check-interval: "5"
  name: nginx
  namespace: default
spec:
  externalTrafficPolicy: Local
  ports:
  - name: tcp
    port: 80
    protocol: TCP
    targetPort: 80
  - name: https
    port: 443
    protocol: TCP
    targetPort: 443
  selector:
    app: nginx
  loadBalancerClass: "alibabacloud.com/nlb"
  type: LoadBalancer

Configure HTTP health checks

Annotation: Multiple annotations, as shown in the following table. To configure HTTP health checks, you must specify all these annotations. TCP health checks are enabled by default.

Annotation	Description	Supported CCM versions
service.beta.kubernetes.io/alibaba-cloud-loadbalancer-health-check-flag	Enable or disable health checks. Valid values: `on`: Enable. `off`: Disable. Default value: `on`	Version 2.5.0 or later
service.beta.kubernetes.io/alibaba-cloud-loadbalancer-health-check-type	Specify the health check protocol. Valid values: `tcp` `http` Default value: `tcp`
service.beta.kubernetes.io/alibaba-cloud-loadbalancer-health-check-uri	Specify the health check path. Length: 1 to 80 characters. Use only letters, digits, and hyphens (-). Must start with a forward slash (/). For more information, see CreateServerGroup. Note This parameter takes effect only when `service.beta.kubernetes.io/alibaba-cloud-loadbalancer-health-check-type` is set to `HTTP`.
service.beta.kubernetes.io/alibaba-cloud-loadbalancer-health-check-domain	Specify the domain name for health checks. Valid values: $SERVER_IP: Use the private IP address of the backend server. domain: Specify a custom domain name. Length: 1 to 80 characters. Use only lowercase letters, digits, hyphens (-), and periods (.). Note This parameter takes effect only when `service.beta.kubernetes.io/alibaba-cloud-loadbalancer-health-check-type` is set to `HTTP`.
service.beta.kubernetes.io/alibaba-cloud-loadbalancer-health-check-connect-port	Specify the server port for health checks. Valid values: 0 to 65535. Default value: 0, which means use the server port.
service.beta.kubernetes.io/alibaba-cloud-loadbalancer-health-check-connect-timeout	Set the maximum timeout for health check responses. Unit: seconds. Valid values: 1 to 300. Default value: 5
service.beta.kubernetes.io/alibaba-cloud-loadbalancer-healthy-threshold	Set the number of consecutive successful health checks required to mark a server as healthy. Valid values: 2 to 10. Default value: 2
service.beta.kubernetes.io/alibaba-cloud-loadbalancer-unhealthy-threshold	Set the number of consecutive failed health checks required to mark a server as unhealthy. Valid values: 2 to 10. Default value: 2
service.beta.kubernetes.io/alibaba-cloud-loadbalancer-health-check-interval	Set the interval between health checks. Unit: seconds. Valid values: 1 to 50. Default value: 10
service.beta.kubernetes.io/alibaba-cloud-loadbalancer-health-check-method	Specify the health check method. Valid values: `GET` `HEAD` Note This parameter takes effect only when `service.beta.kubernetes.io/alibaba-cloud-loadbalancer-health-check-type` is set to `HTTP`.

apiVersion: v1
kind: Service
metadata:
  annotations:
    service.beta.kubernetes.io/alibaba-cloud-loadbalancer-zone-maps: "${zone-A}:${vsw-A},${zone-B}:${vsw-B}" # For example, cn-hangzhou-k:vsw-i123456,cn-hangzhou-j:vsw-j654321
    service.beta.kubernetes.io/alibaba-cloud-loadbalancer-health-check-flag: "on"
    service.beta.kubernetes.io/alibaba-cloud-loadbalancer-health-check-type: "http"
    service.beta.kubernetes.io/alibaba-cloud-loadbalancer-health-check-uri: "/test/index.html"
    service.beta.kubernetes.io/alibaba-cloud-loadbalancer-health-check-domain: "www.test.com"
    service.beta.kubernetes.io/alibaba-cloud-loadbalancer-healthy-threshold: "4"
    service.beta.kubernetes.io/alibaba-cloud-loadbalancer-unhealthy-threshold: "4"
    service.beta.kubernetes.io/alibaba-cloud-loadbalancer-health-check-connect-timeout: "10"
    service.beta.kubernetes.io/alibaba-cloud-loadbalancer-health-check-interval: "5"
    # Optional annotation to set the health check method.
    service.beta.kubernetes.io/alibaba-cloud-loadbalancer-health-check-method: "head"
  name: nginx
  namespace: default
spec:
  externalTrafficPolicy: Local
  ports:
  - name: tcp
    port: 80
    protocol: TCP
    targetPort: 80
  - name: https
    port: 443
    protocol: TCP
    targetPort: 443
  selector:
    app: nginx
  loadBalancerClass: "alibabacloud.com/nlb"
  type: LoadBalancer

Set the vServer group type

Annotation: service.beta.kubernetes.io/alibaba-cloud-loadbalancer-server-group-type

Description

Supported CCM versions

Specify the vServer group type. Valid values:

Ip: IP address type. Supports adding backend servers using IP addresses.
Instance (default): Server type. Supports adding ECS instances and ENIs.

Default value: Instance

For more information about NLB vServer group types, see NLB vServer groups.

Version 2.8.0 or later

apiVersion: v1
kind: Service
metadata:
  annotations:
    service.beta.kubernetes.io/alibaba-cloud-loadbalancer-zone-maps: "${zone-A}:${vsw-A},${zone-B}:${vsw-B}" # For example, cn-hangzhou-k:vsw-i123456,cn-hangzhou-j:vsw-j654321
    service.beta.kubernetes.io/alibaba-cloud-loadbalancer-server-group-type: "Ip"
  name: nginx
  namespace: default
spec:
  externalTrafficPolicy: Local
  ports:
  - name: tcp
    port: 80
    protocol: TCP
    targetPort: 80
  - name: https
    port: 443
    protocol: TCP
    targetPort: 443
  selector:
    app: nginx
  loadBalancerClass: "alibabacloud.com/nlb"
  type: LoadBalancer

Reuse an existing vServer group

Annotation: service.beta.kubernetes.io/alibaba-cloud-loadbalancer-vgroup-port

Reuse an existing vServer group. This annotation applies only when you reuse an existing NLB. For usage examples, see Deploy services across clusters by reusing an existing load balancer.

Set the traffic weight for the Service

Annotation: service.beta.kubernetes.io/alibaba-cloud-loadbalancer-weight

In scenarios where multiple Services reuse the same NLB, use this annotation to set the traffic weight for the current Service. This annotation applies only when you reuse an existing vServer group. For usage examples, see Deploy services across clusters by reusing an existing load balancer.

Ignore backend server weight updates

Annotation: service.beta.kubernetes.io/alibaba-cloud-loadbalancer-ignore-weight-update

Description

Supported CCM versions

Skip updating backend server weights in the vServer group during Service synchronization. Use this annotation when you manually manage backend server weights outside of CCM. Valid values:

on
off

Default value: off

Version 2.11.1 or later

apiVersion: v1
kind: Service
metadata:
  annotations:
    service.beta.kubernetes.io/alibaba-cloud-loadbalancer-zone-maps: "${zone-A}:${vsw-A},${zone-B}:${vsw-B}" # For example, cn-hangzhou-k:vsw-i123456,cn-hangzhou-j:vsw-j654321
    service.beta.kubernetes.io/alibaba-cloud-loadbalancer-ignore-weight-update: "on"
  name: nginx
  namespace: default
spec:
  externalTrafficPolicy: Local
  ports:
  - name: tcp
    port: 80
    protocol: TCP
    targetPort: 80
  - name: https
    port: 443
    protocol: TCP
    targetPort: 443
  selector:
    app: nginx
  loadBalancerClass: "alibabacloud.com/nlb"
  type: LoadBalancer