Annotations used by ACK to configure NLB instances - Container Service for Kubernetes

You can add annotations to the YAML file of a Service to configure load balancing. Network Load Balancer (NLB) instances are next-generation Layer 4 load balancers developed by Alibaba Cloud for the Internet of Everything (IoE). NLB provides ultrahigh performance and auto scaling. This topic describes how to use annotations to configure load balancing. You can configure NLB instances, listeners, and backend server groups.

Category	Hyperlink
Annotation usage notes
Common operations to manage NLB instances	Create an Internet-facing NLB instance
	Create an internal-facing NLB instance
	Specify the name of an NLB instance
	Specify the resource group to which an NLB instance belongs
	Create a dual-stack NLB instance
	Add additional tags to an NLB instance
	Use an existing NLB instance
Common operations to manage listeners	Configure security groups for a listener
	Configure both TCP and UDP for a listener
	Create a TCP listener
	Create a UDP listener
	Create a TCP/SSL listener
	Enable mutual authentication
	Specify a TLS security policy
	Configure Proxy Protocol
	Specify the maximum number of connections that can be created per second
	Specify the timeout period of idle connections for a listener
Common operations to manage backend server groups	Set the scheduling algorithm
	Configure connection draining
	Configure client IP preservation
	Configure TCP health checks
	Configure HTTP health checks
	Specify the backend server group type

Precautions

The Kubernetes version of your cluster must be V1.24 or later and the cloud controller manager (CCM) version must be V2.5.0 or later.
To configure an NLB instance for a Service, set the spec.loadBalancerClass parameter of the Service to alibabacloud.com/nlb. Otherwise, a Classic Load Balancer (CLB) instance is created by default.
You cannot modify the spec.loadBalancerClass parameter of a Service to change the type of Server Load Balancer (SLB) instance after the Service is created.
You cannot use the Container Service for Kubernetes (ACK) console to manage NLB instances. To manage NLB instances, use kubectl.

Common operations to manage NLB instances

Create an Internet-facing NLB instance

Annotation: service.beta.kubernetes.io/alibaba-cloud-loadbalancer-zone-maps

Description

Default

Supported CCM version

The zones of the NLB instance.

You can log on to the NLB console to view the regions and zones that support NLB. Select at least two zones for each NLB instance. Separate multiple zones with commas (,). Example: cn-hangzhou-k:vsw-i123456,cn-hangzhou-j:vsw-j654321.

None

CCM 2.5.0 and later

apiVersion: v1
kind: Service
metadata:
  annotations:
    service.beta.kubernetes.io/alibaba-cloud-loadbalancer-zone-maps: "${zone-A}:${vsw-A},${zone-B}:${vsw-B}" # Example: cn-hangzhou-k:vsw-i123456,cn-hangzhou-j:vsw-j654321. 
  name: nginx
  namespace: default
spec:
  externalTrafficPolicy: Local
  ports:
  - name: tcp
    port: 80
    protocol: TCP
    targetPort: 80
  - name: https
    port: 443
    protocol: TCP
    targetPort: 443
  selector:
    app: nginx
  loadBalancerClass: "alibabacloud.com/nlb"
  type: LoadBalancer

Create an internal-facing NLB instance

Annotation: service.beta.kubernetes.io/alibaba-cloud-loadbalancer-address-type

Description

Default

Supported CCM version

Set the network type of an NLB instance to internal-facing. You can change the value of the annotation to change the network type of an NLB instance. Valid values:

internet: Internet-facing.
intranet: internal-facing.

internet

CCM 2.5.0 and later

apiVersion: v1
kind: Service
metadata:
  annotations:
    service.beta.kubernetes.io/alibaba-cloud-loadbalancer-zone-maps: "${zone-A}:${vsw-A},${zone-B}:${vsw-B}" # Example: cn-hangzhou-k:vsw-i123456,cn-hangzhou-j:vsw-j654321. 
    service.beta.kubernetes.io/alibaba-cloud-loadbalancer-address-type: "intranet"
  name: nginx
  namespace: default
spec:
  externalTrafficPolicy: Local
  ports:
  - name: tcp
    port: 80
    protocol: TCP
    targetPort: 80
  - name: https
    port: 443
    protocol: TCP
    targetPort: 443
  selector:
    app: nginx
  loadBalancerClass: "alibabacloud.com/nlb"
  type: LoadBalancer

Specify the name of an NLB instance

Annotation: service.beta.kubernetes.io/alibaba-cloud-loadbalancer-name

Description	Default	Supported CCM version
Specify the name of an NLB instance. The name must be 2 to 128 characters in length, and can contain letters, digits, periods (.), underscores (_), and hyphens (-). The name must start with a letter.	None	CCM 2.5.0 and later

apiVersion: v1
kind: Service
metadata:
  annotations:
    service.beta.kubernetes.io/alibaba-cloud-loadbalancer-zone-maps: "${zone-A}:${vsw-A},${zone-B}:${vsw-B}" # Example: cn-hangzhou-k:vsw-i123456,cn-hangzhou-j:vsw-j654321. 
    service.beta.kubernetes.io/alibaba-cloud-loadbalancer-name: "${your-nlb-name}" # The name of the NLB instance. 
  name: nginx
  namespace: default
spec:
  externalTrafficPolicy: Local
  ports:
  - name: tcp
    port: 80
    protocol: TCP
    targetPort: 80
  - name: https
    port: 443
    protocol: TCP
    targetPort: 443
  selector:
    app: nginx
  loadBalancerClass: "alibabacloud.com/nlb"
  type: LoadBalancer

Specify the resource group to which an NLB instance belongs

Annotation: service.beta.kubernetes.io/alibaba-cloud-loadbalancer-resource-group-id

Description

Default

Supported CCM version

The ID of the resource group to which the NLB instance belongs. The resource group cannot be changed after it is specified.

You can log on to the Resource Management console to query resource group IDs.

None

CCM 2.5.0 and later

apiVersion: v1
kind: Service
metadata:
  annotations:
    service.beta.kubernetes.io/alibaba-cloud-loadbalancer-zone-maps: "${zone-A}:${vsw-A},${zone-B}:${vsw-B}" # Example: cn-hangzhou-k:vsw-i123456,cn-hangzhou-j:vsw-j654321. 
    service.beta.kubernetes.io/alibaba-cloud-loadbalancer-resource-group-id:  "${your-resource-group-id}" # The ID of a resource group. 
  name: nginx
  namespace: default
spec:
  externalTrafficPolicy: Local
  ports:
  - name: tcp
    port: 80
    protocol: TCP
    targetPort: 80
  - name: https
    port: 443
    protocol: TCP
    targetPort: 443
  selector:
    app: nginx
  loadBalancerClass: "alibabacloud.com/nlb"
  type: LoadBalancer

Create a dual-stack NLB instance

Annotation: service.beta.kubernetes.io/alibaba-cloud-loadbalancer-ip-version

Description

Default

Supported CCM version

Specify the IP version of an NLB instance. The IP version cannot be changed after it is specified. To use this annotation, make sure that the kube-proxy mode of the cluster is set to IPVS. Valid values:

ipv4: IPv4.
DualStack: dual stack.
- IPv6 must be enabled for the vSwitches specified in the service.beta.kubernetes.io/alibaba-cloud-loadbalancer-zone-maps parameter.
- The IPv6 address can be used only in an IPv6 network.

ipv4

CCM 2.5.0 and later

apiVersion: v1
kind: Service
metadata:
  annotations:
    service.beta.kubernetes.io/alibaba-cloud-loadbalancer-zone-maps: "${zone-A}:${vsw-A},${zone-B}:${vsw-B}" # Example: cn-hangzhou-k:vsw-i123456,cn-hangzhou-j:vsw-j654321. 
    service.beta.kubernetes.io/alibaba-cloud-loadbalancer-ip-version: "DualStack"
  name: nginx
  namespace: default
spec:
  externalTrafficPolicy: Local
  ports:
  - name: tcp
    port: 80
    protocol: TCP
    targetPort: 80
  - name: https
    port: 443
    protocol: TCP
    targetPort: 443
  selector:
    app: nginx
  sessionAffinity: None
  loadBalancerClass: "alibabacloud.com/nlb"
  type: LoadBalancer

Add additional tags to an NLB instance

Annotation: service.beta.kubernetes.io/alibaba-cloud-loadbalancer-additional-resource-tags

Description	Default	Supported CCM version
You can add additional tags. Separate multiple tags with commas (,). Example: `k1=v1,k2=v2`.	None	CCM 2.5.0 and later

apiVersion: v1
kind: Service
metadata:
  annotations:
    service.beta.kubernetes.io/alibaba-cloud-loadbalancer-zone-maps: "${zone-A}:${vsw-A},${zone-B}:${vsw-B}" # Example: cn-hangzhou-k:vsw-i123456,cn-hangzhou-j:vsw-j654321. 
    service.beta.kubernetes.io/alibaba-cloud-loadbalancer-additional-resource-tags: "Key1=Value1,Key2=Value2"
  name: nginx
  namespace: default
spec:
  externalTrafficPolicy: Local
  ports:
  - name: tcp
    port: 80
    protocol: TCP
    targetPort: 80
  - name: https
    port: 443
    protocol: TCP
    targetPort: 443
  selector:
    app: nginx
  sessionAffinity: None
  loadBalancerClass: "alibabacloud.com/nlb"
  type: LoadBalancer

Use an existing NLB instance

The following annotations are used.

Annotation

Description

Default

Supported CCM version

service.beta.kubernetes.io/alibaba-cloud-loadbalancer-id

The ID of an existing NLB instance.

None

service.beta.kubernetes.io/alibaba-cloud-loadbalancer-force-override-listeners

The annotation specifies whether to overwrite the existing listeners of the NLB instance. Valid values:

true: The CCM creates, updates, or deletes listeners for the NLB instance based on the configurations of the Service.
false: The CCM does not make any changes to the listeners of the NLB instance.

false

CCM 2.5.0 and later

apiVersion: v1
kind: Service
metadata:
  annotations:
    service.beta.kubernetes.io/alibaba-cloud-loadbalancer-id: "${your-nlb-id}" # The ID of an existing NLB instance. 
    service.beta.kubernetes.io/alibaba-cloud-loadbalancer-force-override-listeners: "true"
  name: nginx
  namespace: default
spec:
  externalTrafficPolicy: Local
  ports:
  - name: tcp
    port: 80
    protocol: TCP
    targetPort: 80
  - name: https
    port: 443
    protocol: TCP
    targetPort: 443
  selector:
    app: nginx
  sessionAffinity: None
  loadBalancerClass: "alibabacloud.com/nlb"
  type: LoadBalancer

Common operations to manage listeners

Configure security groups for listeners

Annotation: service.beta.kubernetes.io/alibaba-cloud-loadbalancer-security-group-ids

Description	Default	Supported CCM version
The security groups for listeners. Separate multiple IDs with commas (,). Example: `sg-aaaaa,sg-bbbbb`.	None	CCM 2.6.0 and later

apiVersion: v1
kind: Service
metadata:
  annotations:
    service.beta.kubernetes.io/alibaba-cloud-loadbalancer-zone-maps: "${zone-A}:${vsw-A},${zone-B}:${vsw-B}" # Example: cn-hangzhou-k:vsw-i123456,cn-hangzhou-j:vsw-j654321. 
    service.beta.kubernetes.io/alibaba-cloud-loadbalancer-security-group-ids: "sg-aaaaa,sg-bbbbb" # Separate multiple security group IDs with commas (,). 
  name: nginx
  namespace: default
spec:
  externalTrafficPolicy: Local
  ports:
  - name: tcp
    port: 80
    protocol: TCP
    targetPort: 80
  - name: https
    port: 443
    protocol: TCP
    targetPort: 443
  selector:
    app: nginx
  loadBalancerClass: "alibabacloud.com/nlb"
  type: LoadBalancer

Configure both TCP and UDP for a listener

Description	Default	Supported CCM version
Only clusters whose Kubernetes version is 1.24 and later support this feature. For more information about how to update the Kubernetes version of a cluster, see Update the Kubernetes version of an ACK cluster.	None	N/A

apiVersion: v1
kind: Service
metadata:
  annotations:
      service.beta.kubernetes.io/alibaba-cloud-loadbalancer-zone-maps: "${zone-A}:${vsw-A},${zone-B}:${vsw-B}" # Example: cn-hangzhou-k:vsw-i123456,cn-hangzhou-j:vsw-j654321. 
  name: nginx
  namespace: default
spec:
  externalTrafficPolicy: Local
  ports:
  - name: tcp
    port: 80
    protocol: TCP
    targetPort: 80
  - name: udp
    port: 80
    protocol: UDP
    targetPort: 81
  selector:
    app: nginx
  sessionAffinity: None
  loadBalancerClass: "alibabacloud.com/nlb"
  type: LoadBalancer

Create a TCP listener

apiVersion: v1
kind: Service
metadata:
  annotations:
    service.beta.kubernetes.io/alibaba-cloud-loadbalancer-zone-maps: "${zone-A}:${vsw-A},${zone-B}:${vsw-B}" # Example: cn-hangzhou-k:vsw-i123456,cn-hangzhou-j:vsw-j654321. 
  name: nginx
  namespace: default
spec:
  externalTrafficPolicy: Local
  ports:
  - name: tcp
    port: 80
    protocol: TCP
    targetPort: 80
  - name: https
    port: 443
    protocol: TCP
    targetPort: 443
  selector:
    app: nginx
  sessionAffinity: None
  loadBalancerClass: "alibabacloud.com/nlb"
  type: LoadBalancer

Create a UDP listener

apiVersion: v1
kind: Service
metadata:
  annotations:
    service.beta.kubernetes.io/alibaba-cloud-loadbalancer-zone-maps: "${zone-A}:${vsw-A},${zone-B}:${vsw-B}" # Example: cn-hangzhou-k:vsw-i123456,cn-hangzhou-j:vsw-j654321. 
  name: nginx
  namespace: default
spec:
  externalTrafficPolicy: Local
  ports:
  - name: udp
    port: 80
    protocol: UDP
    targetPort: 80
  selector:
    app: nginx
  sessionAffinity: None
  loadBalancerClass: "alibabacloud.com/nlb"
  type: LoadBalancer

Create a TCP/SSL listener

The following annotations are used.

Annotation	Description	Default	Supported CCM version
service.beta.kubernetes.io/alibaba-cloud-loadbalancer-protocol-port	The type of listener. Separate multiple listener types with commas (,). Example: `TCP:80,TCPSSL:443`.	None	CCM 2.5.0 and later
service.beta.kubernetes.io/alibaba-cloud-loadbalancer-cert-id	The server certificate ID. You can log on to the Certificate Management Service console to view SSL certificate IDs.	None	CCM 2.5.0 and later

apiVersion: v1
kind: Service
metadata:
  annotations:
    service.beta.kubernetes.io/alibaba-cloud-loadbalancer-zone-maps: "${zone-A}:${vsw-A},${zone-B}:${vsw-B}" # Example: cn-hangzhou-k:vsw-i123456,cn-hangzhou-j:vsw-j654321. 
    service.beta.kubernetes.io/alibaba-cloud-loadbalancer-protocol-port: "tcpssl:443"
    # If the cluster is deployed in a region in the Chinese mainland, the SSL certificate ID appended with region information is ${your-cert-id}-cn-hangzhou. 
    # If the cluster is deployed in a region outside the Chinese mainland, the SSL certificate ID appended with region information is ${your-cert-id}-ap-southeast-1. 
    service.beta.kubernetes.io/alibaba-cloud-loadbalancer-cert-id: "${The appended certificate ID}"
  name: nginx
  namespace: default
spec:
  externalTrafficPolicy: Local
  ports:
  - name: tcp
    port: 80
    protocol: TCP
    targetPort: 80
  - name: https
    port: 443
    protocol: TCP
    targetPort: 443
  selector:
    app: nginx
  loadBalancerClass: "alibabacloud.com/nlb"
  type: LoadBalancer

Enable mutual authentication

The following annotations are used.

Annotation	Description	Default	Supported CCM version
service.beta.kubernetes.io/alibaba-cloud-loadbalancer-protocol-port	The type of listener. Separate multiple listener types with commas (,). Example: `TCP:80,TCPSSL:443`.	None	CCM 2.5.0 and later
service.beta.kubernetes.io/alibaba-cloud-loadbalancer-cert-id	The server certificate ID. You can log on to the Certificate Management Service console to view SSL certificate IDs.	None
service.beta.kubernetes.io/alibaba-cloud-loadbalancer-cacert-id	The certificate authority (CA) certificate ID. You can log on to the Certificate Management Service console to view CA certificate IDs.	None
service.beta.kubernetes.io/alibaba-cloud-loadbalancer-cacert	The annotation specifies whether to enable mutual authentication. Valid values: `on`: enables mutual authentication. `off`: disables mutual authentication.	None

apiVersion: v1
kind: Service
metadata:
  annotations:
    service.beta.kubernetes.io/alibaba-cloud-loadbalancer-zone-maps: "${zone-A}:${vsw-A},${zone-B}:${vsw-B}" # Example: cn-hangzhou-k:vsw-i123456,cn-hangzhou-j:vsw-j654321. 
    service.beta.kubernetes.io/alibaba-cloud-loadbalancer-protocol-port: "tcpssl:443"   
    # If the cluster is deployed in a region in the Chinese mainland, the SSL certificate ID appended with region information is ${your-cert-id}-cn-hangzhou. 
    # If the cluster is deployed in a region outside the Chinese mainland, the SSL certificate ID appended with region information is ${your-cert-id}-ap-southeast-1. 
    service.beta.kubernetes.io/alibaba-cloud-loadbalancer-cert-id: "${The appended certificate ID}" 
    service.beta.kubernetes.io/alibaba-cloud-loadbalancer-cacert-id: "${your-cacert-id}"  # The CA certificate ID. 
    service.beta.kubernetes.io/alibaba-cloud-loadbalancer-cacert: "on"
name: nginx
  namespace: default
spec:
  externalTrafficPolicy: Local
  ports:
  - name: tcp
    port: 80
    protocol: TCP
    targetPort: 80
  - name: https
    port: 443
    protocol: TCP
    targetPort: 443
  selector:
    app: nginx
  loadBalancerClass: "alibabacloud.com/nlb"
  type: LoadBalancer

Specify a TLS security policy

The following annotations are used.

Annotation	Description	Default	Supported CCM version
service.beta.kubernetes.io/alibaba-cloud-loadbalancer-protocol-port	The type of listener. Separate multiple listener types with commas (,). Example: `TCP:80,TCPSSL:443`.	None	CCM 2.5.0 and later
service.beta.kubernetes.io/alibaba-cloud-loadbalancer-cert-id	The server certificate ID. You can log on to the Certificate Management Service console to view SSL certificate IDs.	None
service.beta.kubernetes.io/alibaba-cloud-loadbalancer-tls-cipher-policy	The ID of the security policy. System and custom security policies are supported. Valid values: `tls_cipher_policy_1_0` `tls_cipher_policy_1_1` `tls_cipher_policy_1_2` `tls_cipher_policy_1_2_strict` `tls_cipher_policy_1_2_strict_with_1_3`	tls_cipher_policy_1_0

apiVersion: v1
kind: Service
metadata:
  annotations:
    service.beta.kubernetes.io/alibaba-cloud-loadbalancer-zone-maps: "${zone-A}:${vsw-A},${zone-B}:${vsw-B}" # Example: cn-hangzhou-k:vsw-i123456,cn-hangzhou-j:vsw-j654321. 
    service.beta.kubernetes.io/alibaba-cloud-loadbalancer-protocol-port: "tcpssl:443"
    # If the cluster is deployed in a region in the Chinese mainland, the SSL certificate ID appended with region information is ${your-cert-id}-cn-hangzhou. 
    # If the cluster is deployed in a region outside the Chinese mainland, the SSL certificate ID appended with region information is ${your-cert-id}-ap-southeast-1. 
    service.beta.kubernetes.io/alibaba-cloud-loadbalancer-cert-id: "${The appended certificate ID}" 
    service.beta.kubernetes.io/alibaba-cloud-loadbalancer-tls-cipher-policy: "tls_cipher_policy_1_0"
  name: nginx
  namespace: default
spec:
  externalTrafficPolicy: Local
  ports:
  - name: tcp
    port: 80
    protocol: TCP
    targetPort: 80
  - name: https
    port: 443
    protocol: TCP
    targetPort: 443
  selector:
    app: nginx
  loadBalancerClass: "alibabacloud.com/nlb"
  type: LoadBalancer

Configure Proxy Protocol

Annotation: service.beta.kubernetes.io/alibaba-cloud-loadbalancer-proxy-protocol

Description

Default

Supported CCM version

The annotation specifies whether to enable Proxy Protocol to pass client IP addresses to backend servers. Valid values:

on:: enables Proxy Protocol.
off: disables Proxy Protocol.

Important

Before you enable Proxy Protocol, check whether the backend application has Proxy Protocol V2 enabled. If Proxy Protocol V2 is disabled for the backend application, requests cannot be forwarded to the backend application.

off

CCM 2.5.0 and later

apiVersion: v1
kind: Service
metadata:
  annotations:
    service.beta.kubernetes.io/alibaba-cloud-loadbalancer-zone-maps: "${zone-A}:${vsw-A},${zone-B}:${vsw-B}" # Example: cn-hangzhou-k:vsw-i123456,cn-hangzhou-j:vsw-j654321. 
    service.beta.kubernetes.io/alibaba-cloud-loadbalancer-proxy-protocol: "on"
  name: nginx
  namespace: default
spec:
  externalTrafficPolicy: Local
  ports:
  - name: tcp
    port: 80
    protocol: TCP
    targetPort: 80
  - name: https
    port: 443
    protocol: TCP
    targetPort: 443
  selector:
    app: nginx
  loadBalancerClass: "alibabacloud.com/nlb"
  type: LoadBalancer

Specify the maximum number of connections that can be created per second

Annotation: service.beta.kubernetes.io/alibaba-cloud-loadbalancer-cps

Description	Default	Supported CCM version
The maximum number of connections that can be created per second on the NLB instance. Valid values: 0 to 1000000. A value of 0 indicates that the number of connections is unlimited.	None	CCM 2.5.0 and later

apiVersion: v1
kind: Service
metadata:
  annotations:
    service.beta.kubernetes.io/alibaba-cloud-loadbalancer-zone-maps: "${zone-A}:${vsw-A},${zone-B}:${vsw-B}" # Example: cn-hangzhou-k:vsw-i123456,cn-hangzhou-j:vsw-j654321. 
    service.beta.kubernetes.io/alibaba-cloud-loadbalancer-cps: "100"
  name: nginx
  namespace: default
spec:
  externalTrafficPolicy: Local
  ports:
  - name: tcp
    port: 80
    protocol: TCP
    targetPort: 80
  - name: https
    port: 443
    protocol: TCP
    targetPort: 443
  selector:
    app: nginx
  loadBalancerClass: "alibabacloud.com/nlb"
  type: LoadBalancer

Specify the timeout period of idle connections for a listener

Annotation: service.beta.kubernetes.io/alibaba-cloud-loadbalancer-idle-timeout

Description	Default	Supported CCM version
The timeout period of idle connections. Unit: seconds. Valid values: 10 to 900.	900	CCM 2.5.0 and later

apiVersion: v1
kind: Service
metadata:
  annotations:
    service.beta.kubernetes.io/alibaba-cloud-loadbalancer-zone-maps: "${zone-A}:${vsw-A},${zone-B}:${vsw-B}" # Example: cn-hangzhou-k:vsw-i123456,cn-hangzhou-j:vsw-j654321. 
    service.beta.kubernetes.io/alibaba-cloud-loadbalancer-idle-timeout: "60"
  name: nginx
  namespace: default
spec:
  externalTrafficPolicy: Local
  ports:
  - name: tcp
    port: 80
    protocol: TCP
    targetPort: 80
  - name: https
    port: 443
    protocol: TCP
    targetPort: 443
  selector:
    app: nginx
  loadBalancerClass: "alibabacloud.com/nlb"
  type: LoadBalancer

Common operations to manage backend server groups

Set the scheduling algorithm

Annotation: service.beta.kubernetes.io/alibaba-cloud-loadbalancer-scheduler

Description

Default

Supported CCM version

The scheduling algorithm. Valid values:

wrr (default): The weighted round robin (WRR) algorithm is used. Backend servers with higher weights receive more requests than backend servers with lower weights.
rr: The round-robin algorithm is used. Requests are forwarded to backend servers in sequence.
sch: The source IP hashing algorithm. Requests from the same source IP address are forwarded to the same backend server.
tch: The four-element hashing algorithm is used. Consistent hashing is based on the following factors: source IP address, destination IP address, source port, and destination port. Requests that contain the same information based on the four factors are forwarded to the same backend server.

wrr

CCM 2.5.0 and later

apiVersion: v1
kind: Service
metadata:
  annotations:
    service.beta.kubernetes.io/alibaba-cloud-loadbalancer-zone-maps: "${zone-A}:${vsw-A},${zone-B}:${vsw-B}" # Example: cn-hangzhou-k:vsw-i123456,cn-hangzhou-j:vsw-j654321. 
    service.beta.kubernetes.io/alibaba-cloud-loadbalancer-scheduler: "sch"
  name: nginx
  namespace: default
spec:
  externalTrafficPolicy: Local
  ports:
  - name: tcp
    port: 80
    protocol: TCP
    targetPort: 80
  - name: https
    port: 443
    protocol: TCP
    targetPort: 443
  selector:
    app: nginx
  loadBalancerClass: "alibabacloud.com/nlb"
  type: LoadBalancer

Configure connection draining

The following annotations are used.

Annotation

Description

Default

Supported CCM version

service.beta.kubernetes.io/alibaba-cloud-loadbalancer-connection-drain

The annotation specifies whether to enable connection draining. Valid values:

on:: enables connection draining.
off: disables connection draining.

off

CCM 2.5.0 and later

service.beta.kubernetes.io/alibaba-cloud-loadbalancer-connection-drain-timeout

The timeout period of connection draining. Unit: seconds. Valid values: 10 to 900.

None

apiVersion: v1
kind: Service
metadata:
  annotations:
    service.beta.kubernetes.io/alibaba-cloud-loadbalancer-zone-maps: "${zone-A}:${vsw-A},${zone-B}:${vsw-B}" # Example: cn-hangzhou-k:vsw-i123456,cn-hangzhou-j:vsw-j654321. 
    service.beta.kubernetes.io/alibaba-cloud-loadbalancer-connection-drain: "on"
    service.beta.kubernetes.io/alibaba-cloud-loadbalancer-connection-drain-timeout: "30"
  name: nginx
  namespace: default
spec:
  externalTrafficPolicy: Local
  ports:
  - name: tcp
    port: 80
    protocol: TCP
    targetPort: 80
  - name: https
    port: 443
    protocol: TCP
    targetPort: 443
  selector:
    app: nginx
  loadBalancerClass: "alibabacloud.com/nlb"
  type: LoadBalancer

Configure client IP preservation

Annotation: service.beta.kubernetes.io/alibaba-cloud-loadbalancer-preserve-client-ip

Description

Default

Supported CCM version

The annotation specifies whether to enable client IP preservation. Valid values:

on:: enables client IP preservation.
off: disables client IP preservation.

off

CCM 2.5.0 and later

apiVersion: v1
kind: Service
metadata:
  annotations:
    service.beta.kubernetes.io/alibaba-cloud-loadbalancer-zone-maps: "${zone-A}:${vsw-A},${zone-B}:${vsw-B}" # Example: cn-hangzhou-k:vsw-i123456,cn-hangzhou-j:vsw-j654321. 
    service.beta.kubernetes.io/alibaba-cloud-loadbalancer-preserve-client-ip: "on"
  name: nginx
  namespace: default
spec:
  externalTrafficPolicy: Local
  ports:
  - name: tcp
    port: 80
    protocol: TCP
    targetPort: 80
  - name: https
    port: 443
    protocol: TCP
    targetPort: 443
  selector:
    app: nginx
  loadBalancerClass: "alibabacloud.com/nlb"
  type: LoadBalancer

Configure TCP health checks

The following annotations are used. To configure TCP health checks, all annotations in the following table are required. By default, health checks are enabled for TCP ports.

Annotation	Description	Default	Supported CCM version
service.beta.kubernetes.io/alibaba-cloud-loadbalancer-health-check-flag	The annotation specifies whether to enable health checks. Valid values: `on:`: enables health checks. `off`: disables health checks.	on
service.beta.kubernetes.io/alibaba-cloud-loadbalancer-health-check-type	The protocol that you want to use for health checks. Valid values: `tcp` `http`	tcp
service.beta.kubernetes.io/alibaba-cloud-loadbalancer-health-check-connect-port	The backend server port that is used for health checks. Valid values: 0 to 65535. Default value: 0. This value indicates that the health check port specified on a backend server is used.	0
service.beta.kubernetes.io/alibaba-cloud-loadbalancer-health-check-connect-timeout	The timeout period of health checks. Unit: seconds. Valid values: 1 to 300.	5
service.beta.kubernetes.io/alibaba-cloud-loadbalancer-healthy-threshold	The number of consecutive successful health checks that must occur before an unhealthy backend server is declared healthy. Valid values: 2 to 10.	2
service.beta.kubernetes.io/alibaba-cloud-loadbalancer-unhealthy-threshold	The number of consecutive failed health checks that must occur before a healthy backend server is declared unhealthy. Valid values: 2 to 10.	2
service.beta.kubernetes.io/alibaba-cloud-loadbalancer-health-check-interval	The interval at which health checks are performed. Unit: seconds. Valid values: 5 to 50.	10

apiVersion: v1
kind: Service
metadata:
  annotations:
    service.beta.kubernetes.io/alibaba-cloud-loadbalancer-zone-maps: "${zone-A}:${vsw-A},${zone-B}:${vsw-B}" # Example: cn-hangzhou-k:vsw-i123456,cn-hangzhou-j:vsw-j654321. 
    service.beta.kubernetes.io/alibaba-cloud-loadbalancer-health-check-flag: "on"
    service.beta.kubernetes.io/alibaba-cloud-loadbalancer-health-check-type: "tcp"
    service.beta.kubernetes.io/alibaba-cloud-loadbalancer-health-check-connect-timeout: "8"
    service.beta.kubernetes.io/alibaba-cloud-loadbalancer-healthy-threshold: "4"
    service.beta.kubernetes.io/alibaba-cloud-loadbalancer-unhealthy-threshold: "4"
    service.beta.kubernetes.io/alibaba-cloud-loadbalancer-health-check-interval: "5"
  name: nginx
  namespace: default
spec:
  externalTrafficPolicy: Local
  ports:
  - name: tcp
    port: 80
    protocol: TCP
    targetPort: 80
  - name: https
    port: 443
    protocol: TCP
    targetPort: 443
  selector:
    app: nginx
  loadBalancerClass: "alibabacloud.com/nlb"
  type: LoadBalancer

Configure HTTP health checks

The following annotations are used. To configure TCP health checks, all annotations in the following table are required. By default, health checks are enabled for TCP ports.

Annotation	Description	Default	Supported CCM version
service.beta.kubernetes.io/alibaba-cloud-loadbalancer-health-check-flag	The annotation specifies whether to enable health checks. Valid values: `on:`: enables health checks. `off`: disables health checks.	on	CCM 2.5.0 and later
service.beta.kubernetes.io/alibaba-cloud-loadbalancer-health-check-type	The protocol that is used for health checks. Valid values: `tcp` `http`	tcp
service.beta.kubernetes.io/alibaba-cloud-loadbalancer-health-check-uri	The path to which health check requests are sent. The value must be 1 to 80 characters in length, and can contain only letters, digits, and special characters. It must start with a forward slash (/). For more information, see CreateServerGroup. Note This annotation takes effect only when `service.beta.kubernetes.io/alibaba-cloud-loadbalancer-health-check-type` is set to `HTTP`.	None
service.beta.kubernetes.io/alibaba-cloud-loadbalancer-health-check-domain	The domain name that is used for health checks. Valid values: $_ip: a private server IP domain: a domain name The domain name must be 1 to 80 characters in length, and can contain letters, digits, hyphens (-), and periods (.). Note This annotation takes effect only when `service.beta.kubernetes.io/alibaba-cloud-loadbalancer-health-check-type` is set to `HTTP`.	None
service.beta.kubernetes.io/alibaba-cloud-loadbalancer-health-check-connect-port	The backend server port that is used for health checks. Valid values: 0 to 65535. Default value: 0. This value indicates that the health check port specified on a backend server is used.	0
service.beta.kubernetes.io/alibaba-cloud-loadbalancer-health-check-connect-timeout	The timeout period of health checks. Unit: seconds. Valid values: 1 to 300.	5
service.beta.kubernetes.io/alibaba-cloud-loadbalancer-healthy-threshold	The number of consecutive successful health checks that must occur before an unhealthy backend server is declared healthy. Valid values: 2 to 10.	2
service.beta.kubernetes.io/alibaba-cloud-loadbalancer-unhealthy-threshold	The number of consecutive failed health checks that must occur before a healthy backend server is declared unhealthy. Valid values: 2 to 10.	2
service.beta.kubernetes.io/alibaba-cloud-loadbalancer-health-check-interval	The interval at which health checks are performed. Unit: seconds. Valid values: 5 to 50.	10
service.beta.kubernetes.io/alibaba-cloud-loadbalancer-health-check-method	The HTTP method that is used for health checks. Valid values: `GET` `HEAD` Note This annotation takes effect only when `service.beta.kubernetes.io/alibaba-cloud-loadbalancer-health-check-type` is set to `HTTP`.

apiVersion: v1
kind: Service
metadata:
  annotations:
    service.beta.kubernetes.io/alibaba-cloud-loadbalancer-zone-maps: "${zone-A}:${vsw-A},${zone-B}:${vsw-B}" # Example: cn-hangzhou-k:vsw-i123456,cn-hangzhou-j:vsw-j654321. 
    service.beta.kubernetes.io/alibaba-cloud-loadbalancer-health-check-flag: "on"
    service.beta.kubernetes.io/alibaba-cloud-loadbalancer-health-check-type: "http"
    service.beta.kubernetes.io/alibaba-cloud-loadbalancer-health-check-uri: "/test/index.html"
    service.beta.kubernetes.io/alibaba-cloud-loadbalancer-health-check-domain: "www.test.com"
    service.beta.kubernetes.io/alibaba-cloud-loadbalancer-healthy-threshold: "4"
    service.beta.kubernetes.io/alibaba-cloud-loadbalancer-unhealthy-threshold: "4"
    service.beta.kubernetes.io/alibaba-cloud-loadbalancer-health-check-connect-timeout: "10"
    service.beta.kubernetes.io/alibaba-cloud-loadbalancer-health-check-interval: "5"
    # Specify the health check method. The annotation is optional. 
    service.beta.kubernetes.io/alibaba-cloud-loadbalancer-health-check-method: "head"
  name: nginx
  namespace: default
spec:
  externalTrafficPolicy: Local
  ports:
  - name: tcp
    port: 80
    protocol: TCP
    targetPort: 80
  - name: https
    port: 443
    protocol: TCP
    targetPort: 443
  selector:
    app: nginx
  loadBalancerClass: "alibabacloud.com/nlb"
  type: LoadBalancer

Specify the backend server group type

Annotation: service.beta.kubernetes.io/alibaba-cloud-loadbalancer-server-group-type

Description

Default

Supported CCM version

Specify the backend server group type. Valid values:

Ip: allows you to add backend servers by specifying IP addresses.
Instance (default): allows you to add Elastic Compute Service (ECS) instances and elastic network interfaces (ENIs) as backend servers.

For more information about NLB server groups, see Overview of NLB server groups.

Instance

CCM 2.8.0 and later

apiVersion: v1
kind: Service
metadata:
  annotations:
    service.beta.kubernetes.io/alibaba-cloud-loadbalancer-zone-maps: "${zone-A}:${vsw-A},${zone-B}:${vsw-B}" # Example: cn-hangzhou-k:vsw-i123456,cn-hangzhou-j:vsw-j654321. 
    service.beta.kubernetes.io/alibaba-cloud-loadbalancer-server-group-type: "Ip"
  name: nginx
  namespace: default
spec:
  externalTrafficPolicy: Local
  ports:
  - name: tcp
    port: 80
    protocol: TCP
    targetPort: 80
  - name: https
    port: 443
    protocol: TCP
    targetPort: 443
  selector:
    app: nginx
  loadBalancerClass: "alibabacloud.com/nlb"
  type: LoadBalancer

Container Service for Kubernetes:Configure NLB instances by using annotations

Table of contents

Precautions

Common operations to manage NLB instances

Create an Internet-facing NLB instance

Create an internal-facing NLB instance

Specify the name of an NLB instance

Specify the resource group to which an NLB instance belongs

Create a dual-stack NLB instance

Add additional tags to an NLB instance

Use an existing NLB instance

Common operations to manage listeners

Configure security groups for listeners

Configure both TCP and UDP for a listener

Create a TCP listener

Create a UDP listener

Create a TCP/SSL listener

Enable mutual authentication

Specify a TLS security policy

Configure Proxy Protocol

Specify the maximum number of connections that can be created per second

Specify the timeout period of idle connections for a listener

Common operations to manage backend server groups

Set the scheduling algorithm

Configure connection draining

Configure client IP preservation

Configure TCP health checks

Configure HTTP health checks

Specify the backend server group type