You can add annotations to the YAML file of a Service to configure load balancing. This topic describes how to use annotations to configure load balancing. You can configure Server Load Balancer (SLB) instance, listeners, and backend server groups.

Usage notes

  • The values of annotations are case-sensitive.
  • Starting from September 11, 2019, the keyword alicloud in annotations is changed to alibaba-cloud.

    Example:

    Before the change: service.beta.kubernetes.io/alicloud-loadbalancer-id

    After the change: service.beta.kubernetes.io/alibaba-cloud-loadbalancer-id

    The system still supports annotations that use the alicloud keyword. You do not need to modify the existing annotations.

SLB

Common annotations used to configure load balancing

  • Create an Internet-facing SLB instance
    apiVersion: v1
    kind: Service
    metadata:
      name: nginx
      namespace: default
    spec:
      ports:
      - port: 80
        protocol: TCP
        targetPort: 80
      selector:
        run: nginx
      type: LoadBalancer
  • Create an internal-facing SLB instance
    apiVersion: v1
    kind: Service
    metadata:
      annotations:
        service.beta.kubernetes.io/alibaba-cloud-loadbalancer-address-type: "intranet"
      name: nginx
      namespace: default
    spec:
      ports:
      - port: 80
        protocol: TCP
        targetPort: 80
      selector:
        run: nginx
      type: LoadBalancer
  • Create an SLB instance of a specified specification
    For more information about the specifications of SLB instances, see CreateLoadBalancer. Use the following template to create a LoadBalancer Service and an SLB instance of a specified specification. You can also use the template to update the specification of an existing SLB instance.
    Notice If you use the SLB console to update the specification of an existing SLB instance, the Cloud Controller Manager (CCM) may restore the SLB instance to the original specification. Proceed with caution. You can use the SLB console to modify the specifications of only SLB instances that use the pay-by-specification metering method.
    apiVersion: v1
    kind: Service
    metadata:
      annotations:
        service.beta.kubernetes.io/alibaba-cloud-loadbalancer-instance-charge-type: "PayBySpec"
        service.beta.kubernetes.io/alibaba-cloud-loadbalancer-spec: "slb.s1.small"
      name: nginx
      namespace: default
    spec:
      ports:
      - port: 443
        protocol: TCP
        targetPort: 443
      selector:
        run: nginx
      type: LoadBalancer
  • Use an existing SLB instance
    • By default, the CCM does not overwrite the listeners of an existing SLB instance. To overwrite the listeners of an existing SLB instance, set the annotation service.beta.kubernetes.io/alibaba-cloud-loadbalancer-force-override-listeners to true.
      Note The following list explains why the CCM does not overwrite the listeners of an existing SLB instance:
      • If the listeners of the SLB instance are associated with applications, service interruptions may occur after the configurations of the listeners are overwritten.
      • The CCM supports limited backend configurations and cannot handle complex configurations. If you require complex backend configurations, you can manually configure listeners in the SLB console without overwriting the existing listeners.
      In both cases, we recommend that you do not overwrite the listeners of existing SLB instances. However, you can overwrite an existing listener if the port of the listener is no longer in use.
    • You cannot add additional tags to an existing SLB instance by using the annotation service.beta.kubernetes.io/alibaba-cloud-loadbalancer-additional-resource-tags.
    apiVersion: v1
    kind: Service
    metadata:
      annotations:
        service.beta.kubernetes.io/alibaba-cloud-loadbalancer-id: "${YOUR_LOADBALANCER_ID}"
      name: nginx
      namespace: default
    spec:
      ports:
      - port: 443
        protocol: TCP
        targetPort: 443
      selector:
        run: nginx
      type: LoadBalancer
  • Use an existing SLB instance and forcefully overwrite the listeners of the SLB instance
    Use the following template to create a LoadBalancer Service and forcefully overwrite an existing listener of the SLB instance. If you specify a different listener port number, the original listener is deleted.
    Notice If you use an existing SLB instance and set forceoverride to true, do not associate a listener of the SLB instance with multiple Services. Otherwise, a configuration conflict occurs when these Services overwrite the configuration of the listener.
    apiVersion: v1
    kind: Service
    metadata:
      annotations:
        service.beta.kubernetes.io/alibaba-cloud-loadbalancer-id: "${YOUR_LOADBALANCER_ID}"
        service.beta.kubernetes.io/alibaba-cloud-loadbalancer-force-override-listeners: "true"
      name: nginx
      namespace: default
    spec:
      ports:
      - port: 443
        protocol: TCP
        targetPort: 443
      selector:
        run: nginx
      type: LoadBalancer
  • Specify the primary zone and secondary zone when you create an SLB instance
    • SLB instances cannot be deployed across zones in specific regions, such as the Indonesia (Jakarta) region.
    • After you specify the primary zone and secondary zone for an SLB instance, the zones cannot be changed.
    apiVersion: v1
    kind: Service
    metadata:
      annotations:
        service.beta.kubernetes.io/alibaba-cloud-loadbalancer-master-zoneid: "ap-southeast-5a"
        service.beta.kubernetes.io/alibaba-cloud-loadbalancer-slave-zoneid: "ap-southeast-5a"
      name: nginx
      namespace: default
    spec:
      ports:
      - port: 80
        protocol: TCP
        targetPort: 80
      selector:
        run: nginx
      type: LoadBalancer
  • Create a pay-by-bandwidth SLB instance
    • Only Internet-facing SLB instances support the pay-by-bandwidth metering method.

      For more information about limits on metering methods for Internet-facing SLB instances, see Change the metering method of an Internet-facing SLB instance.

    • All annotations in the following template are required.

      Modify the annotation values based on your business requirements.

    apiVersion: v1
    kind: Service
    metadata:
      annotations:
        service.beta.kubernetes.io/alibaba-cloud-loadbalancer-charge-type: "paybybandwidth"
        service.beta.kubernetes.io/alibaba-cloud-loadbalancer-bandwidth: "2"
      name: nginx
      namespace: default
    spec:
      ports:
      - port: 443
        protocol: TCP
        targetPort: 443
      selector:
        run: nginx
      type: LoadBalancer
    Note The annotation service.beta.kubernetes.io/alibaba-cloud-loadbalancer-bandwidth specifies the bandwidth limit.
  • Specify a vSwitch for an SLB instance
    • Obtain the ID of a vSwitch in the Virtual Private Cloud (VPC) console. Then, use the annotations in the following template to specify the vSwitch for an SLB instance.
    • The specified vSwitch and the cluster must be deployed in the same VPC.
    • All annotations in the following template are required.
    apiVersion: v1
    kind: Service
    metadata:
      annotations:
       service.beta.kubernetes.io/alibaba-cloud-loadbalancer-address-type: "intranet"
       service.beta.kubernetes.io/alibaba-cloud-loadbalancer-vswitch-id: "${YOUR_VSWITCH_ID}"
      name: nginx
      namespace: default
    spec:
      ports:
      - port: 443
        protocol: TCP
        targetPort: 443
      selector:
        run: nginx
      type: LoadBalancer
  • Add additional tags to an SLB instance

    Separate multiple tags with commas (,). Example: "k1=v1,k2=v2".

    apiVersion: v1
    kind: Service
    metadata:
      annotations:
        service.beta.kubernetes.io/alibaba-cloud-loadbalancer-additional-resource-tags: "Key1=Value1,Key2=Value2" 
      name: nginx
      namespace: default
    spec:
      ports:
      - port: 80
        protocol: TCP
        targetPort: 80
      selector:
        run: nginx
      type: LoadBalancer
  • Create an IPv6 SLB instance
    • The kube-proxy mode must be set to IPVS.
    • The assigned IPv6 address can be used only in an IPv6 network.
    • You cannot change the IP address type after you create an IPv6 SLB instance.
    apiVersion: v1
    kind: Service
    metadata:
      annotations:
        service.beta.kubernetes.io/alibaba-cloud-loadbalancer-ip-version: "ipv6"
      name: nginx
    spec:
      ports:
      - port: 80
        protocol: TCP
        targetPort: 80
      selector:
        app: nginx
      type: LoadBalancer
  • Enable deletion protection for an SLB instance

    By default, deletion protection is enabled for SLB instances.

    Notice If you manually enable deletion protection in the SLB console for an SLB instance that is created for a LoadBalancer Service, you can run the kubectl delete svc {your-svc-name} command to delete the SLB instance.
    apiVersion: v1
    kind: Service
    metadata:
      annotations:
        service.beta.kubernetes.io/alibaba-cloud-loadbalancer-delete-protection: "on"
      name: nginx
    spec:
      externalTrafficPolicy: Local
      ports:
      - port: 80
        protocol: TCP
        targetPort: 80
      selector:
        app: nginx
      type: LoadBalancer
  • Enable the configuration read-only mode for an SLB instance

    By default, the configuration read-only mode is enabled for an SLB instance.

    apiVersion: v1
    kind: Service
    metadata:
      annotations:
        service.beta.kubernetes.io/alibaba-cloud-loadbalancer-modification-protection: "ConsoleProtection"
      name: nginx
    spec:
      externalTrafficPolicy: Local
      ports:
      - port: 80
        protocol: TCP
        targetPort: 80
      selector:
        app: nginx
      type: LoadBalancer
  • Specify the name of an SLB instance
    apiVersion: v1
    kind: Service
    metadata:
      annotations:
        service.beta.kubernetes.io/alibaba-cloud-loadbalancer-name: "your-svc-name"
      name: nginx
    spec:
      externalTrafficPolicy: Local
      ports:
      - port: 80
        protocol: TCP
        targetPort: 80
      selector:
        app: nginx
      type: LoadBalancer
  • Specify a resource group to which an SLB instance belongs
    Log on to the Resource Management console to obtain the ID of a resource group. Then, use the annotation in the following template to specify the resource group to which the SLB instance belongs.
    Note You cannot modify the resource group after the SLB instance is created.
    apiVersion: v1
    kind: Service
    metadata:
      annotations:
        service.beta.kubernetes.io/alibaba-cloud-loadbalancer-resource-group-id: "rg-xxxx"
      name: nginx
    spec:
      externalTrafficPolicy: Local
      ports:
      - port: 80
        protocol: TCP
        targetPort: 80
      selector:
        app: nginx
      type: LoadBalancer
  • Configure a hostname for the Service
    • your_service_name must comply with the naming conventions of domain names.
    • After you add the annotation, the external IP address of the Service is changed from the default SLB IP address to your_service_name. If you access the SLB IP address from within the cluster, the request is first forwarded to the corresponding SLB instance.
    • After you add the annotation, if the listener protocol is TCP or UDP and you access the SLB IP address from within the cluster, a loopback issue occurs. For more information, see Why am I unable to access an SLB instance?
    • This annotation does not automatically associate a domain name with the SLB instance. If you want to associate a domain name with the SLB instance, log on to the Domains console, purchase a domain name, and then associate the domain name with the SLB instance. For more information, see Purchase a domain name.
    apiVersion: v1
    kind: Service
    metadata:
      annotations:
        service.beta.kubernetes.io/alibaba-cloud-loadbalancer-protocol-port: "http:80"
        service.beta.kubernetes.io/alibaba-cloud-loadbalancer-hostname: "${your_service_hostname}"
      name: nginx-svc
      namespace: default
    spec:
      ports:
      - name: http
        port: 80
        protocol: TCP
        targetPort: 80
      selector:
        app: nginx
      type: LoadBalancer

    Expected output:

    NAME         TYPE           CLUSTER-IP       EXTERNAL-IP            PORT(S)                      AGE
    nginx-svc    loadBalancer   47.100.XX.XX     www.example.com        80:30248/TCP,443:32670/TCP   10s
  • Create a pay-by-LCU SLB instance
    Supported metering methods:
    • PayBySpec: The SLB instance is billed based on its specification. This is the default metering method.
    • PayByCLCU: The SLB instance is billed based on Loadbalancer Capacity Unit (LCUs).

    You cannot change the specification of a pay-by-specification SLB instance. This means that the PayByCLCU and service.beta.kubernetes.io/alibaba-cloud-loadbalancer-spec are mutually exclusive. After you create a pay-by-specification SLB instance, you can only change the metering method of the SLB instance to PayByCLCU.

    apiVersion: v1
    kind: Service
    metadata:
      annotations:
        service.beta.kubernetes.io/alibaba-cloud-loadbalancer-instance-charge-type: "PayByCLCU"
      name: nginx
      namespace: default
    spec:
      ports:
      - port: 80
        protocol: TCP
        targetPort: 80
      selector:
        run: nginx
      type: LoadBalancer

Listeners

Common annotations that are used to configure listeners

  • Set the session persistence period for a TCP-based SLB instance
    • The annotation service.beta.kubernetes.io/alibaba-cloud-loadbalancer-persistence-timeout applies only to TCP listeners.
    • If an SLB instance has multiple TCP listeners, the configuration applies to all the TCP listeners.
    apiVersion: v1
    kind: Service
    metadata:
      annotations:
        service.beta.kubernetes.io/alibaba-cloud-loadbalancer-persistence-timeout: "1800"
      name: nginx
      namespace: default
    spec:
      ports:
      - port: 443
        protocol: TCP
        targetPort: 443
      selector:
        run: nginx
      type: LoadBalancer
  • Enable session persistence for an HTTP or HTTPS listener by inserting a cookie
    • The annotations in the following template apply only to HTTP-based and HTTPS-based SLB instances.
    • If an SLB instance has multiple HTTP or HTTPS listeners, the configuration applies to all the HTTP or HTTPS listeners.
    • To configure session persistence by inserting a cookie, all annotations in the following template are required.
    apiVersion: v1
    kind: Service
    metadata:
      annotations:
        service.beta.kubernetes.io/alibaba-cloud-loadbalancer-sticky-session: "on"
        service.beta.kubernetes.io/alibaba-cloud-loadbalancer-sticky-session-type: "insert"
        service.beta.kubernetes.io/alibaba-cloud-loadbalancer-cookie-timeout: "1800"
        service.beta.kubernetes.io/alibaba-cloud-loadbalancer-protocol-port: "http:80"
      name: nginx
      namespace: default
    spec:
      ports:
      - port: 80
        protocol: TCP
        targetPort: 80
      selector:
        run: nginx
      type: LoadBalancer
  • Enable access control for an SLB instance
    • Create an access control list (ACL) in the SLB console and record the ACL ID. Then, use the annotations in the following template to enable access control for an SLB instance.
    • A whitelist allows access only from specified IP addresses. A blacklist blocks access only from specific IP addresses.
    • All annotations in the following template are required.
    apiVersion: v1
    kind: Service
    metadata:
      annotations:
        service.beta.kubernetes.io/alibaba-cloud-loadbalancer-acl-status: "on"
        service.beta.kubernetes.io/alibaba-cloud-loadbalancer-acl-id: "${YOUR_ACL_ID}"
        service.beta.kubernetes.io/alibaba-cloud-loadbalancer-acl-type: "white"
      name: nginx
      namespace: default
    spec:
      ports:
      - port: 443
        protocol: TCP
        targetPort: 443
      selector:
        run: nginx
      type: LoadBalancer
  • Configure port forwarding for an SLB instance
    • Port forwarding allows an SLB instance to forward requests from an HTTP port to an HTTPS port.
    • Create a certificate in the SLB console and record the certificate ID. Then, use the following annotations to configure port forwarding for an SLB instance.
    • All annotations in the following template are required.
    apiVersion: v1
    kind: Service
    metadata:
      annotations:
        service.beta.kubernetes.io/alibaba-cloud-loadbalancer-protocol-port: "https:443,http:80"
        service.beta.kubernetes.io/alibaba-cloud-loadbalancer-cert-id: "${YOUR_CERT_ID}"
        service.beta.kubernetes.io/alibaba-cloud-loadbalancer-forward-port: "80:443"
      name: nginx
      namespace: default
    spec:
      ports:
      - name: https
        port: 443
        protocol: TCP
        targetPort: 80
      - name: http
        port: 80
        protocol: TCP
        targetPort: 80
      selector:
        run: nginx
      type: LoadBalancer
  • Set the scheduling algorithm for an SLB instance
    • rr: Requests are distributed to backend servers in sequence. This is the default scheduling algorithm.
    • wrr: Backend servers with higher weights receive more requests than backend servers with lower weights.
    apiVersion: v1
    kind: Service
    metadata:
      annotations:
        service.beta.kubernetes.io/alibaba-cloud-loadbalancer-scheduler: "wrr"
      name: nginx
      namespace: default
    spec:
      ports:
      - port: 443
        protocol: TCP
        targetPort: 443
      selector:
        run: nginx
      type: LoadBalancer
  • Create a UDP listener
    apiVersion: v1
    kind: Service
    metadata:
      annotations:
        service.beta.kubernetes.io/alibaba-cloud-loadbalancer-protocol-port: "udp:80"
      name: nginx
      namespace: default
    spec:
      ports:
      - port: 80
        protocol: TCP
        targetPort: 80
      selector:
        run: nginx
      type: LoadBalancer
  • Create an HTTP listener
    apiVersion: v1
    kind: Service
    metadata:
      annotations:
        service.beta.kubernetes.io/alibaba-cloud-loadbalancer-protocol-port: "http:80"
      name: nginx
      namespace: default
    spec:
      ports:
      - port: 80
        protocol: TCP
        targetPort: 80
      selector:
        run: nginx
      type: LoadBalancer
  • Create an HTTPS listener

    You must first create a certificate in the SLB console. Then, you can use the certificate ID and the following template to create a LoadBalancer Service and an HTTPS-based SLB instance.

    Note The HTTPS listeners of the SLB instance decrypt HTTPS requests into HTTP requests and forward the requests to pods on the backend servers.
    apiVersion: v1
    kind: Service
    metadata:
      annotations:
        service.beta.kubernetes.io/alibaba-cloud-loadbalancer-protocol-port: "https:443"
        service.beta.kubernetes.io/alibaba-cloud-loadbalancer-cert-id: "${YOUR_CERT_ID}"
      name: nginx
      namespace: default
    spec:
      ports:
      - port: 443
        protocol: TCP
        targetPort: 80
      selector:
        run: nginx
      type: LoadBalancer
  • Create a listener that has health checks enabled
    • Enable TCP health checks
      • By default, the health check feature is enabled for TCP listeners and cannot be disabled. The annotation service.beta.kubernetes.io/alibaba-cloud-loadbalancer-health-check-flag does not take effect.
      • To enable TCP health checks, all annotations in the following template are required.
      apiVersion: v1
      kind: Service
      metadata:
        annotations:
          service.beta.kubernetes.io/alibaba-cloud-loadbalancer-health-check-type: "tcp"
          service.beta.kubernetes.io/alibaba-cloud-loadbalancer-health-check-connect-timeout: "8"
          service.beta.kubernetes.io/alibaba-cloud-loadbalancer-healthy-threshold: "4"
          service.beta.kubernetes.io/alibaba-cloud-loadbalancer-unhealthy-threshold: "4"
          service.beta.kubernetes.io/alibaba-cloud-loadbalancer-health-check-interval: "3"
        name: nginx
        namespace: default
      spec:
        ports:
        - port: 80
          protocol: TCP
          targetPort: 80
        selector:
          run: nginx
        type: LoadBalancer
    • Enable UDP health checks
      • By default, the health check feature is enabled for UDP listeners and cannot be disabled. The annotation service.beta.kubernetes.io/alibaba-cloud-loadbalancer-health-check-flag does not take effect.
      • To enable UDP health checks, all annotations in the following template are required.
      apiVersion: v1
      kind: Service
      metadata:
        annotations:
          service.beta.kubernetes.io/alibaba-cloud-loadbalancer-protocol-port: "udp:80"
          service.beta.kubernetes.io/alibaba-cloud-loadbalancer-health-check-interval: "5"
          service.beta.kubernetes.io/alibaba-cloud-loadbalancer-health-check-connect-timeout: "10"
          service.beta.kubernetes.io/alibaba-cloud-loadbalancer-healthy-threshold: "3"
          service.beta.kubernetes.io/alibaba-cloud-loadbalancer-unhealthy-threshold: "3"
        name: nginx
        namespace: default
      spec:
        ports:
        - port: 80
          protocol: TCP
          targetPort: 80
        selector:
          run: nginx
        type: LoadBalancer
    • Enable HTTP health checks
      apiVersion: v1
      kind: Service
      metadata:
        annotations:
          service.beta.kubernetes.io/alibaba-cloud-loadbalancer-health-check-flag: "on"
          service.beta.kubernetes.io/alibaba-cloud-loadbalancer-health-check-type: "http"
          service.beta.kubernetes.io/alibaba-cloud-loadbalancer-health-check-uri: "/test/index.html"
          service.beta.kubernetes.io/alibaba-cloud-loadbalancer-healthy-threshold: "4"
          service.beta.kubernetes.io/alibaba-cloud-loadbalancer-unhealthy-threshold: "4"
          service.beta.kubernetes.io/alibaba-cloud-loadbalancer-health-check-timeout: "10"
          service.beta.kubernetes.io/alibaba-cloud-loadbalancer-health-check-interval: "3"
          service.beta.kubernetes.io/alibaba-cloud-loadbalancer-protocol-port: "http:80"
          # Specify the health check method. This annotation is optional.
          service.beta.kubernetes.io/alibaba-cloud-loadbalancer-health-check-method: "head"
        name: nginx
        namespace: default
      spec:
        ports:
        - port: 80
          protocol: TCP
          targetPort: 80
        selector:
          run: nginx
        type: LoadBalancer
  • Enable connection draining for a listener.

    Only TCP and UDP are supported. To enable connection draining for a listener, all annotations in the following template are required.

    apiVersion: v1
    kind: Service
    metadata:
      annotations:
        service.beta.kubernetes.io/alibaba-cloud-loadbalancer-connection-drain: "on"
        service.beta.kubernetes.io/alibaba-cloud-loadbalancer-connection-drain-timeout: "30"
      name: nginx
      namespace: default
    spec:
      ports:
      - port: 80
        protocol: TCP
        targetPort: 80
      selector:
        run: nginx
      type: LoadBalancer
  • Use the X-Forwarded-Proto header to identify the listener protocol of an SLB instance.
    Only HTTP and HTTPS are supported.
    apiVersion: v1
    kind: Service
    metadata:
      annotations:
        service.beta.kubernetes.io/alibaba-cloud-loadbalancer-protocol-port: "http:80"
        service.beta.kubernetes.io/alibaba-cloud-loadbalancer-xforwardedfor-proto: "on"
      name: nginx
      namespace: default
    spec:
      ports:
      - port: 80
        protocol: TCP
        targetPort: 80
      selector:
        run: nginx
      type: LoadBalancer
  • Specify the timeout period of idle connections for a listener
    Only HTTP and HTTPS are supported.
    apiVersion: v1
    kind: Service
    metadata:
      annotations:
        service.beta.kubernetes.io/alibaba-cloud-loadbalancer-protocol-port: "http:80"
        service.beta.kubernetes.io/alibaba-cloud-loadbalancer-idle-timeout: "30"
      name: nginx
      namespace: default
    spec:
      ports:
      - port: 80
        protocol: TCP
        targetPort: 80
      selector:
        run: nginx
      type: LoadBalancer
  • Disable HTTP/2 for a listener.
    Only HTTPS is supported.
    apiVersion: v1
    kind: Service
    metadata:
      annotations:
        service.beta.kubernetes.io/alibaba-cloud-loadbalancer-protocol-port: "https:443"
        service.beta.kubernetes.io/alibaba-cloud-loadbalancer-cert-id: "${YOUR_CERT_ID}"
        service.beta.kubernetes.io/alibaba-cloud-loadbalancer-http2-enabled: "off"
      name: nginx
      namespace: default
    spec:
      ports:
      - port: 443
        protocol: TCP
        targetPort: 80
      selector:
        run: nginx
      type: LoadBalancer
  • Specify the timeout period of requests for a listener.
    Only HTTP and HTTPS are supported.
    apiVersion: v1
    kind: Service
    metadata:
      annotations:
        service.beta.kubernetes.io/alibaba-cloud-loadbalancer-protocol-port: "http:80"
        service.beta.kubernetes.io/alibaba-cloud-loadbalancer-request-timeout: "60"
      name: nginx
      namespace: default
    spec:
      ports:
      - port: 80
        protocol: TCP
        targetPort: 80
      selector:
        run: nginx
      type: LoadBalancer
  • Specify the timeout period of connections for a listener.
    Only TCP is supported.
    apiVersion: v1
    kind: Service
    metadata:
      annotations:
        service.beta.kubernetes.io/alibaba-cloud-loadbalancer-established-timeout: "60"
      name: nginx
      namespace: default
    spec:
      ports:
      - port: 80
        protocol: TCP
        targetPort: 80
      selector:
        run: nginx
      type: LoadBalancer
  • Configure security policies for a listener
    Each security policy contains Transport Layer Security (TLS) protocol versions and cipher suites available for HTTPS. Only HTTPS is supported. For more information, see CreateLoadBalancerHTTPSListener.
    apiVersion: v1
    kind: Service
    metadata:
      annotations:
        service.beta.kubernetes.io/alibaba-cloud-loadbalancer-protocol-port: "https:443,http:80"
        service.beta.kubernetes.io/alibaba-cloud-loadbalancer-cert-id: "${YOUR_CERT_ID}"
        service.beta.kubernetes.io/alibaba-cloud-loadbalancer-tls-cipher-policy: "tls_cipher_policy_1_2"
      name: nginx
      namespace: default
    spec:
      ports:
      - name: https
        port: 443
        protocol: TCP
        targetPort: 443
      - name: http
        port: 80
        protocol: TCP
        targetPort: 80
      selector:
        run: nginx
      type: LoadBalancer

Backend server groups

Common annotations that are used to configure backend servers

  • Add worker nodes that have specified labels as the backend servers of an SLB instance

    Separate multiple labels with commas (,). Example: "k1=v1,k2=v2". A node must have all the specified labels before you can add the node as a backend server.

    apiVersion: v1
    kind: Service
    metadata:
      annotations:
        service.beta.kubernetes.io/alibaba-cloud-loadbalancer-backend-label: "failure-domain.beta.kubernetes.io/zone=ap-southeast-5a"
      name: nginx
      namespace: default
    spec:
      ports:
      - port: 443
        protocol: TCP
        targetPort: 443
      selector:
        run: nginx
      type: LoadBalancer
  • Add the nodes where pods that run the service are deployed as the backend servers of an SLB instance
    • By default, externalTrafficPolicy is set to Cluster for a Service. In Cluster mode, all nodes in the cluster are added as the backend servers of the SLB instance. In Local mode, only nodes where the pods are deployed are added as the backend servers of the SLB instance.

    • In Local mode, you must set the scheduling algorithm to weighted round-robin (WRR).

      Note

      For CCM 1.9.3.164-g2105d2e-aliyun and later, node weights are calculated based on the number of pods that run on each node for Services whose externalTrafficPolicy is set to Local. For more information about node weight calculation, see How does the CCM calculate node weights in Local mode?.

    apiVersion: v1
    kind: Service
    metadata:
      annotations:
        service.beta.kubernetes.io/alibaba-cloud-loadbalancer-scheduler: "wrr"
      name: nginx
      namespace: default
    spec:
      externalTrafficPolicy: Local
      ports:
      - port: 80
        protocol: TCP
        targetPort: 80
      selector:
        run: nginx
      type: LoadBalancer
  • Remove backend servers in the Unschedulable state from an SLB instance
    • You can run the kubectl cordon and kubectl drain commands to set a node to the Unschedulable state. By default, the annotation service.beta.kubernetes.io/alibaba-cloud-loadbalancer-remove-unscheduled-backend is set to off. In this case, nodes in the Unschedulable state are not removed from the backend server groups of an SLB instance.
    • To remove backend servers in the Unschedulable state from an SLB instance, set the annotation service.beta.kubernetes.io/alibaba-cloud-loadbalancer-remove-unscheduled-backend to on.
    apiVersion: v1
    kind: Service
    metadata:
      annotations:
        service.beta.kubernetes.io/alibaba-cloud-loadbalancer-remove-unscheduled-backend: "on"
      name: nginx
    spec:
      externalTrafficPolicy: Local
      ports:
      - name: http
        port: 30080
        protocol: TCP
        targetPort: 80
      selector:
        app: nginx
      type: LoadBalancer
  • Add pods that are assigned elastic network interfaces (ENIs) as the backend servers of an SLB instance
    When the Terway network plug-in is used, you can use the annotation service.beta.kubernetes.io/backend-type: "eni" to add pods that are assigned ENIs as the backend servers of an SLB instance. This improves network forwarding performance.
      apiVersion: v1
      kind: Service
      metadata:
        annotations:
          service.beta.kubernetes.io/backend-type: "eni"
        name: nginx
      spec:
        ports:
        - name: http
          port: 30080
          protocol: TCP
          targetPort: 80
        selector:
          app: nginx
        type: LoadBalancer
    Note You can also change eni in service.beta.kubernetes.io/backend-type: "eni" to ecs. This allows you to add Elastic Compute Service (ECS) instances as backend severs of an SLB instance.
  • Reuse an existing vServer group

    The annotation service.beta.kubernetes.io/alibaba-cloud-loadbalancer-vgroup-port can be used to reuse an existing vServer group that is added to an SLB instance. This annotation takes effect only when the SLB instance is reused. For more information, see Use the CCM to deploy services across clusters.

  • Set weights for Services to enable weighted round robin

    When a reused SLB instance is shared among multiple Services, the annotation service.beta.kubernetes.io/alibaba-cloud-loadbalancer-weight can be used to set the weight of each Service to enable weighted round robin. This annotation takes effect only when an existing vServer group is reused. For more information, see Use the CCM to deploy services across clusters.

Common annotations

  • Common annotations that are used to configure SLB instances

    Annotation Type Description Default Supported CCM version
    service.beta.kubernetes.io/alibaba-cloud-loadbalancer-address-type string The type of SLB instance. Valid values: internet and intranet.
    • internet: accesses the Service over the Internet. This is the default value. The address type of the SLB instance must be set to Internet.
    • intranet: accesses the Service over an internal network. The address type of the SLB instance must be set to intranet.
    internet CCM 1.9.3 and later
    service.beta.kubernetes.io/alibaba-cloud-loadbalancer-charge-type string The metering method of an SLB instance. Valid values: paybytraffic and paybybandwidth. paybytraffic CCM 1.9.3 and later
    service.beta.kubernetes.io/alibaba-cloud-loadbalancer-id string The ID of the SLB instance. You can specify an existing SLB instance by using the annotation service.beta.kubernetes.io/alibaba-cloud-loadbalancer-id. By default, if you specify an existing SLB instance, the CCM does not overwrite the listeners of the SLB instance. To overwrite the listeners, set the annotation service.beta.kubernetes.io/alibaba-cloud-loadbalancer-force-override-listeners to true. None CCM 1.9.3.81-gca19cd4-aliyun and later
    service.beta.kubernetes.io/alibaba-cloud-loadbalancer-spec string The specification of the SLB instance. For more information, see CreateLoadBalancer. slb.s1.small CCM 1.9.3 and later
    service.beta.kubernetes.io/alibaba-cloud-loadbalancer-master-zoneid string The ID of the zone for the primary backend server. None CCM 1.9.3.10-gfb99107-aliyun and later
    service.beta.kubernetes.io/alibaba-cloud-loadbalancer-slave-zoneid string The ID of the zone for the secondary backend servers. None CCM 1.9.3.10-gfb99107-aliyun and later
    service.beta.kubernetes.io/alibaba-cloud-loadbalancer-force-override-listeners string Specifies whether to overwrite the listeners of an existing SLB instance that is specified for a Service. false: does not overwrite the listeners of the existing SLB instance. CCM 1.9.3.81-gca19cd4-aliyun and later
    service.beta.kubernetes.io/alibaba-cloud-loadbalancer-bandwidth string The bandwidth of an SLB instance. This annotation applies only to Internet-facing SLB instances. 50 CCM 1.9.3.10-gfb99107-aliyun and later
    service.beta.kubernetes.io/alibaba-cloud-loadbalancer-vswitch-id string The ID of the vSwitch to which an SLB instance belongs. To set this annotation, you must set the annotation service.beta.kubernetes.io/alibaba-cloud-loadbalancer-address-type to intranet. None CCM 1.9.3 and later
    service.beta.kubernetes.io/alibaba-cloud-loadbalancer-additional-resource-tags string The tags that you want to add to an SLB instance. Separate multiple tags with commas (,). Example: "k1=v1,k2=v2". None CCM 1.9.3 and later
    service.beta.kubernetes.io/alibaba-cloud-loadbalancer-ip-version string The IP protocol of an SLB instance. Valid values: ipv4 and ipv6. ipv4 CCM 1.9.3.220-g24b1885-aliyun and later
    service.beta.kubernetes.io/alibaba-cloud-loadbalancer-delete-protection string Specifies whether to enable deletion protection for an SLB instance. Valid values: on and off. on CCM 1.9.3.313-g748f81e-aliyun and later
    service.beta.kubernetes.io/alibaba-cloud-loadbalancer-modification-protection string Specifies whether to enable the configuration read-only mode for an SLB instance. Valid values: ConsoleProtection and NonProtection. ConsoleProtection CCM 1.9.3.313-g748f81e-aliyun and later
    service.beta.kubernetes.io/alibaba-cloud-loadbalancer-resource-group-id string The resource group to which an SLB instance belongs. None CCM 1.9.3.313-g748f81e-aliyun and later
    service.beta.kubernetes.io/alibaba-cloud-loadbalancer-name string The name of the CLB instance. None CCM 1.9.3.313-g748f81e-aliyun and later
    service.beta.kubernetes.io/alibaba-cloud-loadbalancer-hostname string Specifies a hostname for the Service. The hostname must comply with the naming conventions of domain names. None CCM 2.3.0 and later
    service.beta.kubernetes.io/alibaba-cloud-loadbalancer-instance-charge-type string The metering method of the SLB instance. Valid values:
    • PayBySpec: The SLB instance is billed based on its specification. This is the default value.
    • PayByCLCU: The SLB instance is billed based on LCUs.
    PayBySpec CCM 2.4.0 and later
  • Common annotations that are used to configure listeners

    Annotation Type Description Default Supported CCM version
    service.beta.kubernetes.io/alibaba-cloud-loadbalancer-scheduler string The scheduling algorithm. Valid values: wrr and rr.
    • wrr: Backend servers with higher weights receive more requests than backend servers with lower weights.
    • rr: Requests are distributed to backend servers in sequence. This is the default scheduling algorithm.
    rr CCM 1.9.3 and later
    service.beta.kubernetes.io/alibaba-cloud-loadbalancer-protocol-port string The listening port. Separate multiple ports with commas (,). Example: https:443,http:80. None CCM 1.9.3 and later
    service.beta.kubernetes.io/alibaba-cloud-loadbalancer-persistence-timeout string The session persistence period. Unit: seconds.

    This annotation applies only to TCP listeners. Valid values: 0 to 3600.

    Default value: 0. By default, session persistence is disabled.

    For more information, see CreateLoadBalancerTCPListener.

    0 CCM 1.9.3 and later
    service.beta.kubernetes.io/alibaba-cloud-loadbalancer-sticky-session string Specifies whether to enable session persistence. Valid values: on and off.
    Note This annotation applies only to HTTP and HTTPS listeners.

    For more information, see CreateLoadBalancerHTTPListener and CreateLoadBalancerHTTPSListener.

    off CCM 1.9.3 and later
    service.beta.kubernetes.io/alibaba-cloud-loadbalancer-sticky-session-type string The method that is used to process cookies. Valid values:
    • insert: inserts a cookie.
    • server: rewrites a cookie.
    Note
    • This annotation applies only to HTTP and HTTPS listeners.
    • If the annotation service.beta.kubernetes.io/alibaba-cloud-loadbalancer-sticky-session is set to on, you must specify this annotation.

    For more information, see CreateLoadBalancerHTTPListener and CreateLoadBalancerHTTPSListener.

    None CCM 1.9.3 and later
    service.beta.kubernetes.io/alibaba-cloud-loadbalancer-cookie-timeout string The timeout period of a cookie. Valid values: 1 to 86400. Unit: seconds.
    Note If service.beta.kubernetes.io/alibaba-cloud-loadbalancer-sticky-session is set to on and service.beta.kubernetes.io/alibaba-cloud-loadbalancer-sticky-session-type is set to insert, you must specify this annotation.

    For more information, see CreateLoadBalancerHTTPListener and CreateLoadBalancerHTTPSListener.

    None CCM 1.9.3 and later
    service.beta.kubernetes.io/alibaba-cloud-loadbalancer-cookie string The name of a cookie configured on a backend server.

    The name must be 1 to 200 characters in length, and can contain only ASCII letters and digits. It cannot contain commas (,), semicolons (;), or space characters. It cannot start with a dollar sign ($).

    Note

    If service.beta.kubernetes.io/alibaba-cloud-loadbalancer-sticky-session is set to on and service.beta.kubernetes.io/alibaba-cloud-loadbalancer-sticky-session-type is set to server, you must specify this annotation.

    For more information, see CreateLoadBalancerHTTPListener and CreateLoadBalancerHTTPSListener.

    None CCM 1.9.3 and later
    service.beta.kubernetes.io/alibaba-cloud-loadbalancer-cert-id string The ID of a certificate for an SLB instance. You must first upload the certificate in the SLB console. None CCM 1.9.3.164-g2105d2e-aliyun and later
    service.beta.kubernetes.io/alibaba-cloud-loadbalancer-health-check-flag string Valid values: on and off.
    • Default value for TCP listeners: on. You cannot modify the value.
    • Default value for HTTP listeners: off.
    Default value: off. This annotation is optional for TCP listeners. By default, the health check feature is enabled for TCP listeners. This feature cannot be disabled. CCM 1.9.3 and later
    service.beta.kubernetes.io/alibaba-cloud-loadbalancer-health-check-type string The type of health check. Valid values: tcp and http.

    For more information, see CreateLoadBalancerTCPListener.

    tcp CCM 1.9.3 and later
    service.beta.kubernetes.io/alibaba-cloud-loadbalancer-health-check-uri string The URI that is used for health checks.
    Note If the type of health check is TCP, this annotation is optional.

    For more information, see CreateLoadBalancerTCPListener.

    None CCM 1.9.3 and later
    service.beta.kubernetes.io/alibaba-cloud-loadbalancer-health-check-connect-port string The port that is used for health checks. Valid values: 1 to 65535. None CCM 1.9.3 and later
    service.beta.kubernetes.io/alibaba-cloud-loadbalancer-healthy-threshold string The number of consecutive successful health checks that must occur before an unhealthy backend server is declared healthy.

    Valid values: 2 to 10.

    For more information, see CreateLoadBalancerTCPListener.

    3 CCM 1.9.3 and later
    service.beta.kubernetes.io/alibaba-cloud-loadbalancer-unhealthy-threshold string The number of consecutive failed health checks that must occur before a healthy backend server is declared unhealthy. Valid values:

    2~10

    For more information, see CreateLoadBalancerTCPListener.

    3 CCM 1.9.3 and later
    service.beta.kubernetes.io/alibaba-cloud-loadbalancer-health-check-interval string The interval between two consecutive health checks.

    Valid values: 1 to 50. Unit: seconds.

    For more information, see CreateLoadBalancerTCPListener.

    2 CCM 1.9.3 and later
    service.beta.kubernetes.io/alibaba-cloud-loadbalancer-health-check-connect-timeout string The timeout period of a health check. This annotation applies to TCP health checks. If a backend server does not respond within the specified timeout period, the server fails to pass the health check.

    Valid values: 1 to 300. Unit: seconds.

    Note If the value of service.beta.kubernetes.io/alibaba-cloud-loadbalancer-health-check-connect-timeout is smaller than that of service.beta.kubernetes.io/alibaba-cloud-loadbalancer-health-check-interval, service.beta.kubernetes.io/alibaba-cloud-loadbalancer-health-check-connect-timeout does not take effect. In this case, the value of service.beta.kubernetes.io/alibaba-cloud-loadbalancer-health-check-interval is used as the timeout period.

    For more information, see CreateLoadBalancerTCPListener.

    5 CCM 1.9.3 and later
    service.beta.kubernetes.io/alibaba-cloud-loadbalancer-health-check-timeout string The timeout period of a health check. This annotation applies to HTTP health checks. If a backend server does not respond within the specified timeout period, the server fails to pass the health check.

    Valid values: 1 to 300. Unit: seconds.

    Note If the value of service.beta.kubernetes.io/alibaba-cloud-loadbalancer-health-check-timeout is smaller than that of service.beta.kubernetes.io/alibaba-cloud-loadbalancer-health-check-interval, service.beta.kubernetes.io/alibaba-cloud-loadbalancer-health-check-timeout does not take effect. In this case, the value of service.beta.kubernetes.io/alibaba-cloud-loadbalancer-health-check-interval is used as the timeout period.

    For more information, see CreateLoadBalancerTCPListener.

    5 CCM 1.9.3 and later
    service.beta.kubernetes.io/alibaba-cloud-loadbalancer-health-check-domain string The domain name that is used for health checks.
    • $_ip: the private IP addresses of backend servers. If you do not set this annotation or set the annotation to $_ip, the SLB instance uses the private IP address of each backend server as the domain name for health checks.
    • domain: The domain name must be 1 to 80 characters in length, and can contain only letters, digits, periods (.), and hyphens (-).
    None CCM 1.9.3 and later
    service.beta.kubernetes.io/alibaba-cloud-loadbalancer-health-check-httpcode string The HTTP status code that is returned when a health check is successful. Separate multiple HTTP status codes with commas (,). Valid values:
    • http_2xx
    • http_3xx
    • http_4xx
    • http_5xx
    Default value: http_2xx.
    http_2xx CCM 1.9.3 and later
    service.beta.kubernetes.io/alibaba-cloud-loadbalancer-acl-status string Specifies whether to enable access control for a listener. Valid values: on and off. off CCM 1.9.3.164-g2105d2e-aliyun and later
    service.beta.kubernetes.io/alibaba-cloud-loadbalancer-acl-id string The ID of the network access control list (ACL) that is associated with a listener. If the annotation service.beta.kubernetes.io/alibaba-cloud-loadbalancer-acl-status is set to on, this annotation is required. None CCM 1.9.3.164-g2105d2e-aliyun and later
    service.beta.kubernetes.io/alibaba-cloud-loadbalancer-acl-type string The type of the network ACL.

    Valid values: white and black.

    • white: specifies the network ACL as a whitelist. Only requests from the IP addresses or CIDR blocks in the network ACL are forwarded. Whitelists apply to scenarios when you want to allow only specified IP addresses to access an application. Your service may be adversely affected if the whitelist is not properly configured. After a whitelist is set, the SLB instance forwards only requests from the IP addresses in the whitelist. If a whitelist is configured but no IP address is added to the whitelist, the listener forwards all requests.
    • black: specifies the network ACL as a blacklist. All requests from the IP addresses or CIDR blocks in the network ACL are rejected. Blacklists apply to scenarios when you want to block access from specified IP addresses to an application. If a blacklist is configured but no IP address is added to the blacklist, the listener forwards all requests. If the annotation service.beta.kubernetes.io/alibaba-cloud-loadbalancer-acl-status is set to on, this annotation is required.
    None CCM 1.9.3.164-g2105d2e-aliyun and later
    service.beta.kubernetes.io/alibaba-cloud-loadbalancer-forward-port string Redirects HTTP requests to the specified port of an HTTPS listener. Example: 80:443. None CCM 1.9.3.164-g2105d2e-aliyun and later
    service.beta.kubernetes.io/alibaba-cloud-loadbalancer-connection-drain string Specifies whether to enable connection draining. Valid values: on and off. None CCM 2.0.1 and later
    service.beta.kubernetes.io/alibaba-cloud-loadbalancer-connection-drain-timeout string The timeout period of connection draining. Unit: seconds. Value values: 10 to 900. None CCM 2.0.1 and later
    service.beta.kubernetes.io/alibaba-cloud-loadbalancer-xforwardedfor-proto string Specifies whether to use the X-Forwarded-Proto header to obtain the listener protocol of an SLB instance. Valid values: on and off. off CCM 2.1.0 and later
    service.beta.kubernetes.io/alibaba-cloud-loadbalancer-idle-timeout string Specifies the timeout period of idle connections for a listener. Unit: seconds. Valid values: 1 to 60. 15 CCM 2.1.0 and later
    service.beta.kubernetes.io/alibaba-cloud-loadbalancer-http2-enabled string Specifies whether to enable HTTP/2. Valid values: on and off. on CCM 2.1.0 and later
    service.beta.kubernetes.io/alibaba-cloud-loadbalancer-established-timeout string The timeout period of connections. Only TCP is supported.

    Unit: seconds. Value values: 10 to 900. For more information, see CreateLoadBalancerTCPListener.

    None CCM 2.3.0 and later
    service.beta.kubernetes.io/alibaba-cloud-loadbalancer-request-timeout string The timeout period of requests.

    HTTP and HTTPS are supported. Unit: seconds. Valid values: 1 to 180. If no response is received from the backend server within the timeout period, SLB returns an HTTP 504 error code to the client.

    60 CCM 2.3.0 and later
    service.beta.kubernetes.io/alibaba-cloud-loadbalancer-health-check-method string The health check method used by HTTP listeners. Valid values: head and get. None CCM 2.3.0 and later
    service.beta.kubernetes.io/alibaba-cloud-loadbalancer-tls-cipher-policy string The TLS security policy. Each security policy contains TLS protocol versions and cipher suites available for HTTPS. For more information, see CreateLoadBalancerHTTPSListener. Valid values:
    • tls_cipher_policy_1_0
    • tls_cipher_policy_1_1
    • tls_cipher_policy_1_2
    • tls_cipher_policy_1_2_strict
    • tls_cipher_policy_1_2_strict_with_1_3
    tls_cipher_policy_1_0 CCM 2.4.0 and later
  • Common annotations that are used to configure backend server groups

    Annotation Type Description Default Supported CCM version
    service.beta.kubernetes.io/alibaba-cloud-loadbalancer-backend-label string Specifies that worker nodes that have matching labels are added as backend servers of an SLB instance. None CCM 1.9.3 and later
    externalTrafficPolicy string The policy that is used to add nodes as backend servers. Valid values:
    • Cluster: adds all nodes as backend servers.
    • Local: adds the nodes where the pods are deployed as backend servers.
    Cluster CCM 1.9.3 and later
    service.beta.kubernetes.io/alibaba-cloud-loadbalancer-remove-unscheduled-backend string Removes backend servers in the Unschedulable state from an SLB instance. Valid values: on and off. off CCM 1.9.3.164-g2105d2e-aliyun and later
    service.beta.kubernetes.io/backend-type string The type of backend servers added to an SLB instance.
    Valid values:
    • eni: adds pods as the backend servers of an SLB instance. This parameter takes effect only in Terway network mode. This improves network forwarding performance.
    • ecs: adds ECS instances as the backend servers of an SLB instance.
    When the Flannel network plug-in is used, the default value is ecs.
    When the Terway network plug-in is used:
    • The default value is ecs. This applies to Container Service for Kubernetes (ACK) clusters that were created before August 10, 2020.
    • The default value is eni. This applies to ACK clusters that are created after August 10, 2020.
    CCM 1.9.3.164-g2105d2e-aliyun and later
    service.beta.kubernetes.io/alibaba-cloud-loadbalancer-vgroup-port string Reuses an existing vServer group. This annotation takes effect only when the existing SLB instance is reused. Separate multiple ports and vServer groups with commas (,). None CCM 2.0.1 and later
    service.beta.kubernetes.io/alibaba-cloud-loadbalancer-weight string Specifies the weight of a Service when an SLB instance is shared by multiple Services. This annotation takes effect only when the existing vServer group is reused. None CCM 2.0.1 and later