Considerations for configuring a LoadBalancer Service and the policies used by the CCM to update SLB resources. - Container Service for Kubernetes

When you specify Type=LoadBalancer for a Service, the Cloud Controller Manager (CCM) creates and configures Server Load Balancer (SLB) resources for the Service, including SLB instances, listeners, and backend server groups. This topic describes the considerations for configuring a LoadBalancer Service and the policies that are used by the CCM to update SLB resources.

Policies used by the CCM to update SLB resources

Container Service for Kubernetes (ACK) allows you to specify an existing SLB instance for a Service. You can also use the CCM to automatically create an SLB instance for the Service. The two methods use different policies to update SLB resources. The following table describes the differences.

Resource object	Existing SLB instance	SLB instance created and managed by the CCM
SLB	Use the following annotation to specify an existing SLB instance for a Service: `service.beta.kubernetes.io/alibaba-cloud-loadbalancer-id`. The CCM uses the specified SLB instance to enable load balancing. You can use other annotations to configure the SLB instance. The CCM automatically creates vServer groups for the instance. If the Service is deleted, the CCM does not delete the existing SLB instance that is specified in the annotation.	The CCM automatically creates, configures, and manages SLB resources based on the Service configuration, including the SLB instance, listeners, and vServer groups. If the Service is deleted, the CCM deletes the created SLB instance.
Listener	Use the following annotation to configure listeners: `service.beta.kubernetes.io/alibaba-cloud-loadbalancer-force-override-listeners`. If you set the annotation to `false`, the CCM does not configure or manage listeners for the SLB instance. If you set the annotation to `true`, the CCM configures and manages listeners for the SLB instance based on the Service configuration. If the SLB instance has existing listeners, the CCM creates new listeners to replace the existing ones.	The CCM configures listeners for the SLB instance based on the Service configuration.
Backend server group	When the endpoints of a Service change or the cluster nodes change, the CCM automatically updates the vServer groups of the SLB instance created for the Service. If the cluster uses the Terway network plug-in, the CCM associates pod IP addresses instead of Elastic Compute Service (ECS) nodes with SLB instances by default. If the cluster uses the Flannel network plug-in, the policies for updating backend server groups vary based on the mode of the Service. If `spec.externalTrafficPolicy = Cluster` is specified for a Service, the CCM adds all cluster nodes to the SLB instance by default. If node labels are specified in the Service configuration, the CCM adds only cluster nodes that have the specified labels to the SLB instance. Important The number of SLB instances to which an ECS instance can be added is limited. If a Service is in Cluster mode, the quota is consumed at a high rate. When the quota is used up, Service reconciliation fails. To fix this issue, set the Local mode for the Service. If `spec.externalTrafficPolicy = Local` is specified for a Service, the CCM adds only the nodes where the backend pods of the Service are deployed to the SLB instance. This can reduce the consumption rate of SLB quotas. In addition, source IP addresses can also be preserved in Layer 4 load balancing. If the cluster uses the Flannel network plug-in, the CCM does not add master nodes to the SLB instances used by the cluster. If the cluster uses the Flannel network plug-in, the CCM does not remove a node after you run the `kubectl drain` command on the node. In addition, the CCM does not remove a node after you run the `kubectl cordon` command to mark the node as unschedulable. To remove such a node, set the `service.beta.kubernetes.io/alibaba-cloud-loadbalancer-remove-unscheduled-backend` annotation to `on`.

Usage notes

Before you reuse an existing SLB instance, check whether the instance meets the following requirements:
- The SLB instance that you want to reuse is created in the SLB console. You cannot reuse an SLB instance that is created by the CCM.
- To reuse an internal-facing SLB instance for a cluster, the SLB instance and the cluster must be deployed in the same virtual private cloud (VPC).
Considerations for using the CCM to configure an SLB instance
- The CCM configures SLB instances only for LoadBalancer Services.
  Important
  If you change the of a Service from Type=LoadBalancer to Type!=LoadBalancer, the CCM automatically deletes the configurations related to the SLB instance created for Service. As a result, you cannot use the SLB instance to access the Service.
- When specific conditions are met, the CCM uses a declarative API to automatically update the configuration of an SLB instance based on the Service configuration. If you set service.beta.kubernetes.io/alibaba-cloud-loadbalancer-force-override-listeners to true, the SLB configurations that you update in the SLB console may be overwritten.
  Important
  If the SLB instance is created and managed by the CCM, we recommend that you do not modify the configuration of the SLB instance in the SLB console. Otherwise, the CCM may overwrite the configuration and the Service may become unavailable.

Quotas

VPC

A node in a cluster is mapped to a route entry in a route table. By default, each route table for a VPC can contain up to 200 entries. If the number of nodes in a cluster exceeds 200, apply for a quota increase in the log on to the Quota Center console and submit an application
For more information about the limits and quotas related to VPC, see Limits and quotas.
To query VPC resource quotas, go to the Quota Management page in the VPC console.

SLB

The CCM creates SLB instances for LoadBalancer Services. By default, you can have at most 60 SLB instances within your Alibaba Cloud account. To create more SLB instances, apply for a quota increase in the log on to the Quota Center console and submit an application.
The CCM automatically adds ECS instances to the backend server groups of an SLB instance based on the Service configurations.
- By default, an ECS instance can be added to at most 50 backend server groups. To add the ECS instance to more backend server groups, apply for a quota increase in the log on to the Quota Center console and submit an application.
- By default, you can add at most 200 backend servers to an SLB instance. To add more backend servers to an SLB instance, apply for a quota increase in the log on to the Quota Center console and submit an application.
The CCM automatically creates listeners that use Service ports for SLB instances. By default, each SLB instance supports at most 50 listeners. To increase the number of listeners supported by each SLB instance, apply for a quota increase in the log on to the Quota Center console and submit an application.
For more information about the limits on SLB, see Limits.
To query SLB resource quotas, go to the Quota Management page in the SLB console.