Considerations for configuring a LoadBalancer type Service - Container Service for Kubernetes

When you specify Type=LoadBalancer for a Service, the Cloud Controller Manager (CCM) creates and configures Server Load Balancer (SLB) resources for the Service, including SLB instances, listeners, and vServer groups. This topic describes the considerations for configuring a LoadBalancer type Service and the policies that are used by the CCM to update SLB resources.

Policies that are used by the CCM to update SLB resources

Container Service for Kubernetes (ACK) allows you to specify an existing SLB instance for a Service. You can also use the CCM to automatically create an SLB instance for the Service. The two methods use different policies to update SLB resources. The following table describes the differences.


Resource object	Existing SLB instance	SLB instance created and managed by the CCM
SLB	Use the following annotation to specify an existing SLB instance for a Service: `service.beta.kubernetes.io/alibaba-cloud-loadbalancer-id` The CCM uses the specified SLB instance to enable load balancing. You can use other annotations to configure the SLB instance. The CCM automatically creates vServer groups for the instance. If the Service is deleted, the CCM does not delete the existing SLB instance that is specified in the annotation.	The CCM automatically creates, configures, and manages SLB resources based on the Service configuration, including the SLB instance, listeners, and vServer groups. When the Service is deleted, the CCM deletes the created SLB instance.
Listener	Use the following annotation to configure listeners: `service.beta.kubernetes.io/alibaba-cloud-loadbalancer-force-override-listeners:`. If you set the annotation to false, the CCM does not configure or manage listeners for the SLB instance. If you set the annotation to true, the CCM configures and manages listeners for the SLB instance based on the Service configuration. If the SLB instance has existing listeners, the CCM creates new listeners to replace the existing ones.	The CCM configures listeners for the SLB instance based on the Service configuration.
Backend server group	When the endpoint of an Elastic Compute Service (ECS) instance in a vServer group for a Service changes or the cluster nodes are changed, the CCM updates the vServer groups. The policies for updating vServer groups vary based on the mode of the Service. If `spec.externalTrafficPolicy = Cluster` is specified for a Service, the CCM adds all cluster nodes to the vServer group of the SLB instance. If node labels are specified in the Service configuration, the CCM adds cluster nodes that have the specified labels to the vServer group of the SLB instance. Important SLB limits the number of vServer groups to which an ECS instance can be added. If a Service is in Cluster mode, the quota is consumed at a high rate. When the quota is used up, Service reconciliation fails. To fix this issue, set the Local mode for the Service. If `spec.externalTrafficPolicy = Local` is specified for a Service, the CCM adds only the nodes where the pods that are related to the Service are deployed to the vServer group of the SLB instance. This can reduce the consumption rate of the resources. Source IP addresses can also be preserved in Layer 4 load balancing. The CCM does not add master nodes of a cluster to the vServer group of an SLB instance. Assume that you have run the `kubectl drain` command to remove a node from a cluster, or run the `kubectl cordon` command to mark a node as unschedulable. By default, the CCM does not remove such a node from the vServer group of an SLB instance. To remove such a node, set annotation `service.beta.kubernetes.io/alibaba-cloud-loadbalancer-remove-unscheduled-backend` to `on`.

Precautions

Before you reuse an existing SLB instance, check whether the instance meets the following requirements:
- The SLB instance that you want to reuse is created in the SLB console. You cannot reuse an SLB instance that is created by CCM.
- To reuse an internal-facing SLB instance for a cluster, the SLB instance and the cluster must be deployed in the same virtual private cloud (VPC).
Considerations for using the CCM to configure an SLB instance
- The CCM configures SLB instances only for LoadBalancer type Services.
  Important If you change the type of a Service from Type=LoadBalancer to Type!=LoadBalancer, the CCM automatically deletes the configurations related to the SLB instance. As a result, you cannot use the SLB instance to access the Service that is associated with the SLB instance.
- When specific conditions are met, the CCM uses a declarative API to automatically update the configuration of an SLB instance based on the Service configuration. If you set service.beta.kubernetes.io/alibaba-cloud-loadbalancer-force-override-listeners: to true, the SLB configurations that you update in the SLB console may be overwritten.
  Important If the SLB instance is created and managed by the CCM, we recommend that you do not modify the configuration of the SLB instance in the SLB console. Otherwise, the CCM may overwrite the configuration and the Service may become unavailable.

Quotas

VPC

A node in a cluster is mapped to a route entry in a route table. By default, each route table for a VPC can contain up to 200 entries. If the number of nodes in a cluster exceeds 200, apply for a quota increase in Go to the Quota Center page to submit a ticket.
For more information about VPC resource quotas, see Limits and quotas.
To query the VPC resource quotas, go to the Quota Management page in the VPC console.

SLB

The CCM creates SLB instances for Services that have Type=LoadBalancer configured. By default, you can have at most 60 SLB instances within your Alibaba Cloud account. To create more SLB instances, apply for a quota increase in Go to the Quota Center page to submit a ticket.
The CCM automatically adds Elastic Compute Service (ECS) instances to the backend server groups of an SLB instance based on the Service configurations.
- By default, an ECS instance can be added to at most 50 backend server groups. To add the ECS instance to more backend server groups, apply for a quota increase in Go to the Quota Center page to submit a ticket.
- By default, you can add at most 200 backend servers to an SLB instance. To add more backend servers to an SLB instance, apply for a quota increase in Go to the Quota Center page to submit a ticket.
The CCM automatically creates listeners that use Service ports for SLB instances. By default, each SLB instance supports at most 50 listeners. To increase the number of listeners supported by each SLB instance, apply for a quota increase in Go to the Quota Center page to submit a ticket.
For more information about SLB resource quotas, see Limits.
To query the SLB resource quotas, go to the Quota Management page in the SLB console.