This topic introduces cloud controller manager (CCM) and provides usage notes and release notes for the component.

Introduction

The CCM allows you to integrate Kubernetes with Alibaba Cloud services, such as Classic Load Balancer (CLB), formerly known as Server Load Balancer (SLB), and Virtual Private Cloud (VPC). The CCM provides features to manage SLB instances and enable cross-node communication.
  • Manage SLB instances

    If you set Type=LoadBalancer for a Service, the CCM automatically creates a CLB instance for the Service, and configures listeners and backend server groups. When the endpoint of an Elastic Compute Service (ECS) instance in a vServer group for a Service is changed or the cluster nodes are changed, the CCM automatically updates the vServer groups of the CLB instance.

  • Enable cross-node communication

    If Flannel is used as the network plug-in of a Kubernetes cluster, the CCM can enable network connections between containers and nodes. This allows you to implement cross-node communication. The CCM writes the pod CIDR block to the route table of the VPC where the cluster is deployed. This enables cross-node communication. You can use this feature after the CCM is installed. No additional configuration is required.

Usage notes

Release notes

November 2021
Version Image address Release date Description Impact
v2.1.0 registry.cn-hangzhou.aliyuncs.com/acs/cloud-controller-manager-amd64:v2.1.0 2021-11-22
  • New features:
    • The annotation service.beta.kubernetes.io/alibaba-cloud-loadbalancer-xforwardedfor-proto can be used to identify the listener protocol of an SLB instance.
    • The annotation service.beta.kubernetes.io/alibaba-cloud-loadbalancer-idle-timeout can be used to set the timeout period of idle connections.
    • The annotation service.beta.kubernetes.io/alibaba-cloud-loadbalancer-http2-enabled can be used to enable HTTP2.
  • Improvements:

    The annotation service.beta.kubernetes.io/alibaba-cloud-loadbalancer-weight can be set to 0, which is suitable for traffic shifting between clusters.

  • Fixed issues:
    • The issue that listeners cannot be created for a CLB instance when a large number of backend pods are added to the CLB instance.
    • The issue that the CLB instance used by a Service is not updated after the targetPort parameter of the Service is updated.
No impact on workloads
September 2021
Version Image address Release date Description Impact
v2.0.1 registry.cn-hangzhou.aliyuncs.com/acs/cloud-controller-manager-amd64:v2.0.1 2021-09-02
  • New features:
    • The annotation service.beta.kubernetes.io/alibaba-cloud-loadbalancer-vgroup-port can be used to reuse an existing vServer group that is attached to an SLB instance. This annotation takes effect only when the SLB instance is reused. For more information, see Use the CCM to deploy services across clusters.
    • When a reused SLB instance is shared among multiple Services, the annotation service.beta.kubernetes.io/alibaba-cloud-loadbalancer-weight can be used to set the weight of each Service to enable weighted round robin. This annotation takes effect only when the existing vServer group is reused. For more information, see Use the CCM to deploy services across clusters.
    • The annotation service.beta.kubernetes.io/alibaba-cloud-loadbalancer-connection-drain can be used to configure connection draining for an SLB instance. Only TCP and UDP are supported.
    • The annotation service.beta.kubernetes.io/alibaba-cloud-loadbalancer-connection-drain-timeout can be used to set the timeout period of connection draining for an SLB instance. Only TCP and UDP are supported.
    • The targetPort field can be set to a String value.
    • Finalizers can be specified for LoadBalancer Services.
  • Improvements:
    • Alpine Linux is upgraded to V3.13 for base images.
    • The port used by Prometheus metrics is changed from 10258 to 8080.
    • The node labels are synchronized by schedule.
No impact on workloads
April 2021
Version Image address Release date Description Impact
v1.9.3.380-gd6d0962-aliyun registry.cn-hangzhou.aliyuncs.com/acs/cloud-controller-manager-amd64:v1.9.3.380-gd6d0962-aliyun 2021-04-20
  • The issue that default server groups cannot be updated is fixed.
  • Events are generated and alerts are triggered when an SLB instance has no backend servers.
No impact on workloads
March 2021
Version Image address Release date Description Impact
v1.9.3.378-g42eac35-aliyun registry.cn-hangzhou.aliyuncs.com/acs/cloud-controller-manager-amd64:v1.9.3.378-g42eac35-aliyun 2021-03-08
New features:
  • ECS instances other than those in the Container Service for Kubernetes (ACK) cluster can be added to a vServer group.
  • The label kubernetes.reused.by.user is automatically added to a reused SLB instance.
Improvements:
  • The number of concurrent threads for processing Services is increased to improve processing speed.
  • The processing logic of virtual-kubelet nodes is optimized to ignore Service updates caused by the status changes of virtual-kubelet nodes.
  • The node label service.beta.kubernetes.io/exclude-node is deprecated. To exclude a node from the management of the CCM, use the label service.alibabacloud.com/exclude-node instead.
  • Resource groups are verified when an SLB instance is reused. The resource group ID specified in annotations must be the ID of the resource group to which the SLB instance belongs. Otherwise, the SLB instance cannot be used to expose more than one Service.
  • The readability of event content is improved.
  • The version priority setting of annotations is optimized. If two versions of an annotation are added to the Service configurations, the later version prevails over the earlier version.
Fixed issues:
  • The issue that route entries failed to be deleted due to incomplete node configurations.
  • The logic of node initialization is optimized to fix the issue of taint missing. This prevents pods from being scheduled to a node for which route entries are not created during the initialization process.
No impact on workloads
December 2020
Version Image address Release date Description Impact
v1.9.3.339-g9830b58-aliyun registry.cn-hangzhou.aliyuncs.com/acs/cloud-controller-manager-amd64:v1.9.3.339-g9830b58-aliyun 2020-12-18
  • Hash values are supported in the configurations of LoadBalancer Services. This way, when the CCM is restarted, only the vServer groups of the related SLB instances are updated if the Service configuration is not changed. The configurations of the related SLB instances and listeners are not updated.
  • SLB API calls are optimized to reduce the chances of throttling.
No impact on workloads
September 2020
Version Image address Release date Description Impact
v1.9.3.316-g8daf1a9-aliyun registry.cn-hangzhou.aliyuncs.com/acs/cloud-controller-manager-amd64:v1.9.3.316-g8daf1a9-aliyun 2020-09-29
  • The occasional failure to update the vServer groups of SLB instances is fixed.
  • The health check port is changed from 10252 to 10258.
No impact on workloads
August 2020
Version Image address Release date Description Impact
v1.9.3.313-g748f81e-aliyun registry.cn-hangzhou.aliyuncs.com/acs/cloud-controller-manager-amd64:v1.9.3.313-g748f81e-aliyun 2020-08-10
  • New features:
    • The annotation service.beta.kubernetes.io/alibaba-cloud-loadbalancer-delete-protection can be used to set deletion protection for SLB instances. By default, deletion protection is enabled for newly created SLB instances.
    • The annotation service.beta.kubernetes.io/alibaba-cloud-loadbalancer-modification-protection can be used to set the configuration read-only mode for SLB instances. By default, the configuration read-only mode is enabled for newly created SLB instances.
    • The annotation service.beta.kubernetes.io/alibaba-cloud-loadbalancer-resource-group-id can be used to specify the resource group to which an SLB instance belongs. This setting applies only when you create an SLB instance and cannot be modified after the instance is created.
    • The annotation service.beta.kubernetes.io/alibaba-cloud-loadbalancer-name can be used to specify the name of an SLB instance.
    • The API operations of Alibaba Cloud services can be called over internal networks instead of the Internet. To call the CCM operations, Internet access is no longer required in all regions.
    • Tags are added to an SLB instance that is created for a LoadBalancer Service. The tags are in the ack.aliyun.com: {your-cluster-id} format. This feature applies to only newly created clusters.
    • The cloud provider ID can be specified in the <cloudProvider>://<optional>/<segments>/<provider id> format, which is compatible with open source Kubernetes.
    • When a LoadBalancer Service is created in a cluster that uses Terway, the backed pods are automatically added to the SLB instance that is associated with the Service. The IP addresses of elastic network interfaces (ENIs) that are allocated to the pods are added as the backend servers of the SLB instance. This improves network performance. For LoadBalancer Services, the targetPort field cannot be set to a string value.
  • Improvements:
    • Alpine Linux is upgraded to V3.11.6 for base images.
    • Listener updates are automatically synchronized to vServer groups.
    • SLB API operations are optimized. You can call the SLB API to create SLB instances with improved speed.
No impact on workloads
June 2020
Version Image address Release date Description Impact
v1.9.3.276-g372aa98-aliyun registry.cn-hangzhou.aliyuncs.com/acs/cloud-controller-manager-amd64: v1.9.3.276-g372aa98-aliyun 2020-06-11
  • New features:
    • The SLB instance attached to the cluster API server cannot be reused by LoadBalancer Services.
    • Prometheus metrics (ccm_node_latencies_duration_milliseconds, ccm_route_latencies_duration_milliseconds, and ccm_slb_latencies_duration_milliseconds) are added to monitor the synchronization latency of the CCM.
    • Events are collected to monitor the synchronization process between a Service and the related SLB instance.
  • Improvements:
    • Weight calculation is optimized for Services in Local mode. To enable the Local mode, set externalTrafficPolicy=Local in Service configurations. This improves load balancing among pods. For more information, see How does CCM calculate node weights in Local mode?.

    • API calls of cloud services are optimized to improve efficiency and reduce the chances of throttling.
    • When you delete a node with the label service.beta.kubernetes.io/exclude-node, the related route entries are no longer deleted.
  • Fixed issues:
    • The issue that persistence timeout cannot be set to 0 by adding annotations during Service updates.
    • The issue that bandwidth cannot be set to 100 by adding annotations during Service updates.
No impact on workloads
March 2020
Version Image address Release date Description Impact
v1.9.3.239-g40d97e1-aliyun registry.cn-hangzhou.aliyuncs.com/acs/cloud-controller-manager-amd64: v1.9.3.239-g40d97e1-aliyun 2020-03-05
  • New features:

    For LoadBalancer Services, the CCM allows you to specify both ECS nodes and ENIs as the backend servers of the related SLB instances.

  • Improvements:
    • The API operations of Alibaba Cloud services can be called over internal networks instead of the Internet. To call the CCM operations, Internet access is no longer required in regions other than China (Beijing), China (Shanghai), and UAE (Dubai).

    • The API operation that is used to query VPC route entries is changed to DescribeRouteEntryList. This provides higher performance when hundreds of queries are received within a short period of time.

No impact on workloads
December 2019
Version Image address Release date Description Impact
v1.9.3.220-g24b1885-aliyun registry.cn-hangzhou.aliyuncs.com/acs/cloud-controller-manager-amd64: v1.9.3.220-g24b1885-aliyun 2019-12-31
  • vSwitch IDs are supported. You can set vSwitch IDs in CloudConfig by using the following format: :vswithid1,:vswitchid2.
  • Backoff is supported when throttling is enabled. Backoff allows failed requests to rejoin the reconcile queue every 30 to 180 seconds.
  • The number of worker threads to be reconciled is adjusted to 2. This allows you to fully utilize the queries per second (QPS) quota on API calls to speed up the reconcile process.
  • The issue that the CCM quits unexpectedly due to concurrent Map reads and writes based on the aliyungo SDK is fixed.
  • When a node is removed from an ACK cluster, the related route entries are automatically deleted from the VPC route table by the CCM.
  • The issue that port configurations cannot be changed due to port dependencies for HTTP port forwarding is fixed.
  • If the backend server of an SLB instance is an ECS instance, the serverip field is no longer required when you change the backend server. This prevents errors caused by the changes of default serverip values in API requests when you add backend servers.
  • The route entries of a node are added to the VPC route table only if the status of the node is known.
  • NAT IP addresses are no longer added to node metadata by the CCM. This fixes the issue that the API server occasionally fails to connect to kubelet.
  • When you modify the configurations of a listener, the start listener operation is called only if the listener is in the inactive state. This prevents throttling on API requests.
No impact on workloads
November 2019
Version Image address Release date Description Impact
v1.9.3.193-g6cddde4-aliyun registry.cn-hangzhou.aliyuncs.com/acs/cloud-controller-manager-amd64:v1.9.3.193-g6cddde4-aliyun 2019-11-19
  • The label service.beta.kubernetes.io/exclude-node can be added to a node. After the label is added, the node is no longer managed by the CCM.
  • Multiple backedn pods can be added to an SLB instance at a time. The network type of the pods must be Terway.
  • The node weight cannot be less than 1 for Services in Local mode (when externalTrafficPolicy=Local is set for the Services).
  • The issue that vServer groups are repeatedly created when concurrent requests are processed is fixed.
  • The issue that stale data is generated due to caching when you set node weights is fixed.
No impact on workloads
September 2019
Version Image address Release date Description Impact
v1.9.3.164-g2105d2e-aliyun registry.cn-hangzhou.aliyuncs.com/acs/cloud-controller-manager-amd64:v1.9.3-164-g2105d2e-aliyun 2019-09-11
  • The annotation service.beta.kubernetes.io/alibaba-cloud-loadbalancer-cert-id can be used to renew a certificate.
  • The annotation service.beta.kubernetes.io/alibaba-cloud-loadbalancer-forward-port can be used to enable port forwarding from an HTTP port to an HTTPS port.
  • The following annotations can be used to create SLB instances with access control list (ACL) settings: service.beta.kubernetes.io/alibaba-cloud-loadbalancer-acl-status, service.beta.kubernetes.io/alibaba-cloud-loadbalancer-acl-id, and service.beta.kubernetes.io/alibaba-cloud-loadbalancer-acl-type.
  • The annotation service.beta.kubernetes.io/alibaba-cloud-loadbalancer-remove-unscheduled-backend can be used to remove unschedulable nodes.
  • The annotation service.beta.kubernetes.io/backend-type: "eni" can be used to add pods in Terway mode as the backend servers of an SLB instance. This improves network forwarding performance.
  • Services in Local mode (when externalTrafficPolicy=Local is set for the Services) can automatically set node weights based on the number of pods on each node.
No impact on workloads
April 2019
Version Image address Release date Description Impact
v1.9.3.105-gfd4e547-aliyun registry.cn-hangzhou.aliyuncs.com/acs/cloud-controller-manager-amd64:v1.9.3.105-gfd4e547-aliyun 2019-04-15
  • Multiple route tables can be created for a VPC. Configuration files can be used to set multiple route tables for a cluster.
  • The issue that updated HTTP configurations do not take effect is fixed.
No impact on workloads
March 2019
Version Image address Release date Description Impact
v1.9.3.81-gca19cd4-aliyun registry.cn-hangzhou.aliyuncs.com/acs/cloud-controller-manager-amd64:v1.9.3.81-gca19cd4-aliyun 2019-03-20
  • Existing SLB instances that are not created by ACK can be reused by ACK managed clusters and ACK dedicated clusters.
  • Custom node names are supported. Node naming is no longer reliant on the nodeName field in Kubernetes.
  • The compatibility issue between CCM 1.8.4 and Kubernetes 1.11.5 is fixed. We recommend that you upgrade the CCM to the latest version.
No impact on workloads
December 2018
Version Image address Release date Description Impact
v1.9.3.59-ge3bc999-aliyun registry.cn-hangzhou.aliyuncs.com/acs/cloud-controller-manager-amd64:v1.9.3.59-ge3bc999-aliyun 2018-12-26
  • An SLB instance can be shared by multiple Kubernetes Services.
    • If an SLB instance is created along with a Service, you cannot reuse this SLB instance when you create other Services. Otherwise, the SLB instance may be deleted. Only SLB instances that are manually created in the console or by calling the API can be used to expose multiple Services.
    • Services that share the same SLB instance cannot use the same frontend listening port. Otherwise, port conflicts may occur.
    • When you reuse an SLB instance, you must use the listener name and vServer group name as identifiers. Do not modify the listener name or vServer group name.
    • You can modify the SLB instance name.
    • SLB instances cannot be used to expose Services in different clusters.
  • VPC route tables are managed in sequence instead of in parallel. This prevents throttling.
No impact on workloads
August 2018
Version Image address Release date Description Impact
v1.9.3.10-gfb99107-aliyun registry.cn-hangzhou.aliyuncs.com/acs/cloud-controller-manager-amd64:v1.9.3.10-gfb99107-aliyun 2018-08-15
  • The annotation service.beta.kubernetes.io/alibaba-cloud-loadbalancer-master-zoneid can be used to specify the primary zone for an automatically created SLB instance.
  • The annotation service.beta.kubernetes.io/alibaba-cloud-loadbalancer-slave-zoneid can be used to specify secondary zones for an automatically created SLB instance.
    Note This parameter does not take effect in regions that do not support SLB instances that are deployed across the primary zone and secondary zones.
  • The annotation service.beta.kubernetes.io/alibaba-cloud-loadbalancer-force-override-listeners can be used to overwrite the existing listeners when you reuse an existing SLB instance.
  • The annotation service.beta.kubernetes.io/alibaba-cloud-loadbalancer-bandwidth can be used to specify the bandwidth when you create a pay-by-bandwidth SLB instance. The bandwidth is shared among listeners of the SLB instance.
No impact on workloads
June 2018
Version Image address Release date Description Impact
v1.9.3 registry.cn-hangzhou.aliyuncs.com/acs/cloud-controller-manager-amd64:v1.9.3 2018-06-25
  • The annotation service.beta.kubernetes.io/alibaba-cloud-loadbalancer-backend-label can be used to add worker nodes with specific labels as the backend servers of an SLB instance.
  • The annotation service.beta.kubernetes.io/alibaba-cloud-loadbalancer-spec can be used to specify the SLB instance type, such as shared-resource or high-performance.
  • The configuration of externalTrafic: Local can be used to set the Local mode for Services. If this mode is enabled, only nodes that host the pods are added as the backend servers of the related SLB instance.
  • If a node is added to or removed from a cluster, the node is automatically added to or removed from the backend servers of the related SLB instances.
  • When the labels of a node are changed, the node is automatically added to or removed from the backend servers of the related SLB instances.
  • Sticky sessions are supported.
  • Listeners are no longer managed by the system when you create a Service by using an existing SLB instance. You must manually add listeners.
No impact on workloads