With the popularization of containers and the serverless concept, the Alibaba Cloud Container Service team and the Elastic Compute Service team worked collaboratively and released multiple Serverless container products and solutions, including Serverless Kubernetes, Elastic Container Instance (ECI), and the virtual-nodes add-on feature. These products support a variety of serverless container application scenarios.
Serverless Kubernetes, released by the Alibaba Cloud Container Service team, has been in the open beta test for a while now. Users don't need to manage servers and can normally perform various Kubernetes API operations, including creating pods, services, ingresses, and jobs. The biggest advantage of Serverless is that it not only avoids the server maintenance burden but also provides applications with powerful elastic scaling. Elasticity is no longer limited to the node scale of clusters. In addition, application billing is based on the resource usage.
We released the virtual-nodes feature to further optimize the Container Service user experience, support more application scenarios and provide extremely powerful elastic scaling for users' Kubernetes clusters.
As shown in the preceding diagram, this virtual node feature adds virtual nodes to classic Kubernetes clusters, allowing the real worker nodes and virtual nodes to exist in these clusters at the same time. Virtual nodes are used to dynamically create ECI container instances. Pods on ECI and pods on the real nodes are interconnected. They run on the same user's VPC network.
Virtual-nodes allows users to obtain "unlimited" elasticity in the original Kubernetes clusters without having to create new clusters. Users can also have "unlimited" computing capacity without creating a large number of real worker nodes. The typical application scenarios of virtual nodes include big data computing, gene computing, video rendering, and hybrid clouds. Virtual nodes are also suitable for temporary scaling of online applications.
With virtual nodes, we can implement "unlimited" scaling based on Kubernetes clusters with limited nodes.
How do we use virtual nodes? Let's first understand its implementation principle.
Elastic Container Instance (ECI) is an agile and secure Serverless container runtime service recently released by the Alibaba Cloud ECS team. ECI has the following features:
Unlike Serverless Kubernetes Container Service, ECI doesn't provide the native Kubernetes API. For example, Kubectl cannot be used to perform operations on pod/service/ingress/job resources. However, the innovative connection between Kubernetes and ECI by using virtual kubelet enables ECI instances to be created dynamically by using the Kubernetes API.
Virtual Kubelet is an open source project originally launched by Microsoft Azure with the aim to provide better product integration on public clouds (like elastic containers) with Kubernetes and implement the serverless feature. From the implementation perspective, Virtual Kubelet provides a mechanism that allows integrating products from multiple different providers. Currently, integration with Azure ACI and AWS Fargate is supported. VK is also applied in IoT Edge scenarios. Recently we submitted the ECI provider, which has been merged on https://github.com/virtual-kubelet/virtual-kubelet/tree/master/providers/alicloud
Virtual Kubelet registers a virtual node with Kubernetes APIServer, continuously listens to pod change events and dynamically creates ECI instances.
Virtual Kubelet innovatively connects Kubernetes and ECI by using virtual nodes. Virtual Kubelet and ECI are the technology infrastructure of virtual nodes.
The following section describes how to deploy virtual nodes in Alibaba Cloud Container Service.
Visit the ECI console and enable ECI.
Go to the Container Service console and create a Managed Kubernetes cluster:
After a cluster is created, you can view vswitchid and securitygroup in the cluster management or node page.
Replace the ECI_VSWITCH, ECI_SECURITY_GROUP, ECI_ACCESS_KEY, and ECI_SECRET_KEY variables in the alicloud-virtual-kubelet.yaml file.
# cat alicloud-virtual-nodes-eci.yaml apiVersion: v1 kind: ServiceAccount metadata: name: alicloud-virtual-nodes-eci namespace: kube-system --- apiVersion: rbac.authorization.k8s.io/v1beta1 kind: ClusterRoleBinding metadata: name: alicloud-virtual-nodes-eci namespace: kube-system roleRef: apiGroup: rbac.authorization.k8s.io kind: ClusterRole name: cluster-admin subjects: - kind: ServiceAccount name: alicloud-virtual-nodes-eci namespace: kube-system --- apiVersion: apps/v1beta2 kind: Deployment metadata: name: alicloud-virtual-nodes-eci namespace: kube-system labels: app: alicloud-virtual-nodes-eci spec: replicas: 1 selector: matchLabels: app: alicloud-virtual-nodes-eci template: metadata: labels: app: alicloud-virtual-nodes-eci spec: serviceAccount: alicloud-virtual-nodes-eci containers: - name: alicloud-virtual-kubelet image: registry.cn-hangzhou.aliyuncs.com/ask/virtual-nodes-eci:v22.214.171.124-aliyun imagePullPolicy: Always args: ["--provider", "alibabacloud"] env: - name: KUBELET_PORT value: "10250" - name: VKUBELET_POD_IP valueFrom: fieldRef: fieldPath: status.podIP - name: VKUBELET_TAINT_KEY value: "virtual-kubelet.io/provider" - name: VKUBELET_TAINT_VALUE value: "alibabacloud" - name: VKUBELET_TAINT_EFFECT value: "NoSchedule" - name: ECI_REGION value: "cn-hangzhou" - name: ECI_VSWITCH value: "xxx" - name: ECI_SECURITY_GROUP value: "xxx" - name: ECI_ACCESS_KEY value: "xxx" - name: ECI_SECRET_KEY value: "xxx"
Execute the yaml file and view the pod and node status. The result shows that a new virtual node has been added to the cluster.
# kubectl apply -f ./alicloud-virtual-nodes-eci.yaml serviceaccount "alicloud-virtual-nodes-eci" created clusterrolebinding "alicloud-virtual-nodes-eci" created deployment "alicloud-virtual-nodes-eci" created # kubectl -n kube-system get pod|grep virtual-nodes alicloud-virtual-nodes-eci-54b748489c-gkcmn 1/1 Running 0 53s # kubectl get node NAME STATUS ROLES AGE VERSION cn-hangzhou.i-bp19cyop0b1skne04djt Ready <none> 10m v1.11.2 cn-hangzhou.i-bp19cyop0b1skne04dju Ready <none> 10m v1.11.2 cn-hangzhou.i-bp19cyop0b1skne04djv Ready <none> 10m v1.11.2 virtual-kubelet Ready agent 55s v1.11.2
In the console, we can also see the presence of this virtual-kubelet virtual node.
Now that a virtual node is present, we need to set nodeSelector and tolerations for the pod to be run and schedule the pod onto the virtual node "virtual-kubelet".
# cat nginx.yaml apiVersion: v1 kind: Pod metadata: name: nginx spec: containers: - image: nginx imagePullPolicy: Always name: nginx nodeSelector: kubernetes.io/role: agent beta.kubernetes.io/os: linux type: virtual-kubelet tolerations: - key: virtual-kubelet.io/provider operator: Exists # kubectl get pod NAME READY STATUS RESTARTS AGE nginx 1/1 Running 0 1m
We can see that the nginx pod is scheduled onto the virtual node and is not running on any worker nodes in the cluster. By using virtual nodes, we implement the serverless deployment of container applications and elastic scaling. Users no longer need to worry about selecting a node scale for clusters or making capacity plans.
Users can follow these steps to quickly deploy virtual nodes. However, there is some complexity because users need to configure AK and vswitch/security groups for clusters. In the future, we will add an Alibaba Cloud Container Service console plug-in for the one-click deployment of virtual nodes. This plug-in will save our users from configuring virtual nodes and provide k8s clusters with powerful elastic scaling. Please stay updated on the latest Alibaba Cloud Container Service news.
Alibaba Clouder - June 9, 2020
Alibaba Cloud Native - June 9, 2022
Alibaba Clouder - July 15, 2020
Alibaba Container Service - July 16, 2019
Alibaba Container Service - April 28, 2020
PM - C2C_Yuan - October 12, 2022
Alibaba Cloud Container Service for Kubernetes is a fully managed cloud container management service that supports native Kubernetes and integrates with other Alibaba Cloud products.Learn More
Provides a control plane to allow users to manage Kubernetes clusters that run based on different infrastructure resourcesLearn More
Accelerate software development and delivery by integrating DevOps with the cloudLearn More
Accelerate and secure the development, deployment, and management of containerized applications cost-effectively.Learn More
More Posts by Alibaba Container Service