Container Service for Kubernetes:Serverless elasticity in virtual nodes

What are virtual nodes

In ACK clusters, nodes are the basic units that provide computing and storage resources for running workloads. Most ACK clusters have at least one ECS node pool. When a pod is created, the kubelet schedules it to an ECS node in the node pool. This scheduling mode is ideal for applications with stable traffic. However, this mode struggles with traffic spikes because creating and starting ECS instances is time-consuming. Virtual nodes allow you to schedule pods directly on Alibaba Cloud Elastic Container Instance (ECI), which reduces the operational burden of node management and lowers costs by avoiding idle node resources.

A virtual node uses the ack-virtual-node component to encapsulate computing resources, which lets you deploy workloads without managing the underlying infrastructure. The component automatically schedules application pods to run on ECI. ECI is a serverless container service where each ECI instance is equivalent to a pod. When you use ECI to deploy containerized applications, you provide a container image and pay only for the resources that your containers consume.

Benefits

Virtual nodes provide the following benefits.

• Fully managed: You do not need to create underlying resource pools, which reduces the O&M workload. Virtual nodes are managed resources and do not require routine O&M operations for Kubernetes nodes, such as system upgrades or security patch installations.

• Large capacity: You can scale out to a maximum of 50,000 pods without capacity planning.

  Important

  If many pods are associated with services, we recommend that you keep the number of pods within 20,000.

• Elasticity in seconds: You can create thousands of pods in a short time. This prevents pod creation latency from affecting services during traffic spikes.

• Secure isolation: Pods are created based on ECI. Each container instance is strongly isolated from others using lightweight sandboxed container technology.

• Cost-effective: Applications are created on demand and billed on a pay-as-you-go basis. You are not charged for idle resources. The serverless architecture also reduces O&M costs.

Scenarios

Virtual nodes are suitable for the following scenarios.

• Online services

  For online services that experience frequent traffic spikes, such as online education and e-commerce, virtual nodes support scaling in seconds. This prevents system failures caused by slow scale-outs during traffic surges and avoids resource waste from idle resources.

• Data processing

  For processing many concurrent online data tasks, such as Spark and Presto tasks, the concurrency is no longer limited by the cost of underlying resources. You can quickly scale out to thousands of pods to meet the requirements of big data processing.

• AI tasks

  For AI tasks such as model training and model inference, which do not run continuously but require significant computing resources, you do not need to reserve resources. Instead, use resources on demand and pay by the second to reduce AI inference costs. Additionally, second-level elasticity lets you quickly respond to burst workloads.

• CI/CD staging environments

  For batch testing tasks in the CI/CD process, such as CI packaging, stress testing, and simulation testing, you can use virtual nodes to create and release container instances at any time. You can use resources on demand and are charged on a per-second basis. This approach provides large-scale resources at a low cost.

• Jobs and CronJobs

  Jobs and CronJobs do not need to run continuously. After a job is complete, it stops, and the corresponding pod is deleted. With virtual nodes, billing stops and computing resources are released automatically when the job is complete. This avoids resource waste from idle resources.