How ECI expands the capacity of 3000 container instances in 6 seconds

Introduction

According to the latest CNCF report, over 90% of users use containers in production environments, and over 80% of users manage containers through Kubernetes. Is it possible that our production environment has been equipped with K8s, which perfectly solves the problem of application deployment? There is a saying in the IT industry that nothing is omnipotent, and K8s is not omnipotent either. K8s solves application orchestration and scheduling, but does not address resource capacity limitations, container security isolation, and high operation and maintenance costs.

The Problems and Dilemmas of Traditional K8s

Low resource efficiency

This resource bar chart comes from a customer of Alibaba Cloud. The CPU usage during peak traffic periods is approximately 7000 to 8000 cores, while during low traffic periods, only a few hundred cores are used. If you plan resources based on peak traffic when doing IT planning or purchasing resources, it can lead to serious resource waste. However, if ECS resources are planned in real-time based on usage, it will result in inability to expand in a timely manner in the face of sudden traffic, affecting the stability of the service.

Weak resource isolation

The container uses the namespace of the system kernel for resource isolation, but the kernel only supports six types of namespace isolation, such as UTS and IPS. We have encountered a client who needs to modify the time of a business Docker in a testing environment, resulting in all container times on one machine being modified. There are also scenarios such as customizing kernel parameters and fair IO sharing, which have the same issues.

At the same time, container security has also been criticized by everyone, for example, privileged containers can directly see all disk data on the machine.

High operation and maintenance costs

Cloud native has brought a lot of convenience to IT, but at the same time, cloud native has also made the entire IT operation and maintenance increasingly complex. A K8S container cluster requires at least the deployment of highly available masters, network plugins, image warehouses, log services, and monitoring components. Even if you work hard to install these components, you still have to face various daily operations and alarm processing, which involves firefighting every day.

Alibaba Cloud Elastic Container Instance ECI emerged as the times require

Is there a secure container solution that is maintenance free and can be used on demand? Alibaba Cloud elastic container instances have emerged.

Alibaba Cloud Elastic Container Instance (ECI) is a container running service provided by Alibaba Cloud that combines container and serverless technology. By using ECI, when deploying containers on Alibaba Cloud, there is no need to purchase and manage cloud server ECS, and Pod and containers can be directly run on Alibaba Cloud, saving the maintenance and management work of the underlying servers. Simply put, an ECI is a Pod that can be orchestrated and scheduled by K8s.

Alibaba Cloud elastic container instances are particularly suitable for sudden business traffic or short cycle task operations. So what is the difference between ECI and customers purchasing ECS themselves and running Docker in ECS? The biggest difference is that if ECI is used, the runtime of the entire container will be operated and maintained by Alibaba Cloud.

ECI has the following advantages:

The underlying resources are hosted by Alibaba Cloud, and users no longer need to manage the underlying VM (virtual machine).

Reuse the entire Alibaba Cloud elastic computing resource pool to ensure sufficient inventory.

Low cost, billed per second, and charged when creating a Pod.

Fast startup, second level startup of the underlying security sandbox.

Strong compatibility, fully compatible with K8s.

The Alibaba Cloud elastic container instance adopts the community's Virtual Kubelet solution to integrate with K8s. When a Pod is created and scheduled to Virtual Kubelet in the cluster, Kubelet will call the ECI interface and start ECI.

The integration methods between ECI and business systems include:

(Recommended) Deploy business through Alibaba Cloud container service Serverless Kubernetes (ASK), providing Kubernetes cluster capability without maintenance, and all underlying Pod resources are hosted using ECI.

(Recommended) Deploy business through Alibaba Cloud container service Kubernetes (ACK) to provide additional massive resilience capabilities for ACK clusters.

Connect with the Kubernetes cluster built by users on ECS through Virtual Node, providing convenient and fast elastic computing resources.

Connect users to the Kubernetes cluster built by IDC offline through Virtual Node, providing unlimited elastic computing capability on the cloud.

Directly connect to business systems through OpenAPI, creating or releasing ECI business containers at any time at low cost.

ECI Fast Start Instance: 3000 container instances in 6 seconds

At the 2021 Yunqi Conference, Alibaba Cloud Serverless container service elastic container instances released new features for fast start instances. On the basis of solving the application deployment issues mentioned above, the elastic container instance innovatively provides product features with fast start. The on-site demonstration successfully launched 3000 ECIs within 6 seconds and all entered the Running state.

04:5

How does Alibaba Cloud start 3000 container instances in 6 seconds?

On the one hand, through a large amount of user level creation history data, machine learning is applied to identify the patterns of user creation of Pods. By predicting pre scheduling, resource reuse, and other means, the scheduling and creation time of ECI is saved. At the same time, Alibaba Cloud's kangaroo sandbox container is used as the engine, supplemented by overlay network and storage solutions, to compress the cold start time of a single ECI instance to less than 3 seconds, There will be a special article in the future that will provide a detailed introduction to the kangaroo engine, and please look forward to it.

On the other hand, in the image retrieval dimension, container images are made into snapshots through image caching, eliminating the need for Pod to pull container images every time it is started. For example, some images in Alibaba Cloud's Da Mo Yuan AI team can reach hundreds of gigabytes. If traditional retrieval takes more than ten minutes, using ECI's image caching solution can achieve Pod second level startup.

Looking Forward to the Future

The Alibaba Cloud Elastic Container instance provides free operation and full hosting services for runtime, GuestOS, underlying computing, network, and storage resources. It also announced the extremely fast instance startup speed at the 2021 Yunqi Conference, helping customers quickly expand and shrink their business systems.

With the further upward movement of cloud vendor service boundaries, ECI expects to provide better elasticity, performance, and cost capabilities compared to customer built container resource pools through large-scale and intensive resource scheduling and end-to-end runtime design. This will be the direction for continuous exploration of Alibaba Cloud elastic container instances in the next 1-2 years.