All Products
Search
Document Center

Container Service for Kubernetes:Scale out elastic container instances

Last Updated:Aug 31, 2023

Virtual nodes enable seamless integration between Kubernetes and Elastic Container Instance. Virtual nodes empower Kubernetes clusters with high elasticity. This way, Kubernetes clusters are no longer limited by the computing capacity of cluster nodes. You can create Elastic Container Instance-based pods as needed to avoid the planning of cluster capacity. This topic describes virtual nodes and elastic container instances. It also describes how to create Elastic Container Instance-based pods by using ack-virtual-node.

Prerequisites

Virtual nodes and elastic container instances

Elastic Container Instance is a serverless compute service that is provided by Alibaba Cloud for containerization. You can use elastic container instances to set up an O&M-free and isolated runtime environment for your containers. Elastic container instances allow you to focus on containerized applications without the need to purchase or manage Elastic Compute Service (ECS) instances. This way, you do not need to perform infrastructure maintenance. You can create elastic container instances to meet your business requirements. You are charged for resource usage on a per second basis.

Virtual nodes can drastically reduce computing costs and improve scaling efficiency. Virtual nodes are suitable in the following scenarios:

  • Online business that requires elastic scaling to handle traffic fluctuations, such as online education and e-commerce. Virtual nodes optimize the maintenance of resource pools. This helps you reduce computing costs.

  • Virtual nodes can reduce costs in computing scenarios where Spark or Presto is used to process data.

  • Continuous integration and continuous delivery (CI/CD) pipeline: Jenkins and GitLab Runner.

  • Jobs: Jobs in AI computing scenarios and CronJobs.

ACK provides multiple serverless container services based on virtual nodes and elastic container instances. These services include serverless Kubernetes (ASK) and ACK on Elastic Container Instance. You can use these services to deploy elastic and maintenance-free workloads. virtual node

Usage notes

Registered clusters support managed elastic container instances and unmanaged elastic container instances:

  • If you create managed elastic container instances in a registered cluster, the components that are used to manage the lifecycle of the Elastic Container Instance-based pods are managed by the ACK console. This reduces maintenance costs and saves cluster resources.

  • If you create unmanaged elastic container instances in a registered cluster, the components that are used to manage the lifecycle of the Elastic Container Instance-based pods are deployed in the cluster.

Important

If managed elastic container instances are created, skip the operations in Step 1 and proceed to Step 2: Install ack-virtual-node in the registered cluster to perform the related operations.

For more information about the supported regions and zones, see Regions and zones.

Step 1: Grant RAM permissions to ack-virtual-node

Use onectl

  1. Install onectl on your on-premises machine. For more information, see Use onectl to manage registered clusters.

  2. Run the following command to grant Resource Access Management (RAM) permissions to ack-virtual-node:

    onectl ram-user grant --addon ack-virtual-node

    Expected output:

    Ram policy ack-one-registered-cluster-policy-ack-virtual-node granted to ram user ack-one-user-ce313528c3 successfully.

Use the console

Before you can install the component in a registered cluster, you must specify an AccessKey pair to authenticate requests that are sent to cloud resources. Before you set the AccessKey pair, create a RAM user and grant the RAM user the permissions to access Alibaba Cloud resources.

  1. Create a RAM user. For more information, see Create a RAM user.

  2. Optional: Create a custom policy. For more information, see Create a custom policy.

    Show sample code

    {
        "Version": "1",
        "Statement": [
            {
                "Action": [
                    "eci:CreateContainerGroup",
                    "eci:DeleteContainerGroup",
                    "eci:DescribeContainerGroups",
                    "eci:DescribeContainerGroupStatus",
                    "eci:DescribeContainerGroupEvents",
                    "eci:DescribeContainerLog",
                    "eci:UpdateContainerGroup",
                    "eci:UpdateContainerGroupByTemplate",
                    "eci:CreateContainerGroupFromTemplate",
                    "eci:RestartContainerGroup",
                    "eci:ExportContainerGroupTemplate",
                    "eci:DescribeContainerGroupMetric",
                    "eci:DescribeMultiContainerGroupMetric",
                    "eci:ExecContainerCommand",
                    "eci:CreateImageCache",
                    "eci:DescribeImageCaches",
                    "eci:DeleteImageCache",
                    "vpc:DescribeVSwitches"
                ],
                "Resource": [
                    "*"
                ],
                "Effect": "Allow"
            }
        ]
    }
  3. Attach a policy to the RAM user. For more information, see Create a RAM user and grant permissions to the RAM user.

    You can create a custom policy or select the AliyunECIFullAccess policy to grant the permissions to the RAM user.

  4. Create an AccessKey pair for the RAM user. For more information, see Obtain an AccessKey pair.

  5. Use the AccessKey pair to create a Secret named alibaba-addon-secret in the registered cluster.

    The system automatically uses the AccessKey pair to access cloud resources when you install ack-virtual-node.

    kubectl -n kube-system create secret generic alibaba-addon-secret --from-literal='access-key-id=<your access key id>' --from-literal='access-key-secret=<your access key secret>'
    Note

    Replace <your access key id> and <your access key secret> with the AccessKey pair that you obtained in the previous step.

Step 2: Install ack-virtual-node

Use onectl

Run the following command to install ack-virtual-node:

onectl addon install ack-virtual-node

Expected output:

Addon ack-virtual-node, version **** installed.

Use the console

  1. Log on to the ACK console. In the left-side navigation pane, click Clusters.

  2. On the Clusters page, click the name of the cluster that you want to manage and choose Operations > Add-ons in the left-side navigation pane.

  3. Click the Others tab, find ack-virtual-node and click Install.

    In the Note dialog box, click OK. The default vSwitch and security group of the cluster are used for elastic container instances that are deployed by ack-virtual-node. For more information, see What to do next.

Step 3: Create Elastic Container Instance-based pods

Before you create Elastic Container Instance-based pods, make sure that the virtual node is in the Ready state.

Run the following command to query the status of the virtual node:

kubectl  get no |grep virtual-kubelet

Expected output:

virtual-kubelet-cn-hangzhou-b   Ready    agent                  18d   v1.20.11-aliyun.1

The output indicates that the virtual node is in the Ready state.

You can use one of the following methods to create an Elastic Container Instance-based pod in an ACK managed cluster:

Method 1: Add labels to pods.

  • Add the alibabacloud.com/eci=true label to the pod that you want to create. Then, an Elastic Container Instance-based pod is created and scheduled to the virtual node. Example:

    1. Run the following command to add a label to the pod:

      kubectl run nginx --image nginx -l alibabacloud.com/eci=true
    2. Run the following command to query the pod:

      kubectl get pod -o wide|grep virtual-kubelet

      Expected output:

      nginx-7fc9f746b6-r4xgx     0/1     ContainerCreating   0          20s   192.168.XX.XX   virtual-kubelet        <none>           <none>

Method 2: Add labels to namespaces.

  • Add the alibabacloud.com/eci=true label to the namespace to which the pod belongs. Then, an Elastic Container Instance-based pod is created and scheduled to the virtual node. Example:

    1. Run the following command to create a virtual node:

      kubectl create ns vk
    2. Run the following command to add a label to the namespace to which the pod belongs:

      kubectl label namespace vk alibabacloud.com/eci=true
    3. Run the following command to schedule the pod to the virtual node:

      kubectl -n vk run nginx --image nginx
    4. Run the following command to query the pod:

      kubectl -n vk get pod -o wide|grep virtual-kubelet

      Expected output:

      nginx-6f489b847d-vgj4d      1/1     Running             0          1m   192.168.XX.XX   virtual-kubelet        <none>           <none>

What to do next

Modify the configurations of the virtual node

The configurations of the virtual node controller determine how Elastic Container Instance-based pods are scheduled to a virtual node and specify the runtime environment of the pod, such as vSwitches and security group settings. You can modify the configurations of the virtual node controller to meet your business requirements. Modified configurations apply only to pods that are scheduled after modifications and do not apply to existing pods that run on the node.

Run the following command to modify the configurations of the virtual node controller:

kubectl -n kube-system edit deployment ack-virtual-node-controller

The following list describes some operations that are commonly performed to modify the configurations of the virtual node controller:

  • Upgrade the version of the virtual node controller

    To use the latest features of virtual nodes, you must upgrade the virtual node controller to the latest version.

  • Modify security group settings (ECI_SECURITY_GROUP)

    You can modify the ECI_SECURITY_GROUP environment variable to change the security group of the pods that are scheduled to the virtual node.

  • Modify vSwitch settings (ECI_VSWITCH)

    You can modify the ECI_VSWITCH environment variable to change the vSwitch of the pods that are scheduled to the virtual node. We recommend that you configure multiple vSwitches that are deployed in different zones to ensure high availability. When elastic container instances in the current zone are out of stock, the virtual node controller creates pods in another zone.

  • Modify kube-proxy settings (ECI_KUBE_PROXY).

    By default, the ECI_KUBE_PROXY environment variable is set to true. This indicates that pods can access ClusterIP Services. If the pods no longer need to access ClusterIP Services, you can set the environment variable to false to disable kube-proxy. In large-scale deployment scenarios, a cluster may need to start a large number of pods. This significantly increases the number of concurrent connections between kube-proxy and the Kubernetes API server. In this case, you can disable kube-proxy to reduce the heavy loads on the API server.

  • Modify the kube-system/eci-profile ConfigMap.

    You can modify the kube-system/eci-profile ConfigMap to specify more parameters for elastic container instances, such as vSwitches and security groups.

Delete a virtual node

  1. After you delete all the pods in a registered cluster, you can uninstall ack-virtual-node.

    Use onectl

    Run the following command to uninstall ack-virtual-node:

    onectl addon uninstall ack-virtual-node

    Expected output:

    Addon ack-virtual-node uninstalled.

    Use the console

    Go to the Add-ons page in the ACK console and uninstall ack-virtual-node.

  2. Run the kubectl delete no command to delete the related virtual node.

    Note

    If you do not delete the Elastic Container Instance-based pods in the cluster before you uninstall ack-virtual-node, the elastic container instances are retained in the cluster.