All Products
Search
Document Center

Use Elastic Container Instance in self-managed Kubernetes clusters

Last Updated: Jun 01, 2021

If you have self-managed Kubernetes clusters in Elastic Computer Service (ECS), virtual nodes (Virtual Kubelet) must be deployed in the clusters before you can use Elastic Container Instance in the clusters. This topic describes how to connect self-managed clusters in ECS to Elastic Container Instance, including how to register self-managed clusters in the Container Service for Kubernetes (ACK) console and deploy virtual nodes, or deploy Virtual Kubelet in self-managed clusters.

Background information

Elastic Container Instance is seamlessly connected to Kubernetes by using virtual nodes based on Virtual Kubelet provided by the Kubernetes community. Kubernetes clusters can obtain high elasticity without being limited by the computing capacities of cluster nodes. If you have self-managed Kubernetes clusters in ECS, you can use one of the following methods to use Elastic Container Instance:

Note

We recommend that you use Method 1. This method allows you to upgrade the Virtual Kubelet version (the ack-virtual-node version) to use new features.

Preparations

The following table describes the parameters that you must obtain before you can deploy Virtual Kubelet.

Parameter

Description

Obtainment method

ECI_ACCESS_KEY and ECI_SECRET_KEY

The AccessKey ID and corresponding AccessKey secret. They serve as the credentials to manage Elastic Container Instance in virtual nodes.

For more information, see Obtain an AccessKey pair.

ALIYUN_CLUSTERID

The ID of the cluster, which is the unique identifier of the cluster.

If you register a cluster, the cluster ID is generated by the system. If you deploy Virtual Kubelet in a cluster, you must customize the cluster ID.

ECI_REGION

The ID of the region. The elastic container instance is deployed within this region.

You can query supported regions by using the Elastic Container Instance console or by calling the DescribeRegions operation.

ECI_VPC

The ID of the virtual private cloud (VPC). The elastic container instance is deployed within this VPC.

You can create and view VPCs on the VPCs page in the VPC console.

ECI_VSWITCH

The ID of the vSwitch. The elastic container instance is associate with this vSwitch.

You can create and view vSwitches on the vSwitch page in the VPC console and select the corresponding vSwitch based on the selected VPC.

ECI_SECURITY_GROUP

The ID of the security group. The elastic container instance is added to this security group.

You can create and view security groups on the Security Groups page in the ECS console and select the corresponding vSwitch based on the selected VPC.

Register self-managed clusters and deploy virtual nodes

You can register self-managed clusters in the ACK console and then deploy virtual nodes to use Elastic Container Instance. Perform the following steps:

  1. Log on to the ACK console.

  2. In the left-side navigation pane, click Clusters.

  3. Create a registered cluster.

    1. On the Clusters page, click Create Kubernetes Cluster.

    2. Click the Register Cluster tab. Specify the parameters for the cluster and click Create Cluster.

      The following table describes the parameters of which you must take note. For more information, see Register an external Kubernetes cluster.

      Parameter

      Description

      Region, VPC, and VSwitch

      Select the required region, VPC, and vSwitch.

      Access to API Server

      By default, an internal-facing Server Load Balancer (SLB) instance is created for the API server. You can select the specifications of the SLB instance to suit your needs.

      EIP

      Specify whether to create and associate an elastic IP address (EIP) to connect to the cluster.

      Security Group

      Automatically create a security group to divide security domains and control network traffic.

      Log Service

      Specify whether to activate Log Service to collect log data from containers.

      Deletion Protection

      Specify whether to enable deletion protection for the cluster. If you enable deletion protection, the cluster cannot be deleted by using the ACK console or by calling API operations.

  4. Register the cluster.

    1. On the Clusters page, find the cluster that you created and click the cluster name.

    2. On the Cluster Information page, click the Connection Information tab.

    3. Create a ConfigMap in the cluster.

      Click the Public Network or Internal Network tab based on your network. Copy the content to a YAML configuration file such as agent.yaml and run the kubectl apply -f agent.yaml command in the cluster to create a ConfigMap.

    4. Run the following command in the cluster to check the connection status:

      kubectl -n kube-system get pod |grep ack-cluster-agent

      The following command output is returned:

      ack-cluster-agent-5f7d568f6-6fc4k              1/1     Running   0          9s
      ack-cluster-agent-5f7d568f6-tf6fp              1/1     Running   0          9s
  5. Deploy virtual nodes.

    1. In the left-side navigation pane, choose Marketplace > App Catalog.

    2. On the Alibaba Cloud Apps tab, find ack-virtual-node and click it.

    3. Set the parameters and select the created cluster for installation.

      On the Parameters tab, enter the vSwitch ID, security group ID, AccessKey ID, and AccessKey secret that you obtained.

      Note

      If you register the cluster in the Internet in Step 4, remove vpc from the value of the repository parameter. Example: registry.cn-hangzhou.aliyuncs.com/acs/virtual-nodes-eci.

      For more information, see Add a virtual node to an external cluster.

Deploy Virtual Kubelet

You can deploy Virtual Kubelet in self-managed clusters to use Elastic Container Instance. You must obtain the latest Virtual Kubelet version before you can deploy Virtual Kubelet. For more information, see ack-virtual-node.

Perform the following steps:

  1. Prepare the vk.yaml configuration file required to deploy Virtual Kubelet.

    The following code provides an example of the YAML configuration file content. You must replace the parameter values with your own information.

    apiVersion: v1
    kind: ServiceAccount
    metadata:
      name: virtual-node-sa
      namespace: kube-system
    ---
    apiVersion: rbac.authorization.k8s.io/v1beta1
    kind: ClusterRoleBinding
    metadata:
      name: virtual-node-role-binding
    roleRef:
      apiGroup: rbac.authorization.k8s.io
      kind: ClusterRole
      name: cluster-admin
    subjects:
      - kind: ServiceAccount
        name: virtual-node-sa
        namespace: kube-system
    ---
    apiVersion: apps/v1
    kind:  Deployment
    metadata:
      name: virtual-node-controller
      namespace: kube-system
      labels:
        app: virtual-node-controller
    spec:
      replicas: 1
      selector:
        matchLabels:
          app: virtual-node-controller
      template:
        metadata:
          labels:
            app: virtual-node-controller
        spec:
          serviceAccount: virtual-node-sa
          containers:
          - name: alicloud-virtual-kubelet
           # The image and its version required to deploy Virtual Kubelet.
            image: registry.cn-beijing.aliyuncs.com/acs/virtual-nodes-eci:v2.0.0.34-252556a33-aliyun
            imagePullPolicy: Always
            args: ["--provider", "alibabacloud"]
            env:
            - name: KUBELET_PORT
              value: "10250"
            - name: VKUBELET_POD_IP
              valueFrom:
                fieldRef:
                  fieldPath: status.podIP
            - name: VKUBELET_TAINT_KEY
              value: "virtual-kubelet.io/provider"
            - name: VKUBELET_TAINT_VALUE
              value: "alibabacloud"
            - name: VKUBELET_TAINT_EFFECT
              value: "NoSchedule"
            - name: ECI_REGION
              value: ${aliyun_region_name}
            - name: ECI_VPC
              value: ${aliyun_vpc_id}
            - name: ECI_VSWITCH
              value: ${aliyun_vswitch_id}
            - name: ECI_SECURITY_GROUP
              value: ${aliyun_sg_id}
            - name: ECI_ACCESS_KEY
              value: ${aliyun_access_key}
            - name: ECI_SECRET_KEY
              value: ${aliyun_secret_key}
            - name: ALIYUN_CLUSTERID
              value: ${custom_define_cluster_id} 
  2. Deploy Virtual Kubelet.

    kubectl apply -f vk.yaml
  3. View the deployment status.

    kubectl get deploy/virtual-node-controller -n kube-system

    The following code provides an example of the command output if Virtual Kubelet is deployed:

    NAME                              READY   UP-TO-DATE   AVAILABLE   AGE
    virtual-node-controller   1/1         1                      1                  161m
  4. View the node information after Virtual Kubelet is deployed.

    kubectl get node -o wide

    After Virtual Kubelet is deployed, the generated virtual node is named virtual-kubelet. If the virtual-kubelet node is displayed in the Ready state, Virtual Kubelet is deployed. Example:

    NAME                  STATUS   ROLES    AGE   VERSION   INTERNAL-IP    EXTERNAL-IP   OS-IMAGE                     KERNEL-VERSION                CONTAINER-RUNTIME
    k8s-master01      Ready    <none>   50d    v1.14.2      192.168.*.*     <none>           CentOS Linux 7 (Core)   3.10.0-957.21.3.el7.x86_64   docker://18.9.6
    k8s-master02      Ready    <none>   50d    v1.14.2      192.168.*.*     <none>           CentOS Linux 7 (Core)   3.10.0-957.21.3.el7.x86_64   docker://18.9.6
    k8s-master03      Ready    <none>   50d    v1.14.2      192.168.*.*     <none>           CentOS Linux 7 (Core)   3.10.0-957.21.3.el7.x86_64   docker://18.9.6
    virtual-kubelet    Ready     agent      82m   v1.11.2      172.30.*.*     <none>           <unknown>                  <unknown>                         <unknown>
Note

If you want to use new features, you may need to upgrade the Virtual Kubelet version. You can run the kubectl edit deployment -n kube-system virtual-node-controller command to edit resources and modify the image tag to the desired version.

Schedule pods to the virtual nodes

When a cluster contains virtual nodes, you can schedule pods to the virtual nodes to use Elastic Container Instance to run the pods. Perform the following steps:

  1. Prepare the test-pod.yaml configuration file required to create pods.

    Example of the YAML configuration file:

    apiVersion: v1
    kind: Pod
    metadata:
      name: test-pod
    spec:
      containers:
      - name: centos
        image: centos:latest
        args:
        - /bin/sh
        - -c
        - date; sleep 6000
      nodeName: virtual-kubelet  #Specifies the virtual node on which the pod runs.
  2. Create a pod.

    kubectl create -f test-pod.yaml
  3. Check the running status of the pod.

    • View the basic information of the pod.

      kubectl get pod/test-pod -o wide

      The following example command output indicates that the pod runs on the virtual-kubelet virtual node.

      NAME       READY   STATUS    RESTARTS   AGE   IP                    NODE                  NOMINATED NODE   READINESS GATES
      test-pod   1/1         Running   0                 95s   192.168.*.*   virtual-kubelet   <none>                      <none>
    • View the YAML configuration file of the pod.

      kubectl  get pod -o yaml test-pod

      The following example command output indicates that Virtual Kubelet has added an annotation for the pod that runs on the virtual node. The elastic container instance ID is specified by the k8s.aliyun.com/eci-instance-id parameter.

      apiVersion: v1
      kind: Pod
      metadata:
        annotations:
          ProviderCreate: done
          k8s.aliyun.com/eci-instance-id: eci-2ze428yrni34mggb****
    • View the elastic container instance in the Elastic Container Instance console.

      After the pod is created, you can view the corresponding elastic container instance in the Elastic Container Instance console. The elastic container instance ID is the value of the k8s.aliyun.com/eci-instance-id parameter in the YAML configuration file. The elastic container instance name is concatenated by using the namespace and name of the pod.

      Self-managed Kubernetes cluster