Elastic Container Instance:Connect a self-managed Kubernetes cluster to a VNode

Prerequisites

• A Kubernetes cluster whose version is between 1.13 and 1.22 is deployed.

• If your Kubernetes cluster is deployed in a data center, make sure that the data center and the virtual private cloud (VPC) are connected with each other by using Express Connect, Smart Access Gateway, or VPN Gateway. For more information, see the following topics:

  • What is Express Connect?

  • What is SAG?

  • What is VPN Gateway?

Preparations

Before you connect a self-managed Kubernetes cluster to a VNode, understand the permissions required by the VNnode and prepare the following parameter information required to create the VNode.

To deploy a VNode, you need to configure standard permissions for native Kubernetes nodes and additional permissions for the VNode, as shown in the following table.

Step 1: Generate a KubeConfig certificate

To connect VNodes to Kubernetes cluster, a KubeConfig certificate is required. Therefore, before you create a VNode, you need to issue a KubeConfig certificate to the VNode. Perform the following operations:

1. Download and execute a configuration script to generate a KubeConfig certificate.

   In TLS Bootstrap or ServiceAcoount mode, run the appropriate command to generate a KubeConfig certificate based on the version of your Kubernetes cluster.

   • TLS Bootstrap

     If you have security requirements, we recommend that you use the TLS Bootstrap mode. In this mode, the vnode-approver component is automatically deployed. The vnode-approver component is used to issue certificate signing requests (CSRs) that are sent by Vnodes. vnode-approver has open source code on GitHub. For more information, see vnode-approver.

     • If the cluster version is between 1.14 and 1.22, run the following command to generate a KubeConfig certificate:

       curl -fsSL https://eci-release.oss-cn-beijing.aliyuncs.com/vnode-deploy.sh | bash -s -- bootstrap

     • If the cluster version is 1.13, run the following command to generate a KubeConfig certificate:

       For clusters of v1.13, the endpoint resource needs to select a leader. You must configure additional permissions by setting EXTRA_PRIVILEDGE=true in the script.

       curl -fsSL https://eci-release.oss-cn-beijing.aliyuncs.com/vnode-deploy.sh | env EXTRA_PRIVILEDGE=true bash -s -- bootstrap

   • ServiceAcoount

     This mode does not support certificate rotation. When you issue a KubeConfig certificate, ensure that the token has a long enough or permanent validity period to prevent the VNode from failing due to expiration of the KubeConfig.

     • If the cluster version is between 1.14 and 1.22, run the following command to generate a KubeConfig certificate:

       curl -fsSL https://eci-release.oss-cn-beijing.aliyuncs.com/vnode-deploy.sh | bash -s -- common

     • If the cluster version is 1.13, run the following command to generate a KubeConfig certificate:

       For clusters of v1.13, the endpoint resource needs to select a leader. You must configure additional permissions by setting EXTRA_PRIVILEDGE=true in the script.

       curl -fsSL https://eci-release.oss-cn-beijing.aliyuncs.com/vnode-deploy.sh | env EXTRA_PRIVILEDGE=true bash -s -- common

2. View KubeConfig.

   The KubeConfig is named vnode-config. Run the following command to view KubeConfig:

   cat vnode-config

   When you view KubeConfig, take note of the following items:

   • Make sure that the VPC in which the VNode is to be created can be connected to the server. To ensure the VPC can be connected to the server, the VNode and the server must reside in the same VPC, or the VPC and the server can communicate with each other.

     • If the server address is an IP address, make sure that you can access the IP address from the VPC in which the VNode resides.

     • If the server address is a domain name, make sure that the domain name can be resolved in an Alibaba Cloud VPC. In addition, from the VPC in which the VNode resides, you must be able to access the IP address that is obtained after the domain name is resolved.

   • If the content of certificate-authority-data is empty, check whether the PATH field is used in ~/.kube/config.

   The following code provides a sample content of the KubeConfig.

   • TLS Bootstrap

     apiVersion: v1
     kind: Config
     current-context: kubernetes-admin@kubernetes
     contexts:
     - name: kubernetes-admin@kubernetes
       context:
         cluster: kubernetes-admin@kubernetes
         user: vnode
         namespace: kube-system
     clusters:
     - name: kubernetes-admin@kubernetes
       cluster:
         certificate-authority-data: "*****************************************=="
         server: https://10.16.XX.XX:6443
     users:
     - name: vnode
       user:
         token: ******.****************

   • ServiceAcoount

     apiVersion: v1
     kind: Config
     current-context: kubernetes-admin@kubernetes
     contexts:
     - name: kubernetes-admin@kubernetes
       context:
         cluster: kubernetes-admin@kubernetes
         user: vnode
         namespace: kube-system
     clusters:
     - name: kubernetes-admin@kubernetes
       cluster:
         certificate-authority-data: "*****************************************=="
         server: https://10.16.XX.XX:6443
     users:
     - name: vnode
       user:
         token: ***********************************************

Step 2: Create a VNode

You can use the Elastic Container Instance console or call an API operation to create a VNode.

Step 3: Configure an anti-affinity policy

DaemonSets cannot run on VNodes because VNodes are not real nodes. After you create a VNode, you need to modify the DaemonSet in kube-proxy and configure nodeAffinity to prevent DaemonSets from being scheduled to the VNode. Perform the following operations:

1. Run the following command to modify the configurations of the DaemonSet:

   kubectl -n kube-system edit ds kube-proxy

2. Configure nodeAffinity.

   Add the following YAML content to spec > template > spec:

   affinity:
     nodeAffinity:
       requiredDuringSchedulingIgnoredDuringExecution:
         nodeSelectorTerms:
         - matchExpressions:
           - key: type
             operator: NotIn
             values:
             - virtual-kubelet

What to do next

After you create the VNode, you can use one of the following methods to schedule pods to the VNode. Then, you can run the pods on the elastic container instances that are deployed in the VNode.

• Manual scheduling

  You can configure the nodeSelector and tolerations parameters or specify the nodeName parameter to schedule pods to the VNode. For more information, see Schedule pods to a VNode.

• Automatic Scheduling

  The ECI Profile feature allows you to customize selectors. Then, Elastic Container Instance automatically schedules pods that meet the conditions of the selectors to the VNode. For more information about specific operations, see Configure an ECI Profile to orchestrate pods.