Elastic Container Instance:Deploy VNode in a self-managed Kubernetes cluster with VNodectl

Limitations

• VNodes are not real nodes. DaemonSets cannot run on VNodes. After creating a VNode, configure nodeAffinity on kube-proxy to prevent DaemonSet pods from being scheduled to it. See Prevent DaemonSets from scheduling to the VNode.

• The ServiceAccount authentication mode does not support certificate rotation. If you use this mode, make sure the token has a long enough validity period to prevent VNode failure.

Prerequisites

Before you begin, make sure you have the following:

• A Kubernetes cluster version 1.16 to 1.30.

• If your cluster runs in a data center: network connectivity between the data center and the virtual private cloud (VPC) where Elastic Container Instance-based pods will run. Supported options: Express Connect, Smart Access Gateway (SAG), or VPN Gateway.

• The following resource IDs and credentials:

  Resource	How to get it
  Region ID	Check Regions and zones for available regions, or query via the Elastic Container Instance console or the DescribeRegions API.
  VPC ID	Create or view VPCs on the VPCs page in the VPC console.
  vSwitch ID	Create or view vSwitches on the vSwitch page in the VPC console. Select a vSwitch in the same VPC.
  Security group ID	Create or view security groups on the Security Groups page in the ECS console. Select a security group in the same VPC.
  AccessKey ID and AccessKey secret	See Create an AccessKey pair.

Use an advanced security group and add the following inbound rules:

All ports 1–65535 from the VPC CIDR blocks

Ports 10250 and 10255 from the Kubernetes API server endpoint

The following permissions are required to create a VNode:

Install VNodectl

Install VNodectl on the master node of your Kubernetes cluster.

Download the package for your operating system and CPU architecture:

The following steps use Linux AMD64 as an example:

1. Connect to your Kubernetes cluster.

2. Download the VNodectl package.

   wget https://eci-docs.oss-cn-beijing.aliyuncs.com/vnode/vnodectl_0.0.5-beta_linux_amd64.tar.gz -O vnodectl.tar.gz

3. Extract the package.

   tar xvf vnodectl.tar.gz

4. Move the binary to a directory in your PATH.

   cp vnodectl /usr/local/bin/vnode

Add --dry to any vnode command to run a client-side validation without applying changes. If an API version compatibility error occurs, add --kubernetes-version to specify your cluster's Kubernetes version.

Configure the cluster

Before creating a VNode, generate a kubeconfig certificate for it. VNodectl supports two authentication modes:

Configure the ~/.vnode/config file

~/.vnode/config stores the configuration VNodectl reads at runtime. Configure it before creating a VNode.

1. Generate the default configuration template.

   Parameter	Required	Description
   name	Yes	Context name
   vnode-name	No	VNode name. Leave blank to let the system generate a unique name.
   region-id	Yes	Region ID where the VNode runs. See the Prerequisites section for details.
   access-key-id	Yes	Your AccessKey ID for API authentication. See Create an AccessKey pair.
   access-key-secret	Yes
   vswitch-id	Yes	vSwitch ID for the VNode. Specify multiple IDs to enable automatic vSwitch selection.
   security-group-id	Yes	Security group ID for the VNode.
   resource-group-id	No	Resource group for the VNode. Defaults to the account's default resource group. See What is Resource Management?
   kubeconfig	Yes	Path to the kubeconfig generated in the previous step. Default: ~/.vnode/deploy/kubeconfig
   taints	No	Taints to bind to the VNode.
   tags	No	Tags for split billing. After binding a custom tag, filter charges by tag in the fee analysis view. See View split bills based on tags.
   network-type	No	Network type for the Elastic Container Instance control service. Leave blank for internet access; set to vpc for VPC access.

   vnode config default

   The following table describes the parameters in the contexts section.

2. Write the template to ~/.vnode/config and edit it.

   vnode config default > ~/.vnode/config
   vim ~/.vnode/config

   The following example shows a minimal configuration with required parameters only:

   kind: vnode
   contexts:
       - name: default
         region-id: cn-guangzhou
         access-key-id: <yourAccessKeyID>
         access-key-secret: <yourAccessKeySecret>
         vswitch-id: vsw-7xv2yk45qp5etidgf****
         security-group-id: sg-7xv5tcch4kjdr65t****
         kubeconfig: ~/.vnode/deploy/kubeconfig
   current-context: default

3. Verify that VNodectl reads the configuration correctly.

   vnode config

4. If you have multiple contexts, switch to the one you want to use.

   vnode config set-context <context-name>

Create a VNode

1. Create the VNode.

   vnode create

   The VNode is created based on ~/.vnode/config. To configure customResources, clusterDNS, or clusterDomain, see Manage VNodes. The expected output includes the VNode ID:

   {"RequestId":"AB772F9D-2FEF-5BFD-AAFB-DA3444851F29","VirtualNodeId":"vnd-7xvetkyase7gb62u****"}

2. Verify the VNode status.

   vnode list

   When STATUS shows Ready, the VNode is running:

   VNODE ID                    VNODE NAME                                   CREATED          STATUS    SCHEDULABLE ZONES
   vnd-7xvetkyase7gb62u****    vnode-cn-guangzhou-a-7xvetkyase7gb62u****    2 minutes ago    Ready     cn-guangzhou-a

3. Confirm the VNode appears in the cluster.

   kubectl get node

   The VNode is listed as an agent node:

   NAME                                    STATUS     ROLES                  AGE    VERSION
   cn-guangzhou.vnd-7xvetkyase7gb62u****   Ready      agent                  174m   v1.20.6
   vnode-test001                           Ready      control-plane,master   23h    v1.20.6
   vnode-test002                           Ready      <none>                 22h    v1.20.6

Prevent DaemonSets from scheduling to the VNode

VNodes are not real nodes, so DaemonSet pods cannot run on them. Configure nodeAffinity on kube-proxy to exclude the VNode.

1. Edit the kube-proxy DaemonSet.

   kubectl -n kube-system edit ds kube-proxy

2. Add the following nodeAffinity block to spec.template.spec.

   affinity:
     nodeAffinity:
       requiredDuringSchedulingIgnoredDuringExecution:
         nodeSelectorTerms:
         - matchExpressions:
           - key: type
             operator: NotIn
             values:
             - virtual-kubelet

Schedule pods to the VNode

After the VNode is created, schedule pods to it using one of the following methods:

• Manual scheduling: Configure nodeSelector and tolerations, or set nodeName on the pod spec. See Schedule pods to a VNode.

• Automatic scheduling: Deploy the eci-profile component and specify a Selector. The system automatically schedules matching pods to the VNode. See Use eci-profile to schedule pods to a VNode.

What's next

• Common commands in VNodectl