Create a custom script for a node pool - Container Service for Kubernetes

ACK One registered clusters let you connect on-premises or third-party Kubernetes clusters to the Alibaba Cloud container platform for unified management and hybrid cloud operations. When local compute capacity is limited, scale out with cloud node pools on ECS. This topic explains how to create custom scripts for cloud node pools.

Prerequisites

Before you begin:

An ACK One registered cluster with an external Kubernetes cluster connected.
The external Kubernetes cluster network is connected to the ACK One registered cluster VPC.
The external Kubernetes cluster proxy configuration is imported to the ACK One registered cluster (private network mode).

Limitations

Sample scripts support only operating systems with YUM (Yellowdog Updater, Modified) as the package manager.
Sample scripts apply to regular ECS nodes only. For GPU-accelerated nodes, submit a ticket to the R&D team for a customized script.

GPU-accelerated nodes require drivers and device plug-ins. See Manually update the NVIDIA driver of a node.

Overview

Three steps:

Prepare the node script — adapt an existing custom script or start from a sample.
Upload the script to a file server accessible from your registered cluster.
Register the script path in the ack-agent-config ConfigMap for scale-out retrieval.

Choose a path for step 1:

Situation	Action
Existing bootstrap script	Use Step 1(A): add required Alibaba Cloud environment variables
No existing bootstrap script	Use Step 1(B): download and customize a sample

Step 1(A): Add Alibaba Cloud environment variables to an existing script

The registered cluster injects four environment variables at scale-out for node pool management.

Important

ALIBABA_CLOUD_PROVIDER_ID is required for the registered cluster to function. Without it in your script, the cluster cannot run normally.

Four environment variables:

Variable	Required	Effect if missing	Example value
`ALIBABA_CLOUD_PROVIDER_ID`	Yes	Registered cluster cannot run normally	`cn-shenzhen.i-wz92ewt14n9wx9mo***`
`ALIBABA_CLOUD_NODE_NAME`	Yes	Nodes may enter an abnormal state	`cn-shenzhen.192.168.1.***`
`ALIBABA_CLOUD_LABELS`	Yes	Node pool management and workload scheduling may fail	`alibabacloud.com/nodepool-id=np0e2031e952c4492bab32f512ce142,ack.aliyun.com=cc3df6d939b0d4463b493b82d0d670,alibabacloud.com/instance-id=i-wz960ockeekr3dok0***,alibabacloud.com/external=true,workload=cpu`
`ALIBABA_CLOUD_TAINTS`	Yes	Node pool taints do not take effect	`workload=ack:NoSchedule`

In ALIBABA_CLOUD_LABELS, only workload=cpu is custom (defined in the node pool configuration). All other labels are system labels.

Add the variable-injection block based on your cluster installation method.

Cluster created with kubeadm

Insert this block after main node configuration and before restarting kubelet.

####### Begin: Alibaba Cloud environment variable injection

# Configure node labels, taints, node name, and provider ID.
KUBELET_CONFIG_FILE="/etc/sysconfig/kubelet"
if [[ $ALIBABA_CLOUD_LABELS != "" ]];then
  option="--node-labels"
  if grep -- "${option}=" $KUBELET_CONFIG_FILE  /dev/null;then
    sed -i "s@${option}=@${option}=${ALIBABA_CLOUD_LABELS},@g" $KUBELET_CONFIG_FILE
  elif grep "KUBELET_EXTRA_ARGS=" $KUBELET_CONFIG_FILE  /dev/null;then
    sed -i "s@KUBELET_EXTRA_ARGS=@KUBELET_EXTRA_ARGS=${option}=${ALIBABA_CLOUD_LABELS} @g" $KUBELET_CONFIG_FILE
  else
    sed -i "/^\[Service\]/a\Environment=\"KUBELET_EXTRA_ARGS=${option}=${ALIBABA_CLOUD_LABELS}\"" $KUBELET_CONFIG_FILE
  fi
fi

if [[ $ALIBABA_CLOUD_TAINTS != "" ]];then
  option="--register-with-taints"
  if grep -- "${option}=" $KUBELET_CONFIG_FILE  /dev/null;then
    sed -i "s@${option}=@${option}=${ALIBABA_CLOUD_TAINTS},@g" $KUBELET_CONFIG_FILE
  elif grep "KUBELET_EXTRA_ARGS=" $KUBELET_CONFIG_FILE  /dev/null;then
    sed -i "s@KUBELET_EXTRA_ARGS=@KUBELET_EXTRA_ARGS=${option}=${ALIBABA_CLOUD_TAINTS} @g" $KUBELET_CONFIG_FILE
  else
    sed -i "/^\[Service\]/a\Environment=\"KUBELET_EXTRA_ARGS=${option}=${ALIBABA_CLOUD_TAINTS}\"" $KUBELET_CONFIG_FILE
  fi
fi

if [[ $ALIBABA_CLOUD_NODE_NAME != "" ]];then
  option="--hostname-override"
  if grep -- "${option}=" $KUBELET_CONFIG_FILE  /dev/null;then
    sed -i "s@${option}=@${option}=${ALIBABA_CLOUD_NODE_NAME},@g" $KUBELET_CONFIG_FILE
  elif grep "KUBELET_EXTRA_ARGS=" $KUBELET_CONFIG_FILE  /dev/null;then
    sed -i "s@KUBELET_EXTRA_ARGS=@KUBELET_EXTRA_ARGS=${option}=${ALIBABA_CLOUD_NODE_NAME} @g" $KUBELET_CONFIG_FILE
  else
    sed -i "/^\[Service\]/a\Environment=\"KUBELET_EXTRA_ARGS=${option}=${ALIBABA_CLOUD_NODE_NAME}\"" $KUBELET_CONFIG_FILE
  fi
fi

if [[ $ALIBABA_CLOUD_PROVIDER_ID != "" ]];then
  option="--provider-id"
  if grep -- "${option}=" $KUBELET_CONFIG_FILE  /dev/null;then
    sed -i "s@${option}=@${option}=${ALIBABA_CLOUD_PROVIDER_ID},@g" $KUBELET_CONFIG_FILE
  elif grep "KUBELET_EXTRA_ARGS=" $KUBELET_CONFIG_FILE  /dev/null;then
    sed -i "s@KUBELET_EXTRA_ARGS=@KUBELET_EXTRA_ARGS=${option}=${ALIBABA_CLOUD_PROVIDER_ID} @g" $KUBELET_CONFIG_FILE
  else
    sed -i "/^\[Service\]/a\Environment=\"KUBELET_EXTRA_ARGS=${option}=${ALIBABA_CLOUD_PROVIDER_ID}\"" $KUBELET_CONFIG_FILE
  fi
fi

####### End: Alibaba Cloud environment variable injection

Proceed to Step 2: Upload the script.

Cluster installed with binary files

For clusters installed with Kubernetes binary files, modify the kubelet boot configuration. The kubelet.service file is usually at /usr/lib/systemd/system/.

cat /usr/lib/systemd/system/kubelet.service EOF
# Custom configurations are not shown.
...
[Service]
ExecStart=/data0/kubernetes/bin/kubelet \
  # Add the following flags:
  --node-ip=${ALIBABA_CLOUD_NODE_NAME} \
  --hostname-override=${ALIBABA_CLOUD_NODE_NAME} \
  --node-labels=${ALIBABA_CLOUD_LABELS} \
  --provider-id=${ALIBABA_CLOUD_PROVIDER_ID} \
  --register-with-taints=${ALIBABA_CLOUD_TAINTS} \

  ....
  --v=4
Restart=on-failure
RestartSec=5
[Install]
WantedBy=multi-user.target
EOF

Proceed to Step 2: Upload the script.

Step 1(B): Use a sample script

Without an existing bootstrap script, start from a sample below. Samples cover Docker and containerd.

Collect the required values

Collect three values from your external cluster:

Step 1: Get the Kubernetes version

Get the Kubernetes version:

For Kubernetes 1.18 and later (use control-plane label):

kubectl get no $(kubectl get nodes -l node-role.kubernetes.io/control-plane -o json | jq -r '.items[0].metadata.name') -o json | jq -r '.status.nodeInfo.kubeletVersion'

For Kubernetes earlier than 1.18 (use master label):

kubectl get no $(kubectl get nodes -l node-role.kubernetes.io/master -o json | jq -r '.items[0].metadata.name') -o json | jq -r '.status.nodeInfo.kubeletVersion'

Expected output:

v1.14.10

Save as KUBE_VERSION.

Step 2: Get the container runtime and version

For Kubernetes 1.18 and later:

kubectl get no $(kubectl get nodes -l node-role.kubernetes.io/control-plane -o json | jq -r '.items[0].metadata.name') -o json | jq -r '.status.nodeInfo.containerRuntimeVersion'

For Kubernetes earlier than 1.18:

kubectl get no $(kubectl get nodes -l node-role.kubernetes.io/master -o json | jq -r '.items[0].metadata.name') -o json | jq -r '.status.nodeInfo.containerRuntimeVersion'

Expected output:

docker://18.6.3      # Docker runtime
containerd://1.4.3   # containerd runtime

Save the version number (without the docker:// or containerd:// prefix) as RUNTIME_VERSION.

Step 3: Generate the kubeadm join command

Important

The --ttl 0 flag is required. Without it, the join token expires and auto scaling stops.

kubeadm token create --ttl 0 --print-join-command

Expected output:

kubeadm join 192.168.8.XXX:6443 --token k8xsq8.4oo8va9wcqpb*** \
    --discovery-token-ca-cert-hash sha256:cb5fc894ab965dfbc4c194e1065869268f8845c3ec40f78f9021dde24610d***

Save as KUBEADM_JOIN_CMD.

Sample script: Docker

Replace the three placeholders with your collected values:

KUBE_VERSION — Kubernetes version (for example, 1.21.0)
RUNTIME_VERSION — Docker version (for example, 18.6.3)
KUBEADM_JOIN_CMD — full kubeadm join ... command

Expand to view the code

#!/bin/bash

# The Kubernetes version of the external cluster.
export KUBE_VERSION=KUBE_VERSION

# The Docker version of the external cluster.
export RUNTIME_VERSION=RUNTIME_VERSION

# The kubeadm join command.
export KUBEADM_JOIN_CMD=KUBEADM_JOIN_CMD

# Disable the firewall.
systemctl stop firewalld
systemctl disable firewalld
systemctl is-enabled firewalld

# Disable SELinux.
sed -i "s/SELINUX=enforcing/SELINUX=disabled/g" /etc/selinux/config
setenforce 0

# Disable swap partitions.
sed -i '/swap/s/^/#/g' /etc/fstab
swapoff -a

# Configure kernel parameters.
cat  /etc/sysctl.d/Kubernetes.conf EOF
net.bridge.bridge-nf-call-ip6tables = 1
net.bridge.bridge-nf-call-iptables = 1
net.ipv4.ip_forward = 1
vm.swappiness = 0
EOF

sysctl --system

# Install network and system utilities.
yum -y install conntrack ipvsadm ipset jq iptables curl sysstat libseccomp wget vim net-tools git
cat  /etc/modules-load.d/ipvs.conf EOF
ip_vs
ip_vs_rr
ip_vs_wrr
ip_vs_sh
nf_conntrack
EOF

systemctl enable --now systemd-modules-load

# Configure the Docker YUM source.
yum-config-manager --add-repo http://mirrors.aliyun.com/docker-ce/linux/centos/docker-ce.repo

# Install Docker dependencies.
yum install -y yum-utils device-mapper-persistent-data lvm2

# View the available Docker versions.
yum list docker-ce --showduplicates | sort -r

# Install Docker.
yum install -y docker-ce-$RUNTIME_VERSION docker-ce-cli-$RUNTIME_VERSION containerd.io

# Configure the Docker daemon.
mkdir /etc/docker
cat  /etc/docker/daemon.json EOF
{
  "exec-opts": ["native.cgroupdriver=systemd"],
  "log-driver": "json-file",
  "log-opts": {
    "max-size": "100m"
  },
  "storage-driver": "overlay2",
  "storage-opts": [
    "overlay2.override_kernel_check=true"
  ],
  "registry-mirrors": ["https://docker.mirrors.ustc.edu.cn"]
}
EOF

mkdir -p /etc/systemd/system/docker.service.d

# Start Docker.
systemctl daemon-reload
systemctl restart docker
systemctl enable docker

# Configure the Kubernetes YUM source.
cat /etc/yum.repos.d/kubernetes.repo EOF
[kubernetes]
name=Kubernetes
baseurl=https://mirrors.aliyun.com/kubernetes/yum/repos/kubernetes-el7-x86_64/
enabled=1
gpgcheck=0
repo_gpgcheck=0
gpgkey=https://mirrors.aliyun.com/kubernetes/yum/doc/yum-key.gpg https://mirrors.aliyun.com/kubernetes/yum/doc/rpm-package-key.gpg
EOF

yum install -y kubelet-$KUBE_VERSION kubeadm-$KUBE_VERSION kubectl-$KUBE_VERSION --disableexcludes=kubernetes

# Configure kubelet.service.
cat /etc/systemd/system/kubelet.service EOF
# ! IMPORTANT !
# This configuration is managed and generated by ACK
# please do not edit it to avoid unexpected failures

[Unit]
Description=kubelet: The Kubernetes Node Agent
Documentation=http://kubernetes.io/docs/

[Service]
ExecStart=/usr/bin/kubelet
Restart=always
StartLimitInterval=0
RestartSec=10

[Install]
WantedBy=multi-user.target
EOF

#  The environment variables of the registered cluster. Do not delete the settings.
# Configure node labels, taints, node name, and provider ID.
KUBELET_CONFIG_FILE="/etc/sysconfig/kubelet"
if [[ $ALIBABA_CLOUD_LABELS != "" ]];then
  option="--node-labels"
  if grep -- "${option}=" $KUBELET_CONFIG_FILE  /dev/null;then
    sed -i "s@${option}=@${option}=${ALIBABA_CLOUD_LABELS},@g" $KUBELET_CONFIG_FILE
  elif grep "KUBELET_EXTRA_ARGS=" $KUBELET_CONFIG_FILE  /dev/null;then
    sed -i "s@KUBELET_EXTRA_ARGS=@KUBELET_EXTRA_ARGS=${option}=${ALIBABA_CLOUD_LABELS} @g" $KUBELET_CONFIG_FILE
  else
    sed -i "/^\[Service\]/a\Environment=\"KUBELET_EXTRA_ARGS=${option}=${ALIBABA_CLOUD_LABELS}\"" $KUBELET_CONFIG_FILE
  fi
fi

if [[ $ALIBABA_CLOUD_TAINTS != "" ]];then
  option="--register-with-taints"
  if grep -- "${option}=" $KUBELET_CONFIG_FILE  /dev/null;then
    sed -i "s@${option}=@${option}=${ALIBABA_CLOUD_TAINTS},@g" $KUBELET_CONFIG_FILE
  elif grep "KUBELET_EXTRA_ARGS=" $KUBELET_CONFIG_FILE  /dev/null;then
    sed -i "s@KUBELET_EXTRA_ARGS=@KUBELET_EXTRA_ARGS=${option}=${ALIBABA_CLOUD_TAINTS} @g" $KUBELET_CONFIG_FILE
  else
    sed -i "/^\[Service\]/a\Environment=\"KUBELET_EXTRA_ARGS=${option}=${ALIBABA_CLOUD_TAINTS}\"" $KUBELET_CONFIG_FILE
  fi
fi

if [[ $ALIBABA_CLOUD_NODE_NAME != "" ]];then
  option="--hostname-override"
  if grep -- "${option}=" $KUBELET_CONFIG_FILE  /dev/null;then
    sed -i "s@${option}=@${option}=${ALIBABA_CLOUD_NODE_NAME},@g" $KUBELET_CONFIG_FILE
  elif grep "KUBELET_EXTRA_ARGS=" $KUBELET_CONFIG_FILE  /dev/null;then
    sed -i "s@KUBELET_EXTRA_ARGS=@KUBELET_EXTRA_ARGS=${option}=${ALIBABA_CLOUD_NODE_NAME} @g" $KUBELET_CONFIG_FILE
  else
    sed -i "/^\[Service\]/a\Environment=\"KUBELET_EXTRA_ARGS=${option}=${ALIBABA_CLOUD_NODE_NAME}\"" $KUBELET_CONFIG_FILE
  fi
fi

if [[ $ALIBABA_CLOUD_PROVIDER_ID != "" ]];then
  option="--provider-id"
  if grep -- "${option}=" $KUBELET_CONFIG_FILE  /dev/null;then
    sed -i "s@${option}=@${option}=${ALIBABA_CLOUD_PROVIDER_ID},@g" $KUBELET_CONFIG_FILE
  elif grep "KUBELET_EXTRA_ARGS=" $KUBELET_CONFIG_FILE  /dev/null;then
    sed -i "s@KUBELET_EXTRA_ARGS=@KUBELET_EXTRA_ARGS=${option}=${ALIBABA_CLOUD_PROVIDER_ID} @g" $KUBELET_CONFIG_FILE
  else
    sed -i "/^\[Service\]/a\Environment=\"KUBELET_EXTRA_ARGS=${option}=${ALIBABA_CLOUD_PROVIDER_ID}\"" $KUBELET_CONFIG_FILE
  fi
fi
# The environment variables of the registered cluster. Do not delete the settings. 

systemctl enable kubelet
systemctl start kubelet

# Join the cluster.
$KUBEADM_JOIN_CMD

Sample script: containerd

Replace the three placeholders with your collected values:

KUBE_VERSION — Kubernetes version (for example, 1.21.0)
RUNTIME_VERSION — containerd version (for example, 1.4.3)
KUBEADM_JOIN_CMD — full kubeadm join ... command

Expand to view the code

#!/bin/bash

# The Kubernetes version of the external cluster.
export KUBE_VERSION=KUBE_VERSION

# The containerd version of the external cluster.
export RUNTIME_VERSION=RUNTIME_VERSION

# The kubeadm join command.
export KUBEADM_JOIN_CMD=KUBEADM_JOIN_CMD

# Disable the firewall.
systemctl stop firewalld
systemctl disable firewalld
systemctl is-enabled firewalld

# Disable SELinux.
sed -i "s/SELINUX=enforcing/SELINUX=disabled/g" /etc/selinux/config
setenforce 0

# Disable swap partitions.
sed -i '/swap/s/^/#/g' /etc/fstab
swapoff -a

# Configure kernel parameters.
cat  /etc/sysctl.d/Kubernetes.conf EOF
net.bridge.bridge-nf-call-ip6tables = 1
net.bridge.bridge-nf-call-iptables = 1
net.ipv4.ip_forward = 1
vm.swappiness = 0
EOF

sysctl --system

# Install network and system utilities.
yum -y install conntrack ipvsadm ipset jq iptables curl sysstat libseccomp wget vim net-tools git
cat  /etc/modules-load.d/ipvs.conf EOF
ip_vs
ip_vs_rr
ip_vs_wrr
ip_vs_sh
nf_conntrack
EOF

systemctl enable --now systemd-modules-load

# Install and configure containerd.
yum -y install yum-utils device-mapper-persistent-data lvm2
yum-config-manager --add-repo http://mirrors.aliyun.com/docker-ce/linux/centos/docker-ce.repo

# Load required kernel modules.
cat /etc/modules-load.d/containerd.conf EOF
overlay
br_netfilter
EOF
modprobe overlay
modprobe br_netfilter

# Install containerd.
yum install containerd.io-$RUNTIME_VERSION -y

# Configure the containerd client (crictl).
cat /etc/crictl.yaml EOF
runtime-endpoint: unix:///run/containerd/containerd.sock
image-endpoint: unix:///run/containerd/containerd.sock
timeout: 10
debug: false
pull-image-on-create: false
disable-pull-on-run: false
EOF

# Configure containerd.
mkdir -p /etc/containerd
containerd config default  /etc/containerd/config.toml
# Use systemd to manage cgroups.
sed -i '/SystemdCgroup/s/false/true/g' /etc/containerd/config.toml
# Pull the sandbox image from Alibaba Cloud.
sed -i '/sandbox_image/s/registry.k8s.io/registry.aliyuncs.com\/google_containers/g' /etc/containerd/config.toml
sed -i 's#sandbox_image = "registry.k8s.io/pause:3.6"#sandbox_image = "registry.aliyuncs.com/google_containers/pause:3.9"#' /etc/containerd/config.toml

# Start containerd.
systemctl enable containerd
systemctl start containerd

# Configure the Kubernetes YUM source.
cat /etc/yum.repos.d/kubernetes.repo EOF
[kubernetes]
name=Kubernetes
baseurl=https://mirrors.aliyun.com/kubernetes/yum/repos/kubernetes-el7-x86_64/
enabled=1
gpgcheck=0
repo_gpgcheck=0
gpgkey=https://mirrors.aliyun.com/kubernetes/yum/doc/yum-key.gpg https://mirrors.aliyun.com/kubernetes/yum/doc/rpm-package-key.gpg
EOF

yum -y install kubeadm-$KUBE_VERSION kubectl-$KUBE_VERSION kubelet-$KUBE_VERSION

# Configure kubelet.service.
cat /etc/systemd/system/kubelet.service EOF
# ! IMPORTANT !
# This configuration is managed and generated by ACK
# please do not edit it to avoid unexpected failures

[Unit]
Description=kubelet: The Kubernetes Node Agent
Documentation=http://kubernetes.io/docs/

[Service]
ExecStart=/usr/bin/kubelet
Restart=always
StartLimitInterval=0
RestartSec=10

[Install]
WantedBy=multi-user.target
EOF

#  The environment variables of the registered cluster. Do not delete the settings.
# Configure node labels, taints, node name, and provider ID.
KUBELET_CONFIG_FILE="/etc/sysconfig/kubelet"
if [[ $ALIBABA_CLOUD_LABELS != "" ]];then
  option="--node-labels"
  if grep -- "${option}=" $KUBELET_CONFIG_FILE  /dev/null;then
    sed -i "s@${option}=@${option}=${ALIBABA_CLOUD_LABELS},@g" $KUBELET_CONFIG_FILE
  elif grep "KUBELET_EXTRA_ARGS=" $KUBELET_CONFIG_FILE  /dev/null;then
    sed -i "s@KUBELET_EXTRA_ARGS=@KUBELET_EXTRA_ARGS=${option}=${ALIBABA_CLOUD_LABELS} @g" $KUBELET_CONFIG_FILE
  else
    sed -i "/^\[Service\]/a\Environment=\"KUBELET_EXTRA_ARGS=${option}=${ALIBABA_CLOUD_LABELS}\"" $KUBELET_CONFIG_FILE
  fi
fi

if [[ $ALIBABA_CLOUD_TAINTS != "" ]];then
  option="--register-with-taints"
  if grep -- "${option}=" $KUBELET_CONFIG_FILE  /dev/null;then
    sed -i "s@${option}=@${option}=${ALIBABA_CLOUD_TAINTS},@g" $KUBELET_CONFIG_FILE
  elif grep "KUBELET_EXTRA_ARGS=" $KUBELET_CONFIG_FILE  /dev/null;then
    sed -i "s@KUBELET_EXTRA_ARGS=@KUBELET_EXTRA_ARGS=${option}=${ALIBABA_CLOUD_TAINTS} @g" $KUBELET_CONFIG_FILE
  else
    sed -i "/^\[Service\]/a\Environment=\"KUBELET_EXTRA_ARGS=${option}=${ALIBABA_CLOUD_TAINTS}\"" $KUBELET_CONFIG_FILE
  fi
fi

if [[ $ALIBABA_CLOUD_NODE_NAME != "" ]];then
  option="--hostname-override"
  if grep -- "${option}=" $KUBELET_CONFIG_FILE  /dev/null;then
    sed -i "s@${option}=@${option}=${ALIBABA_CLOUD_NODE_NAME},@g" $KUBELET_CONFIG_FILE
  elif grep "KUBELET_EXTRA_ARGS=" $KUBELET_CONFIG_FILE  /dev/null;then
    sed -i "s@KUBELET_EXTRA_ARGS=@KUBELET_EXTRA_ARGS=${option}=${ALIBABA_CLOUD_NODE_NAME} @g" $KUBELET_CONFIG_FILE
  else
    sed -i "/^\[Service\]/a\Environment=\"KUBELET_EXTRA_ARGS=${option}=${ALIBABA_CLOUD_NODE_NAME}\"" $KUBELET_CONFIG_FILE
  fi
fi

if [[ $ALIBABA_CLOUD_PROVIDER_ID != "" ]];then
  option="--provider-id"
  if grep -- "${option}=" $KUBELET_CONFIG_FILE  /dev/null;then
    sed -i "s@${option}=@${option}=${ALIBABA_CLOUD_PROVIDER_ID},@g" $KUBELET_CONFIG_FILE
  elif grep "KUBELET_EXTRA_ARGS=" $KUBELET_CONFIG_FILE  /dev/null;then
    sed -i "s@KUBELET_EXTRA_ARGS=@KUBELET_EXTRA_ARGS=${option}=${ALIBABA_CLOUD_PROVIDER_ID} @g" $KUBELET_CONFIG_FILE
  else
    sed -i "/^\[Service\]/a\Environment=\"KUBELET_EXTRA_ARGS=${option}=${ALIBABA_CLOUD_PROVIDER_ID}\"" $KUBELET_CONFIG_FILE
  fi
fi
# The environment variables of the registered cluster. Do not delete the settings. 

systemctl enable kubelet
systemctl start kubelet

# Join the cluster.
$KUBEADM_JOIN_CMD

Step 2: Upload the script

Upload the script to a file server reachable from your registered cluster. Object Storage Service (OSS) is common.

Example URL:

https://kubelet-****.oss-cn-hangzhou-internal.aliyuncs.com/join-ecs-nodes.sh

Save the URL for the next step.

Step 3: Register the script path

Important

Complete this step before creating a node pool. Without a registered script path, scale-out cannot retrieve the script and will fail.

Set addNodeScriptPath in the ack-agent-config ConfigMap (kube-system namespace).

kubectl edit cm ack-agent-config -n kube-system

Set addNodeScriptPath to your script URL:

apiVersion: v1
data:
  addNodeScriptPath: https://kubelet-****.oss-cn-hangzhou-internal.aliyuncs.com/join-ecs-nodes.sh
kind: ConfigMap
metadata:
  name: ack-agent-config
  namespace: kube-system

Save and close the file. The registered cluster uses this script to bootstrap ECS nodes during scale-out.