Overview of the cloud-edge O&M communication component Tunnel - Container Service for Kubernetes

For ACK Edge clusters running Kubernetes earlier than 1.26, edge-tunnel-server and edge-tunnel-agent are deployed by default to create an encrypted cloud-to-edge O&M tunnel so cloud components can reach edge nodes in private networks. This topic describes the tunnel components and how to extend edge monitoring to non-default ports. and edge-tunnel-agent create encrypted tunnels for O&M access to edge nodes in private networks.

For ACK Edge clusters on Kubernetes 1.26 or later, Raven replaces the tunnel components. See Work with the cloud-edge communication component Raven.

How it works

edge-tunnel-server runs as a Deployment on cloud nodes. edge-tunnel-agent runs as a DaemonSet on every edge node.

At cluster creation, ACK provisions a Server Load Balancer (SLB) instance in front of edge-tunnel-server. Each edge-tunnel-agent connects outward through the SLB instance to establish an encrypted tunnel over the Internet.

Once the tunnels are up, requests from kube-apiserver or metrics-server to port 10250 or 10255 on edge nodes are automatically forwarded through edge-tunnel-server. This includes O&M operations such as kubectl logs and kubectl exec, which target kubelet on those ports. No configuration changes are needed.

Warning

If the SLB instance is deleted or stopped, all tunnels stop working. Tunnels may also fail if an edge node's connection to the cloud is lost or unstable.

Prerequisites

Ensure that you have:

An ACK Edge cluster running Kubernetes earlier than 1.26
At least one Elastic Compute Service (ECS) instance in the cloud to host edge-tunnel-server

On 1.16.9-aliyunedge.1, deploy metrics-server on the same ECS node as edge-tunnel-server. On 1.18.8-aliyunedge.1 or later, they can run on separate nodes.

Extend monitoring to non-standard ports

By default, tunnels forward traffic only on ports 10250 and 10255. To use additional ports, configure the edge-tunnel-server-cfg ConfigMap in the kube-system namespace.

Configuration fields and protocols vary by cluster version:

Cluster version	Supported protocols	Configuration field	Value format
1.20.11-aliyunedge.1	HTTP	`http-proxy-ports`	`port1, port2`
1.20.11-aliyunedge.1	HTTPS	`https-proxy-ports`	`port1, port2`
1.20.11-aliyunedge.1	localhost endpoints	`localhost-proxy-ports`	`port1, port2` (default: `10250, 10255, 10266, 10267`)
1.18.8-aliyunedge.1	HTTP only	`dnat-ports-pair`	`port=10264`

Configure additional ports on 1.20.11-aliyunedge.1

This example enables:

Port 9051 over HTTP
Port 9052 and port 8080 over HTTPS
Access to localhost endpoints including https://127.0.0.1:8080

Apply the ConfigMap:

cat <<EOF | kubectl apply -f -
apiVersion: v1
data:
  http-proxy-ports: "9051"
  https-proxy-ports: "9052, 8080"
  localhost-proxy-ports: "10250, 10255, 10266, 10267, 8080"
kind: ConfigMap
metadata:
  name: edge-tunnel-server-cfg
  namespace: kube-system
EOF

Configure additional ports on 1.18.8-aliyunedge.1

On 1.18.8-aliyunedge.1, only HTTP is supported for non-standard ports. Use the dnat-ports-pair field with the format <target-port>=10264.

This example enables cloud access to port 9051 on edge nodes:

Apply the ConfigMap:

cat <<EOF | kubectl apply -f -
apiVersion: v1
data:
  dnat-ports-pair: '9051=10264'
kind: ConfigMap
metadata:
  name: edge-tunnel-server-cfg
  namespace: kube-system
EOF

Next steps

Work with the cloud-edge communication component Raven