Container Service for Kubernetes:User guide for Terway Edge

In cloud node pools

Terway Edge runs as Terway in cloud node pools, the same as in ACK Pro clusters. For more information, see Terway.

On ENS nodes

Terway Edge provides container networking on Edge Node Service (ENS) nodes using ENS ENIs. For more information, see Use Terway on the ENS network.

At the edge

Terway Edge runs as Flannel (Route mode) at the edge. When an edge node joins an ACK Edge cluster, the cluster control plane automatically assigns a pod CIDR block to that node and adds container routes to its host route table.

How packets are routed between containers on different nodes:

• Same local area network (LAN): Flannel configures host routes directly. Packets reach the destination container by following those routes.

• Different LANs: Without routes on the external network devices between the LANs, packets cannot cross from one LAN to another. Use BGP to advertise container routes to those devices — see Advertise container routes with BGP.

Prerequisites

Before you begin, collect the following information from your network team:

• The IP address of your LAN gateway or Layer 3 switch (used as spec.peerIP)

• The autonomous system number (ASN) of the switch (spec.peerAsNumber)

• The ASN to assign to the BGP speakers in the cluster (spec.localAsNumber). Use a private ASN in the range 64512–65535.

• (Optional) The BGP authentication password, if your switch requires one

Step 1: Configure a BGP speaker in the cluster

Create a CustomResourceDefinition (CRD) of the BGPPeer type to configure the BGP speaker.

1. Create a file named bgppeer.yaml with the following content. Adjust the values for your environment.

   Parameter	Required	Description	Value/range
   metadata.name	Yes	Name of the BGP peer	Any valid Kubernetes resource name
   spec.localSpeakers	Yes	Nodes that act as BGP speakers and advertise container routes for all nodes in the LAN	Node names; select at least two nodes
   spec.localAsNumber	Yes	The ASN of the autonomous system (AS) the BGP speakers belong to	Private ASNs: 64512–65535
   spec.peerIP	Yes	IP address of the LAN gateway or Layer 3 switch	Valid IPv4 address
   spec.peerAsNumber	Yes	ASN of the LAN gateway or Layer 3 switch	Valid ASN
   spec.gateway	No	Custom gateway for cross-AS container traffic. Defaults to the LAN gateway or vSwitch address.	Valid IPv4 address
   spec.nodeSelector	Yes	Selects nodes that belong to the same AS. Use all() if all edge nodes are in the same LAN; otherwise, use a node pool label to scope to a node pool.	Label selector or all()
   spec.authPassword	No	BGP session authentication password. Create a Secret in the kube-system namespace first, then reference its key and name here.	Secret reference

   apiVersion: network.openyurt.io/v1alpha1
   kind: BGPPeer
   metadata:
     name: peer
   spec:
     localSpeakers:
       - node-1
       - node-2
     localAsNumber: 65010
     peerIP: 172.16.0.1
     peerAsNumber: 65001
     nodeSelector: alibabacloud.com/nodepool-id=npxxx
     # Optional.
     authPassword:
       secretKeyRef:
         name: bgp-secret
         key: password
   ---
   # Optional.
   apiVersion: v1
   kind: Secret
   metadata:
     name: bgp-secret
     namespace: kube-system
   type: Opaque
   data:
     password: bXlTZWNyZXRWYWx1ZQ==  # The value is encoded in Base64.

2. Apply the configuration.

   kubectl apply -f bgppeer.yaml

3. Verify the BGPPeer was created.

   kubectl get bgppeer peer

   Expected output:

   NAME   LOCALSPEAKERS         LOCALASNUMBER   PEERIP         PEERASNUMBER     AGE
   peer   ["node-1","node-2"]   65010           172.16.0.1     65001            10m

Step 2: Configure BGP on external network devices

1. Configure BGP settings according to your network device model and start the BGP service.

2. Add the BGP speaker nodes you selected in Step 1 as BGP peers on the external device, so the BGP sessions can be established.

Step 3: Verify BGP session establishment and container route advertisement

1. Check the BGPPeer events to confirm sessions were established.

   kubectl describe bgppeers peer-1

   If the output contains FailedEstablish events, the BGP session failed to establish. Check that the BGP settings on the external device match the speaker configuration — peer IP, ASNs, and authentication password if set.

2. On the external network device, verify that the container routes from the cluster appear in its routing table.