In Container Service for Kubernetes (ACK) Edge cluster scenarios with the Terway Edge plug-in deployed, if you run out of vSwitch IP addresses or need to expand your pod CIDR, you can add new pod vSwitches to provision additional IP resources for the cluster.
Prerequisites
An ACK Edge cluster is created with Terway Edge network plug-in deployed.
Edge nodes are provisioned through Edge Node Service (ENS).
Add a vSwitch
If IP resources of the pod vSwitches are insufficient, you can add vSwitches to the ACK Edge cluster by the following steps:
Create a vSwitch on the ENS network.
Log on to the ENS console, and choose in the left-side navigation pane.
On the vSwitches page, click Create vSwitch, enter the required information, and click Create.
Node: Select the ENS edge node.
Network: Select the network associated with ENS.
Name: Assign a name to the new vSwitch.
IPv4 CIDR Block: Select a network range that resides within the CIDR block of the ENS network.
Update the Terway Edge ConfigMap to include the new vSwitch.
kubectl edit cm privateip-config -n kube-systemConfigure the new vSwitch under
vswitches. You can add multiple vSwitches.n-xxxdenotes the network ID in ENS, while["vsw-xxx", "vsw-xxxx"]represents the vSwitch IDs within the network. Make sure you don't remove any existing vSwitch IDs.apiVersion: v1 kind: ConfigMap metadata: annotations: helm.sh/hook: pre-install name: privateip-config namespace: kube-system data: privateip_conf: | { "vswitches": { "n-xxx": ["vsw-xxx", "vsw-xxxx"], "n-xxx": ["vsw-xxx", "vsw-xxxx"] }, "vswitch_selection_policy": "most" }NoteBy default, the pod IP address is allocated from the vSwitch associated with the provisioned Elastic Network Interface (ENI). New pod vSwitches may reside under different vSwitches from those hosting ENS instances and ENI instances.
How do I know if my pod vSwitches have enough IP resources?
In Terway Edge network scenarios, if pod creation fails with the ContainerCreating status, perform the following steps to verify the sufficiency of the IP resources in the vSwitch:
Query the ENS node associated with the pod.
kubectl get pod -o wideExpected output:
NAME READY STATUS RESTARTS AGE IP NODE cube-1 0/1 ContainerCreating 0 41s <none> node-1 cube-2 0/1 ContainerCreating 0 41s <none> node-1View the resource details of the ENS node.
kubectl describe ensnodes node-1Expected output:
Name: node-1 Namespace: Labels: name=node-1 API Version: network.alibabacloud.com/v1beta1 .... Events: Type Reason Age From Message ---- ------ ---- ---- ------- Warning AllocIPFailed 1m ens-multi-ip-node [SDKError] API: AssignPrivateIpAddresses, ErrorCode: InvalidVSwitchId.IpNotEnough, RequestId: xxxxxxx, Message: Don't have enough private IPs in this switch. Warning AllocIPFailed 1m ens-multi-ip-node [SDKError] API: AssignPrivateIpAddresses, ErrorCode: InvalidVSwitchId.IpNotEnough, RequestId: xxxxxxx, Message: Don't have enough private IPs in this switch.If
ErrorCode: InvalidVSwitchId.IpNotEnoughexists inEvents, the pod vSwitch needs more IP resources.
References
For more information about how to use the Terway Edge network plug-in, see Usage guide for Terway Edge.
For instructions about Terway on the ENS network, see Use Terway on the ENS network.