By default, all nodes in a Terway cluster share the network configuration defined in the eni-config ConfigMap in the kube-system namespace. To assign different vSwitches or security groups to specific nodes, create a separate ConfigMap and label the target nodes to reference it. Terway merges the new configuration into the default configuration using JSON Merge Patch (RFC 7396).
Use cases
Configure per-node network settings when:
-
Egress IP isolation: Specific pods need dedicated NAT IP addresses or bandwidth limits, separate from the rest of the cluster.
Prerequisites
Before you begin, ensure that you have:
-
The vSwitches you plan to assign to nodes. See Create and manage a vSwitch
-
The security groups you plan to assign to nodes. See Create a security group
Considerations
-
Existing elastic network interfaces (ENIs) are not updated automatically. When you apply a new ConfigMap to a node, only ENIs created after the configuration takes effect use the new vSwitches and security groups. Existing ENIs continue to use their original settings. Apply per-node network settings to new nodes where possible.
-
vSwitches must be in the same zone as the node. If a vSwitch is in a different zone from the node, the setting has no effect.
-
Security group limit. You can assign up to five security groups per ENI. All security groups must belong to the same VPC and be of the same type. See Associate multiple security groups for an ENI.
Configure per-node network settings
This procedure consists of three steps: creating a ConfigMap that defines the network settings, labeling the target nodes to use that ConfigMap, and verifying that the new ENIs use the correct settings.
Step 1: Create a ConfigMap
Create a ConfigMap named foo in the kube-system namespace. This ConfigMap defines the vSwitches and security groups to apply to labeled nodes.
-
Log on to the ACK console. In the left-side navigation pane, click Clusters.
-
On the Clusters page, click the name of the target cluster. In the left-side navigation pane, choose Configurations > ConfigMaps.
-
On the ConfigMap page, select kube-system from the Namespace drop-down list, then click Create in the upper-right corner.
-
In the Create panel, set ConfigMap Name to foo. Click Add, set Name to eni_conf, and enter the following JSON in the Value field. Replace the
vswitchesandsecurity_groupvalues with your actual IDs.{ "vswitches": { "cn-hangzhou-g": [ "vsw-10000" ], "cn-hangzhou-i": [ "vsw-10001" ] }, "security_group": "sg-10000", "security_groups": [ "sg-10000", "sg-10001" ] }Parameter Description vswitchesA map of zone IDs to vSwitch ID lists. Each vSwitch must be in the same zone as the target nodes. To find zone IDs and vSwitch IDs, go to the VPC console and click vSwitch in the left-side navigation pane. security_groupTo assign a single security group, set this parameter and leave security_groupsas null.security_groupsTo assign one or more security groups, set this parameter and leave security_groupas null. The system applies all security groups listed in both parameters. -
Click OK.
Step 2: Label nodes
Apply the label terway-config=foo to the nodes that should use this ConfigMap.
For existing nodes:
-
In the left-side navigation pane of the cluster details page, choose Nodes > Nodes. Click Manage Labels and Taints in the upper-right corner.
-
On the Labels tab, select the target nodes and click Add Label.
-
In the Add dialog box, set Name to terway-config and Value to foo, then click OK.
-
Restart the Terway pods to apply the change.
-
In the left-side navigation pane, choose Workloads > Pods.
-
Select kube-system from the Namespace drop-down list, search for
terway-eniip, select all matching pods, and click Batch Delete. -
In the confirmation dialog, click OK. Kubernetes automatically recreates the Terway pods.
-
Wait until the Status column shows Running for all pods whose names start with
terway-eniip. The new vSwitches and security groups take effect after the pods are running.
-
For new nodes:
When creating a node pool, add a label with key set to terway-config and value set to foo. See the Create a node pool section of the "Create a node pool" topic.
Step 3: Verify the configuration
After the Terway pods restart, verify that new ENIs use the correct vSwitches and security groups.
-
Log on to the ECS console. In the left-side navigation pane, choose Instances & Images > Instances.
-
Click the name of the target ECS instance. On the instance details page, click the ENIs tab.
-
Confirm that the ENI used to allocate pod IP addresses is associated with the vSwitches and security groups you specified in the ConfigMap.
If a newly created pod's IP address does not fall within the expected vSwitch CIDR block, see the FAQ about container networks.