How to whiten the network segment specified by ACK for pod under the Terway network

The container network of ACK mainly includes Flannel and Terway. Under the Flannel network, because the pod accesses other services through the node NAT, so when we set the whitelist in the database under the Flannel network, we can first pass the client pod through The way of node binding is dispatched to a small number of fixed nodes, and then the ip address of the node can be added directly on the database side. This method is relatively simple and will not be described in detail; however, under the Terway network, podip is Provided by ENI, access external services from pods through ENI, the client ip obtained by external services is the ip address provided by ENI, even if we bind pods and nodes for affinity, the client ip for pods to access external services is ENI Provided ip instead of node ip, podip will still randomly assign ip addresses from the vswitch specified by Terway, and our client pods usually have configurations such as automatic scaling, so it is difficult to satisfy elastic scaling even if podip can be fixed In the scenario, the best way is to directly assign a network segment to the client to assign an ip, and then add white to this network segment in the database, then how should we specify the ip used by the pod in the Terway network mode What about the range?

In fact, Terway provides the function of using the specified vswitch for the pod on the specified node.

Simply put, it is to specify the vswitch used by the pod by adding a label to the specified node, so that when we schedule the pod to a node with a fixed label, the pod can create a podip through the custom vswitch. The following briefly lists the relevant Steps:

Create a separate configmap eni-config-fixed under kube-system, and specify a special vswitch in it, vsw-2zem796p76viir02c6980 10.2.1.0/24
apiVersion: v1
data:
eni_conf: |
{
"vswitches": {"cn-beijing-h":["vsw-2zem796p76viir02c6980"]}
}
kind: ConfigMap
metadata:
name: eni-config-fixed
namespace: kube-system

Create a node pool and label the node with label terway-config:eni-config-fixed; in order to ensure that no other pods will appear on the nodes in the node pool, you can configure the node pool with a stain at the same time, for example: fixed=true:NoSchedule

The nodes expanded through the node pool will have the above label and stain by default
Create a pod and schedule it on the node with the above label (note that tolerance is added)

apiVersion: apps/v1 # for versions before 1.8.0 use apps/v1beta1
kind: Deployment
metadata:
name: nginx-fixed
labels:
app: nginx-fixed
spec:
replicas: 2
selector:
matchLabels:
app: nginx-fixed
template:
metadata:
labels:
app: nginx-fixed
spec:
tolerations:
- key: "fixed"
operator: "Equal"
value: "true"
effect: "NoSchedule"
nodeSelector:
terway-config: eni-config-fixed
containers:
-name: nginx
image: nginx:1.9.0 # replace it with your exactly
ports:
- containerPort: 80

You can see that podip has been assigned podip from our own specified vswitch
In this way, we can directly add white to this vswitch on the database side, so as to implement access control for dynamic podip.

Notice:
It is best to create a new node. If it is an existing node, you need to unbind the ENI and the ECS instance before adding it to the cluster (it can be operated on the ECS console). The way to add it to the cluster is to automatically add the existing node (replace system disk)

Pay attention to label and taint specific node pools, and try to ensure that services that do not need whitening will not be dispatched to these nodes

This usage is actually a configuration override, which will use the configuration in the currently specified configmap to overwrite the previous eni-config configuration.

It is recommended that the number of ips of the specified vswitch be twice (or more) the expected number of pods. On the one hand, it can provide some margin for future expansion, and on the other hand, it can also prevent podips from being lost when a fault occurs and podips cannot be recycled in time. assignable failure