Author | Alibaba senior technical expert Ye Lei
this article introduces Kubernetes ideas about network models. As we all know, Kubernetes has no restrictions on the specific network implementation plan and does not provide particularly good reference cases. Kubernetes limits whether a container network is qualified, that is, Kubernetes container network model. It can be summarized into three chapters of the contract and four goals.
- The three chapters of the contract mean: when evaluating a container network or designing a container network, its access conditions. Which three requirements does it meet? Only then can it be considered as a qualified network solution.
- The four goals mean that when designing the topology of the network and the realization of specific functions of the network, we need to think clearly whether the connectivity can be achieved.
first look at the ground rules:
- article 1: Any two pods can communicate directly without using NAT to receive data and address translation;
- article 2: nodes and pods can communicate directly without obvious address translation;
- article 3: when a pod sees its own IP address, it uses the same IP address as when others see it.
I will Wen Zhonghui my personal understanding about why Kubernetes have some seemingly arbitrary models and requirements for container networks.
in fact, the four goals are to design a K8s system to provide services for the outside world. From the perspective of network, how can the outside world connect to the application inside the container step by step?
- How does the external world communicate with the service? Is there an Internet or a user outside the company, how to use the service? service refers to the service concept in K8s.
- How does a service communicate with its backend pods?
- How does a pod communicate with a pod?
- What is the communication between containers in the pod?
The ultimate goal is to connect the external world to the innermost to provide services for containers.
Interpretation of Basic Constraints
the basic constraints can be interpreted as follows: the complexity of container network development lies in that it is actually parasitic on the Host network. From this perspective, the container network solution can be divided Underlay/Overlay two major factions:
- the standard of Underlay is that it is at the same layer as the Host network. A feature that can be seen from the outside is whether it uses the same CIDR block of the Host network, input and output basic devices, whether the IP address of the container needs to cooperate with the Host network (from the same center or unified division). This is Underlay;
- the difference between Overlay is that it does not need to apply for an IP address from the IPM Management component of the Host network. Generally speaking, it only needs to not conflict with the Host network, and this IP address can be freely allocated.
Why does the community propose perPodperIP simple and arbitrary model? I personally think that for behind of service management some service tracking performance monitoring, have brought very many benefits. Because an IP address is always in the end, it will be of great benefit to the case or all kinds of small things.
what does Netns achieve
the following is a brief description of the kernel foundation that can be implemented through the Network in the Network Namespace. In a narrow sense, runC container technology does not depend on any hardware. Its execution is based on its kernel. The kernel of a process represents a task. If it does not need isolation, the namespace of the host is used, and the nsproxy-namespace proxy is not required.
From the sensory point of view, an isolated network space will have its own network card or network equipment. The NIC may be virtual or physical. It has its own IP address, IP table, Lu Youbiao, and protocol stack status. This refers to the TCP/IP protocol stack. It has its own status, iptables, and ipvs.
From the perspective of the whole sense, this is equivalent to having a completely independent network, which is isolated from the host network. Of course, the code of the protocol stack is public, but the data structure is different.
Relationship between pods and Netns
this figure clearly shows the relationship between Netns in a pod. Each pod has an independent network space, which is shared by the pod net container. General K8s will recommended Loopback interface, in pod net container communication between, and all the container pass pod IP PROVIDE external services. In addition, the Root Netns on the host can be regarded as a special network space, except that its Pid is 1.
typical container network implementation
next, we will briefly introduce a typical container network implementation solution. The container network solution may be one of the most flourishing areas in K8s. It has various implementations. The complexity of the container network lies in that it needs to coordinate with the network at the underlying Iass layer and make some choices in terms of performance and flexibility in IP allocation. This solution is various.
The following is a brief introduction to several major solutions: Flannel, Calico, Canal, and finally WeaveNet. Most of the solutions adopt a policy routing method similar to Calico.
- Flannel is a relatively unified solution, it provides a variety of network backend. Different backend fulfills different topology, it can cover multiple scene;
- Calico policy Routing is mainly adopted, and BGP protocol is adopted between nodes to synchronize routes. It is characterized by its rich functions, especially its good support for Network Point. Everyone knows the requirements of Calico for the underlying Network. Generally, mac addresses are required to be able to pass through directly and cannot cross layer -2 domains;
- of course, some community classmates have integrated the advantages of Flannel and Calico. We call it the grafting innovation project Cilium ;
- last but not least WeaveNet , if you need to encrypt data in use, you can choose to use WeaveNet, its dynamic solution can achieve better encryption.
Flannel solution is currently the most commonly used solution. As shown in the preceding figure, you can see a typical container network solution. The first thing to solve is how the container package arrives at the Host. Here, a Bridge is added. Its backend is independent, that is, you can choose how to leave the Host, which encapsulation method to use, or no encapsulation.
Now to introduce three main backend:
- one is user-state udp, which is the earliest implementation;
- then there is the Vxlan of the kernel, both of which are overlay solutions. The performance of Vxlan is better, but it requires the kernel version, which requires the kernel to support the features of Vxlan;
- if your cluster is not large enough and is in the same layer -2 domain, you can also use host-gw. The backend of this method is basically started by a broadcast routing rule and has high performance.
basic concepts of Network Policy
the following describes the concepts of Network Policy.
As mentioned earlier, the basic model of Kubernetes network requires full interconnection between pods, which may cause some problems: in a Kubernetes cluster, some call chains are not directly called. For example, between two departments, I hope that Department A will not visit the services of Department B, and the concept of strategy can be used at this time.
Basically the idea is this: It uses various selector (label or namespace), find a group pod, or find equivalent to communication both ends of, it can be understood as a whitelist mechanism to determine whether a stream can be connected through its feature description.
Before using Network Policy, note that apiserver needs to turn on these switches, as shown in the preceding figure. Another more important thing is that the Network plug-in we choose needs to support the implementation of Network Policy. You should know that Network Policy is only an object provided by K8s and does not have built-in components for implementation. It depends on whether the container Network solution you choose supports this standard and its completeness, if you choose Flannel and so on, it does not really implement this Policy, then it is useless if you try this.
Configure an instance
next, let's talk about a configuration instance, or what to do when designing a Network Policy? Personally, I think three things need to be decided:
- the first thing is to control the object, just like the spec part in this instance. In spec, through podSelector or namespace selector, you can select a specific group of pods to accept our control;
- the second is to consider the flow direction clearly. Do you need to control the inbound direction or outbound direction? Or both directions need to be controlled?
- The most important part is the third part. If you want to add a control object to the selected direction to describe its stream, which stream can be put in or out? Similar to the quintuple of this stream feature, some selectors can be used to decide which can be used as my remote end, which is the choice of objects; you can also use the IPBlock mechanism to determine which IP addresses can be allowed and which protocols or ports are allowed. In fact, a stream feature is a quintuple, which selects a specific acceptable stream.
this is the end of this article. Let's summarize briefly:
- in the container network of a pod, the core concept is IP. IP is the address base for external communication of each pod. It must be internal and external consistent and conform to the Kubernetes model;
- when introducing the network solution, the topology is the most critical factor that affects the performance of the container network. Do you want to understand how your package is connected to the end of the end, how to reach the Host from the container, and whether to encapsulate or unencapsulate the Host out of the container? Or through policy routing? How is the final arrival at the peer end solved?
- Select the container network and design. If you are not clear about your external network, or you need a solution with the strongest universality, suppose you are not clear about whether the mac is directly connected or whether the Lu Youbiao of the external router can be controlled, then you can choose Flannel use Vxlan as the backend solution. If you are sure that your network is 2-layer directly connected, you can use Calico or Flannel-Hostgw as a backend;
- Finally, Network Policy is a powerful tool for O & M and use, which can accurately control inbound and outbound streams. We also introduced the implementation method. You need to know who you want to control and how to define your flow.
finally, leave some thoughts, you can think about it:
- why is the interface standardized with CNI, but the container network does not have a standard implementation, built in K8s?
- Why does the Network Policy not have a standard controller or a standard implementation, but is provided by the owner of the container Network?
- Is it possible to implement container network without network devices? Considering that RDMA and other solutions are different from TCP/IP.
- In operations in the process network problem more, were difficult troubleshooting, so worth doing or not an open source tool, let it can friendly display from container to the Host between, Host to the Host, in other words, whether there are problems with the network conditions at various stages between encapsulation and unencapsulation can be quickly located. As far as I know, there should be no such tools now.
These are the basic concepts and Network Policy of Kubernetes container Network.
Alibaba Cloud Native WeChat public account (ID:Alicloudnative) focuses on technical fields such as microservices, Serverless, containers, and Service Mesh, focuses on popular cloud native technology trends, and cloud native large-scale implementation practices, be a technology public account that best understands cloud native developers."