Control pod network traffic with Kubernetes NetworkPolicy - Container Compute Service

Alibaba Cloud Container Compute Service (ACS) provides policy-based network control using Kubernetes NetworkPolicy. To control network traffic at the IP address or port level for specific applications in your cluster, you can use network policies. This topic describes how to use network policies in an ACS cluster and includes common scenarios.

Scope

Only CPU pods of the General-purpose and compute-optimized instance types are supported.
Only IPv4 is supported. IPv6 is not supported.

Notes

NetworkPolicy rules let you select namespaces or pods using a LabelSelector. However, as the number of NetworkPolicies applied to a pod increases, the time required for rules to take effect also increases. Additionally, many NetworkPolicy rules can complicate cluster management and troubleshooting. Therefore, keep the number of NetworkPolicies in your cluster below 40.

Step 1: Enable network policies

Install the Poseidon component.
1. Log on to the ACS console. In the left navigation pane, click Clusters.
2. On the Clusters page, click the name of the target cluster. In the left navigation pane, click Add-ons.
3. On the Component Management page, click the Network tab. In the lower-right corner of the Poseidon component card, click Install.
4. On the Install Component Poseidon page, select Enable NetworkPolicy, and then click Confirm.
  After the component is installed, Installed appears in the upper-right corner of the card.

Enable NetworkPolicy in the pod annotation.

For containers that require NetworkPolicy, add the following annotation: network.alibabacloud.com/enable-network-policy-agent: "true". The following is an example:

apiVersion: v1
kind: Pod
metadata:
  annotations:
    network.alibabacloud.com/enable-network-policy-agent: "true"
  name: example
  namespace: default
spec:
  containers:
  - image: nginx
    imagePullPolicy: IfNotPresent
    name: example

Step 2: Create an Nginx test application that can be accessed by other pods

kubectl

Obtain the kubeconfig file of the cluster and use kubectl to connect to the cluster.

Create an Nginx application and use a Service named nginx to communicate with it.

Create the Nginx application:

kubectl run nginx --image=nginx

Expected output:

pod/nginx created

Check if the pod has started:

kubectl get pod

Expected output:

NAME                     READY   STATUS    RESTARTS   AGE
nginx                    1/1     Running   0          45s

Create a Service named nginx:

kubectl expose pod nginx --port=80

Expected output:

service/nginx exposed

View the Service:

kubectl get service

Expected output:

NAME         TYPE        CLUSTER-IP      EXTERNAL-IP   PORT(S)   AGE
kubernetes   ClusterIP   172.XX.XX.1     <none>        443/TCP   30m
nginx        ClusterIP   172.XX.XX.48    <none>        80/TCP    12s

Create a pod named busybox to access the Service named nginx.

kubectl run busybox --rm -ti --image=registry.cn-hangzhou.aliyuncs.com/acs/busybox:v1.29.2 /bin/sh

Expected output:

If you don't see a command prompt, try pressing enter.
/ #
/ #

Get Nginx:

If you don't see a command prompt, try pressing enter.
/ #
/ # wget nginx  # Enter wget nginx here.

Expected output:

Connecting to nginx (172.XX.XX.48:80)
saving to 'index.html'
index.html           100% |****************************************************************************************************************************************************|   612  0:00:00 ETA
'index.html' saved

Step 3: Use network policies

You can use network policies in the following scenarios as needed.

Scenario 1: Use a network policy to allow access only from applications with a specific label

kubectl

Use the following YAML template and run the vim policy.yaml command to create a file named policy.yaml.

vim policy.yaml

The following is the content of the YAML file.

kind: NetworkPolicy
apiVersion: networking.k8s.io/v1
metadata:
  name: access-nginx
spec:
  podSelector:
    matchLabels:
      run: nginx
  ingress:
  - from:
    - podSelector:
        matchLabels:
          access: "true"

Create a network policy from the policy.yaml file.

kubectl apply -f policy.yaml

Expected output:

networkpolicy.networking.k8s.io/access-nginx created

When no access label is defined, test access to the nginx Service. The request times out and access fails.

kubectl run busybox --rm -ti --image=registry.cn-hangzhou.aliyuncs.com/acs/busybox:v1.29.2 /bin/sh

Test access to the nginx Service:

wget nginx

Expected output:

Connecting to nginx (172.19.XX.XX:80)
wget: can't connect to remote host (172.19.XX.XX): Connection timed out

Define an access label.

kubectl run busybox --rm -ti --labels="access=true" --image=registry.cn-hangzhou.aliyuncs.com/acs/busybox:v1.29.2 /bin/sh

Test access to the nginx Service:

wget nginx

Expected output:

Connecting to nginx (172.21.XX.XX:80)
saving to 'index.html'
index.html           100% |****************************************************************************************************************************************************|   612  0:00:00 ETA
'index.html' saved

When the connection progress to Nginx reaches 100%, the request is successful and you can access the Nginx service.

Scenario 2: Use a network policy to restrict the source CIDR blocks that can access an Internet-facing service

kubectl

Create an Alibaba Cloud SLB service for the nginx application and set type=LoadBalancer to expose the nginx service to Internet users.

vim nginx-service.yaml

The following is the template for the nginx-service.yaml file.

# Paste the following YAML content into nginx-service.yaml.
apiVersion: v1
kind: Service
metadata:
  labels:
    run: nginx
  name: nginx-slb
spec:
  externalTrafficPolicy: Local
  ports:
  - port: 80
    protocol: TCP
    targetPort: 80
  selector:
    run: nginx
  type: LoadBalancer

Create a network policy from the nginx-service.yaml file.

kubectl apply -f nginx-service.yaml

Expected output:

service/nginx-slb created

Check if the application exposes the Nginx service:

kubectl get service nginx-slb

Expected output:

NAME        TYPE           CLUSTER-IP      EXTERNAL-IP      PORT(S)        AGE
nginx-slb   LoadBalancer   172.19.xx.xxx   47.110.xxx.xxx   80:32240/TCP   8m

Access the IP address of the SLB instance that you just created, 47.110.xxx.xxx. Access fails.
```
wget 47.110.xxx.xxx
```
Expected output:
```
--2018-11-21 11:46:05--  http://47.110.xx.xxx/
Connecting to 47.110.XX.XX:80... failed: Connection refused.
```
Note
Access fails for the following reasons:
The configured nginx Service can only be accessed by applications with the specific label access=true.
Accessing the SLB IP address is an external access to Kubernetes, which is different from using a network policy to restrict service access to applications with a specific label.
Solution: Modify the network policy to add the allowed source IP address range.

View the IP address of your local machine.

curl myip.ipip.net

Expected output:

Current IP: 10.0.x.x From: China Beijing Beijing        # This is an example. The actual output from your device may vary.

Modify the policy.yaml file that you created.

vim policy.yaml

Modify the policy.yaml file to have the following content:

# The following is the content of the YAML file.
kind: NetworkPolicy
apiVersion: networking.k8s.io/v1
metadata:
  name: access-nginx
spec:
  podSelector:
    matchLabels:
      run: nginx
  ingress:
  - from:
    - podSelector:
        matchLabels:
          access: "true"
    - ipBlock:
        cidr: 100.64.0.0/10
    - ipBlock:
        cidr: 10.0.0.1/24      # Your local IP address. This is an example. The actual address from your device may vary.

Create a network policy from the policy.yaml file.

kubectl apply -f policy.yaml

Expected output:

networkpolicy.networking.k8s.io/access-nginx unchanged

Note

Some networks have multiple egress IP addresses. Use a /24 address range here.
The SLB health check address is within the 100.64.0.0/10 address range. Therefore, you must configure 100.64.0.0/10.

Create the Nginx service.

kubectl run busybox --rm -ti --labels="access=true" --image=registry.cn-hangzhou.aliyuncs.com/acs/busybox:v1.29.2 /bin/sh

Access the Nginx service:

wget 47.110.XX.XX

Expected output:

Connecting to 47.110.XX.XX (47.110.XX.XX:80)
index.html           100% |***********************************************************|   612  0:00:00 ETA

When the connection progress shows 100%, you have successfully accessed the Nginx service.

Scenario 3: Use a network policy to restrict a pod to access only specified addresses

kubectl

Get the list of IP addresses that the domain name www.aliyun.com resolves to.

dig +short www.aliyun.com

Expected output:

www-jp-de-intl-adns.aliyun.com.
www-jp-de-intl-adns.aliyun.com.gds.alibabadns.com.
v6wagbridge.aliyun.com.
v6wagbridge.aliyun.com.gds.alibabadns.com.
106.XX.XX.21
140.XX.XX.4
140.XX.XX.13
140.XX.XX.3

Create the busybox-policy file.

vim busybox-policy.yaml

The following is the template for the busybox-policy file:

# The following is the content of the YAML file.
kind: NetworkPolicy
apiVersion: networking.k8s.io/v1
metadata:
  name: busybox-policy
spec:
  podSelector:
    matchLabels:
      run: busybox
  egress:
  - to:
    - ipBlock:
        cidr: 106.XX.XX.21/32
    - ipBlock:
        cidr: 140.XX.XX.4/32
    - ipBlock:
        cidr: 140.XX.XX.13/32
    - ipBlock:
        cidr: 140.XX.XX.3/32
  - to:
    - ipBlock:
        cidr: 0.0.0.0/0
    - namespaceSelector: {}
    ports:
    - protocol: UDP
      port: 53

Note

In the busybox-policy file, an egress rule is configured to restrict the application's outbound access. You must allow UDP requests here. Otherwise, DNS resolution will fail.

Create a network policy from the busybox-policy file.

kubectl apply -f busybox-policy.yaml

Expected output:

networkpolicy.networking.k8s.io/busybox-policy created

Create busybox.

kubectl run busybox --rm -ti --image=registry.cn-hangzhou.aliyuncs.com/acs/busybox:v1.29.2 /bin/sh

Access a website other than www.aliyun.com, such as www.taobao.com:

wget www.taobao.com

Expected output:

Connecting to www.taobao.com (64.13.XX.XX:80)
wget: can't connect to remote host (64.13.XX.XX): Connection timed out

The message can't connect to remote host indicates that access to the service failed.

Access www.aliyun.com.

wget www.aliyun.com

Expected output:

Connecting to www.aliyun.com (140.205.XX.XX:80)
Connecting to www.aliyun.com (140.205.XX.XX:443)
wget: note: TLS certificate validation not implemented
index.html           100% |***********************************************************|  462k  0:00:00 ETA

When the progress shows 100%, you have successfully accessed the service.

Scenario 4: Use a network policy to control public network access for pods in a namespace

Note

This operation may affect online services that are accessing the public network. We recommend that you perform the following steps in an empty namespace.

kubectl

Create a test namespace to verify the restriction capability.
Create a namespace named test-np.
```
kubectl create ns test-np
```
Expected output:
```
namespace/test-np created
```

Set a default network policy rule for this namespace that only allows outbound access to the private network.

vim default-deny.yaml

The following is an example template for the default-deny.yaml file:

# The following is the content of the YAML file.
kind: NetworkPolicy
apiVersion: networking.k8s.io/v1
metadata:
  namespace: test-np
  name: deny-public-net
spec:
  podSelector: {}
  ingress:
  - from:
    - ipBlock:
        cidr: 0.0.0.0/0
  egress:
  - to:
    - ipBlock:
        cidr: 192.168.0.0/16
    - ipBlock:
        cidr: 172.16.0.0/12
    - ipBlock:
        cidr: 10.0.0.0/8

Check if the default-deny.yaml file has been created successfully.

kubectl apply -f default-deny.yaml

Expected output:

networkpolicy.networking.k8s.io/deny-public-net created

View the network policy:

kubectl get networkpolicy -n test-np

Expected output:

NAME                              POD-SELECTOR          AGE
deny-public-net                   <none>                1m

Allow pods with a special label to access the public network.

vim allow-specify-label.yaml

The example label is public-network=true.

# The following is the content of the YAML file.
kind: NetworkPolicy
apiVersion: networking.k8s.io/v1
metadata:
  name: allow-public-network-for-labels
  namespace: test-np
spec:
  podSelector:
    matchLabels:
      public-network: "true"
  ingress:
  - from:
    - ipBlock:
        cidr: 0.0.0.0/0
  egress:
  - to:
    - ipBlock:
        cidr: 0.0.0.0/0
    - namespaceSelector:
        matchLabels:
          ns: kube-system

Run the following command to create the network policy:

kubectl apply -f allow-specify-label.yaml

Expected output:

networkpolicy.networking.k8s.io/allow-public-network-for-labels created

View the network policy:

kubectl get networkpolicy -n test-np

Expected output:

NAME                              POD-SELECTOR          AGE
allow-public-network-for-labels   public-network=true    1m
deny-public-net                   <none>                 3m

Verify that pods without the special label cannot access the public network.
```
kubectl run -it --namespace test-np --rm --image=registry.cn-hangzhou.aliyuncs.com/acs/busybox:v1.29.2  busybox-intranet
```
```
ping aliyun.com
```
Expected output:
```
PING aliyun.com (106.11.2xx.xxx): 56 data bytes
^C
--- aliyun.com ping statistics ---
9 packets transmitted, 0 packets received, 100% packet loss
```
The message 0 packets received indicates that access failed.
Note
Access fails because the deny-public-net network policy rule restricts default public network access for pods in the test-np namespace. Therefore, pods with default labels that are started in this namespace cannot access the public network.

Verify that pods with the public-network=true label can access the service.

kubectl run -it --namespace test-np --labels public-network=true --rm --image registry.cn-hangzhou.aliyuncs.com/acs/busybox:v1.29.2  busybox-internet

ping aliyun.com

Expected output:

PING aliyun.com (106.11.1xx.xx): 56 data bytes
64 bytes from 106.11.1xx.xx: seq=0 ttl=47 time=4.235 ms
64 bytes from 106.11.1xx.xx: seq=1 ttl=47 time=4.200 ms
64 bytes from 106.11.1xx.xx: seq=2 ttl=47 time=4.182 ms
^C
--- aliyun.com ping statistics ---
3 packets transmitted, 3 packets received, 0% packet loss
round-trip min/avg/max = 4.182/4.205/4.235 ms

The message 0% packet loss indicates that you successfully accessed the service.

Note

Access is successful because the allow-public-network-for-labels network policy rule allows public network access for pods with the public-network=true label. Therefore, the busybox-internet pod, which has this label, can access the public network.