Use network policies in ACS clusters - Container Compute Service

Alibaba Cloud Container Compute Service (ACS) provides policy-based network control using Kubernetes Network Policy. To control network traffic at the IP address or port level, you can apply network policies to specific applications in your cluster. This topic describes how to use network policies in an ACS cluster and provides examples of common scenarios.

Scope

This feature is available only for general-purpose and compute-optimized CPU pods.

Considerations

To use network policies in the console, you must submit an application in the Quota Center console.
You do not need to submit an application if you use the command line to manage network policies.
NetworkPolicy rules allow you to select Namespaces or Pods using a LabelSelector. However, using many NetworkPolicies can increase latency and complicate cluster management and troubleshooting. We recommend that you create fewer than 40 NetworkPolicies in your cluster.

Step 1: Enable network policies

Install the Poseidon component.
1. Log on to the ACS console. In the left-side navigation pane, click Clusters.
2. On the Clusters page, find the target cluster and click its ID. In the left navigation pane, choose Operations > Add-ons.
3. On the Add-ons page, click the Networking tab. On the Poseidon card, click Install.
4. In the Install Poseidon panel, select Enable NetworkPolicy and click OK.
  After the component is installed, Installed appears in the upper-right corner of the card.
Add an annotation to the pod configuration to enable the NetworkPolicy.
To apply a NetworkPolicy to a container, you must add the network.alibabacloud.com/enable-network-policy-agent: "true" annotation to the pod configuration. The following code block shows an example:
```
apiVersion: v1
kind: Pod
metadata:
  annotations:
    network.alibabacloud.com/enable-network-policy-agent: "true"
  name: example
  namespace: default
spec:
  containers:
  - image: nginx
    imagePullPolicy: IfNotPresent
    name: example
```

Step 2: Create an Nginx test application that can be accessed by other pods

Console

On the Clusters page, find the cluster that you want to manage and click its ID. In the left-side navigation pane of the cluster details page, choose Workloads > Deployments.
On the Deployments tab, click Create from Image.

On the Create page, create an application named nginx and expose it using a Service. After you configure the application, click Create.

For this Nginx application, configure only the following parameters and use the default values for the others. For more information, see Create a Deployment.

Page	Configuration item	Description	Example value
Basic Information	Application Name	Enter a custom name.	nginx
Basic Information	Number Of Replicas	Select the appropriate option.	1
Container Configuration	Image Name	Click Select Image, select the image you want, and then click OK.	This example uses the official Nginx image. On the Search tab, select Artifact Center. In the text box, enter `nginx` to search for the image and select the image version you want.
Advanced Configuration	Service	To the right of Service, click Create to set the service configurations.	Name: nginx
			Type: Server Load Balancer Public Network Access Create SLB
			Port Mapping: Name: nginx Service Port: 80 Container Port: 80 Protocol: TCP

Return to the Deployments page and click Create From Image. On the Create page, create a client application named busybox to test access to the nginx Service that you created in the previous step.

For this busybox client application, configure only the following parameters and use the default values for the others. For more information, see Create a Deployment from an image.

Page	Configuration item	Description	Example value
Basic Information	Application Name	Enter a custom name.	busybox
Basic Information	Number Of Replicas	Set the value as needed.	1
Container Configuration	Image Name	Enter the image address directly.	This example uses the image address `registry.cn-hangzhou.aliyuncs.com/acs/busybox:v1.29.2`
Container Configuration	Container Start Item	None	Select stdin and tty

Verify that the busybox client application can access the nginx Service.
1. On the Deployments page, find and click the name of the busybox application.
2. On the Container Group tab, find the busybox-{hash value} pod and click Terminal in the column on the right.
3. In the busybox command-line terminal, run the wget nginx command to test the connection to Nginx.
  Note
  A 200 response indicates that busybox can access the nginx Service.

kubectl

Obtain the cluster's kubeconfig file and use kubectl to connect to the cluster.

Create an Nginx application and expose it using a Service named nginx.

Create the Nginx application:

kubectl run nginx --image=nginx

Expected output:

pod/nginx created

Check whether the pod is running:

kubectl get pod

Expected output:

NAME                     READY   STATUS    RESTARTS   AGE
nginx                    1/1     Running   0          45s

Create the nginx Service:

kubectl expose pod nginx --port=80

Expected output:

service/nginx exposed

View the Service details:

kubectl get service

Expected output:

NAME         TYPE        CLUSTER-IP      EXTERNAL-IP   PORT(S)   AGE
kubernetes   ClusterIP   172.XX.XX.1     <none>        443/TCP   30m
nginx        ClusterIP   172.XX.XX.48    <none>        80/TCP    12s

Create a pod named busybox and access the Service named nginx.

kubectl run busybox --rm -ti --image=registry.cn-hangzhou.aliyuncs.com/acs/busybox:v1.29.2 /bin/sh

Expected output:

If you don't see a command prompt, try pressing enter.
/ #
/ #

Obtain Nginx:

If you don't see a command prompt, try pressing enter.
/ #
/ # wget nginx  # Enter wget nginx here.

Expected output:

Connecting to nginx (172.XX.XX.48:80)
saving to 'index.html'
index.html           100% |****************************************************************************************************************************************************|   612  0:00:00 ETA
'index.html' saved

Step 3: Use a network policy

This section describes common scenarios in which you can use a network policy.

Scenario 1: Restrict access to a service to applications with specific labels

Console

On the Clusters page, find the cluster you want and click its name. In the left-side pane, choose Network > Network Policies.

At the top of the Network Policies page, select the namespace for the network policy. This example uses default. In the upper-right corner of the page, click Create. In the Create panel, configure the network policy.

Configuration item	Description	Example value
Name	Enter a custom name for the network policy.	access-nginx
Pod Selector	Click + Select Workload And Add Labels to specify the pods that use the network policy. Note If you leave the pod selector empty, the network policy applies to all pods in the namespace.	This example uses the following settings: Type: Stateless Set Workload to nginx. Set Label to app=nginx.
Source	Each network policy can contain a whitelist of ingress rules. Each rule allows traffic that matches both the source rules and specific port sections. Rule: podSelector: This selector chooses specific pods in the same namespace as the network policy and allows them as sources for inbound traffic. namespaceSelector: This selector chooses specific namespaces and uses all pods from them as sources for inbound traffic. ipBlock: This selector chooses specific IP CIDR ranges to use as sources for inbound traffic. Port: TCP and UDP protocols are supported. Note If you do not add any rules, no pods are allowed to access the selected pods. If IPvlan is enabled for the cluster, you cannot use ipBlock to restrict traffic from pod CIDR blocks. You must use podSelector.	This example does not add any source rules.
Destination	Each network policy can contain a whitelist of egress rules. Each rule allows traffic that matches both the destination rules and specific port sections. Rule: podSelector: This selector chooses specific pods in the same namespace as the network policy and allows them as destinations for outbound traffic. namespaceSelector: This selector chooses specific namespaces and uses all pods from them as destinations for outbound traffic. ipBlock: This selector chooses specific IP CIDR ranges as destinations for outbound traffic. Port: TCP and UDP protocols are supported. Note If IPvlan is enabled for the cluster, you cannot use ipBlock to restrict traffic from pod CIDR blocks. You must use podSelector.	This example does not add any destination rules.

Click Next and then click OK.
In the busybox command-line terminal, run the wget nginx command to test the connection to the nginx Service. For more information, see Step 2.e.
Note
Because the network policy does not allow access from busybox, the connection times out.
Modify the network policy to allow access from the busybox application.
1. In the list of network policies, find the access-nginx network policy and click Edit in the column on the right.
2. Add a source rule.
  To the right of Source, click +Add and configure the following parameters:
  - To the right of Rule, click +Add. Add a podSelector with the following rule settings:
    Configuration item
    Example value
    Selector
    podSelector
    Type
    stateless
    Workload
    busybox
    Label
    app=busybox
  - To the right of Port, click +Add. Set the port parameter as follows:
    Configuration item
    Example value
    Protocol
    TCP
    Port
    80
3. Click Next and then click OK.
4. Run the wget -O /dev/null nginx command to test the connection from busybox to nginx after you modify the network policy.
  Note
  After the rule for the busybox application is added to the network policy, busybox can access the nginx Service.

kubectl

Use the following YAML template and run the vim policy.yaml command to create a file named policy.yaml.

vim policy.yaml

The following is the content of the YAML file.

kind: NetworkPolicy
apiVersion: networking.k8s.io/v1
metadata:
  name: access-nginx
spec:
  podSelector:
    matchLabels:
      run: nginx
  ingress:
  - from:
    - podSelector:
        matchLabels:
          access: "true"

Create a network policy from the policy.yaml file.

kubectl apply -f policy.yaml

Expected output:

networkpolicy.networking.k8s.io/access-nginx created

Without an access label defined, test access to the nginx Service. The connection times out and fails.

kubectl run busybox --rm -ti --image=registry.cn-hangzhou.aliyuncs.com/acs/busybox:v1.29.2 /bin/sh

Test access to the nginx Service:

wget nginx

Expected output:

Connecting to nginx (172.19.XX.XX:80)
wget: can't connect to remote host (172.19.XX.XX): Connection timed out

Define the access label.

kubectl run busybox --rm -ti --labels="access=true" --image=registry.cn-hangzhou.aliyuncs.com/acs/busybox:v1.29.2 /bin/sh

Test access to the nginx Service:

wget nginx

Expected output:

Connecting to nginx (172.21.XX.XX:80)
saving to 'index.html'
index.html           100% |****************************************************************************************************************************************************|   612  0:00:00 ETA
'index.html' saved

A 100% connection progress indicates that the request is successful and you can access the nginx Service.

Scenario 2: Restrict the source CIDR blocks that can access an Internet-facing service

Console

Set a network policy for the nginx Service. For more information about the parameters, see Restrict access to a service to applications with specific labels.
In the External Endpoint column of the service list, find the public IP address (47.xxx.xx.x) of the nginx Service that you created in Step 2: Create a test Nginx application that can be accessed by other pods. Then, try to access the IP address from a browser.
Note
Because a network policy is applied to the nginx Service, access fails by default.
Add an allowed CIDR block to the network policy to permit client access.
1. Obtain your local public IP address.
2. In the list of network policies, find the access-nginx network policy and click Edit in the column on the right. In the Edit panel, modify the rule.
  To the right of Source, click +Add and configure the following parameters:
  - To the right of Rule, click +Add. Add a rule to allow access from your local public IP address:
    Configuration item
    Example value
    Selector
    ipBlock
    cidr
    <Your native IP address>/32
    For example, 42.xx.xx.xx/32
  - To the right of Rule, click +Add. Add a rule to allow access from the Alibaba Cloud SLB health check CIDR block:
    Configuration item
    Example value
    Selector
    ipBlock
    cidr
    100.0.0.0/8
  - To the right of Port, click +Add. Set the port parameter as follows:
    Configuration item
    Example value
    Protocol
    TCP
    Port
    80
3. Click Next and then click OK.
4. In the External Endpoint column of the service list, click the external endpoint IP address (47.xxx.xx.x:80) to access the nginx Service.
  Note
  After the network policy is updated, the client can access the nginx Service through the Internet-facing SLB.

kubectl

Create an Alibaba Cloud SLB service for the nginx application. Specify type=LoadBalancer to expose the nginx Service to the Internet.

vim nginx-service.yaml

The following is a template for the nginx-service.yaml file.

# Paste the following YAML content into nginx-service.yaml.
apiVersion: v1
kind: Service
metadata:
  labels:
    run: nginx
  name: nginx-slb
spec:
  externalTrafficPolicy: Local
  ports:
  - port: 80
    protocol: TCP
    targetPort: 80
  selector:
    run: nginx
  type: LoadBalancer

Create a network policy from the nginx-service.yaml file.

kubectl apply -f nginx-service.yaml

Expected output:

service/nginx-slb created

Check whether the application exposes the Nginx service to the public network:

kubectl get service nginx-slb

Expected output:

NAME        TYPE           CLUSTER-IP      EXTERNAL-IP      PORT(S)        AGE
nginx-slb   LoadBalancer   172.19.xx.xxx   47.110.xxx.xxx   80:32240/TCP   8m

Access the IP address of the newly created SLB, 47.110.xxx.xxx. The connection fails.
```
wget 47.110.xxx.xxx
```
Expected output:
```
--2018-11-21 11:46:05--  http://47.110.xx.xxx/
Connecting to 47.110.XX.XX:80... failed: Connection refused.
```
Note
Access fails for the following reasons:
The configured nginx Service can be accessed only by applications that have the specific label access=true.
Accessing the SLB's IP address is an external access to Kubernetes, which is different from restricting service access to applications with specific labels using a network policy.
Solution: Modify the network policy to add the allowed source CIDR block.

View the local IP address.

curl myip.ipip.net

Expected output:

Current IP: 10.0.x.x From: China Beijing Beijing        # This is an example. The actual device information prevails.

Modify the created policy.yaml file.

vim policy.yaml

Modify the policy.yaml file to have the following content:

# The following is the content of the YAML file.
kind: NetworkPolicy
apiVersion: networking.k8s.io/v1
metadata:
  name: access-nginx
spec:
  podSelector:
    matchLabels:
      run: nginx
  ingress:
  - from:
    - podSelector:
        matchLabels:
          access: "true"
    - ipBlock:
        cidr: 100.64.0.0/10
    - ipBlock:
        cidr: 10.0.0.1/24      # Local IP address. This is an example. The actual device information prevails.

Create a network policy from the policy.yaml file.

kubectl apply -f policy.yaml

Expected output:

networkpolicy.networking.k8s.io/access-nginx unchanged

Note

Use an address range of /24 because some networks have multiple outbound IP addresses.
The SLB health check address is within the 100.64.0.0/10 address range. For this reason, you must add 100.64.0.0/10 to the allowed CIDR blocks.

Create an Nginx service.

kubectl run busybox --rm -ti --labels="access=true" --image=registry.cn-hangzhou.aliyuncs.com/acs/busybox:v1.29.2 /bin/sh

Access the Nginx service:

wget 47.110.XX.XX

Expected output:

Connecting to 47.110.XX.XX (47.110.XX.XX:80)
index.html           100% |***********************************************************|   612  0:00:00 ETA

A 100% connection progress indicates that you have successfully accessed the Nginx service.

Scenario 3: Restrict a pod to access only a specific address

Console

This section uses www.aliyun.com and registry.aliyuncs.com as examples to show how to configure a network policy that allows a pod to access only registry.aliyuncs.com.

Use the ping command to query the IP address (120.55.XXX.XXX) of registry.aliyuncs.com.

Create a network policy rule that allows the busybox application to access only registry.aliyuncs.com.

In the upper-right corner of the Network Policies page, click Create. In the Create panel, configure the network policy rule.

For more information about the parameter descriptions, see Restrict access to a service to applications with specific labels. The following is an example configuration.

Configuration item	Description	Example value
Name	Enter a custom name for the network policy.	busybox-policy
Pod Selector	Click + Select Workload And Add Labels to specify the pods that use the network policy. Note If you leave the pod selector empty, the network policy applies to all pods in the namespace.	This example uses the following settings: Type: stateless Set Workload to busybox. Set Label to app=busybox.
Destination	To the right of Destination, click +Add. Then, to the right of Rule, click +Add. Add an ipBlock rule for the parsed IP address (120.55.XXX.XXX)/32 of registry.aliyuncs.com that you obtained earlier.	Selector: ipBlock cidr: 120.55.XXX.XXX/32
Destination	To the right of Destination, click +Add. Add a rule for access to UDP port 53 for all namespaces using the namespace selector. This ensures that the application can resolve DNS. To the right of Rule, click +Add. Add a rule for all namespaces using the namespace selector. To the right of Port, click +Add. Add a rule for UDP 53 to ensure that the application can resolve DNS.	Rule: Selector: namespaceSelector Namespace: All Port: Protocol: UDP Port: 53

Click Next and then click OK.
In the busybox terminal, try to access www.aliyun.com and registry.aliyuncs.com.
```
nc -vz -w 1 www.aliyunc.com 443
```
```
nc -vz -w 1 registry.aliyuncs.com 443
```
The result shows that after the network policy is applied, the busybox application can access only registry.aliyuncs.com and cannot access other addresses.

kubectl

Retrieve the IP addresses to which the www.aliyun.com domain name resolves.

dig +short www.aliyun.com

Expected output:

www-jp-de-intl-adns.aliyun.com.
www-jp-de-intl-adns.aliyun.com.gds.alibabadns.com.
v6wagbridge.aliyun.com.
v6wagbridge.aliyun.com.gds.alibabadns.com.
106.XX.XX.21
140.XX.XX.4
140.XX.XX.13
140.XX.XX.3

Create the busybox-policy file.

vim busybox-policy.yaml

The following is a template for the busybox-policy file:

# The following is the content of the YAML file.
kind: NetworkPolicy
apiVersion: networking.k8s.io/v1
metadata:
  name: busybox-policy
spec:
  podSelector:
    matchLabels:
      run: busybox
  egress:
  - to:
    - ipBlock:
        cidr: 106.XX.XX.21/32
    - ipBlock:
        cidr: 140.XX.XX.4/32
    - ipBlock:
        cidr: 140.XX.XX.13/32
    - ipBlock:
        cidr: 140.XX.XX.3/32
  - to:
    - ipBlock:
        cidr: 0.0.0.0/0
    - namespaceSelector: {}
    ports:
    - protocol: UDP
      port: 53

Note

The egress rules in the busybox-policy file restrict the outbound access of the application. You must allow UDP requests to enable DNS resolution.

Create a network policy from the busybox-policy file.

kubectl apply -f busybox-policy.yaml

Expected output:

networkpolicy.networking.k8s.io/busybox-policy created

Create busybox.

kubectl run busybox --rm -ti --image=registry.cn-hangzhou.aliyuncs.com/acs/busybox:v1.29.2 /bin/sh

Access a website other than www.aliyun.com, such as www.taobao.com:

wget www.taobao.com

Expected output:

Connecting to www.taobao.com (64.13.XX.XX:80)
wget: can't connect to remote host (64.13.XX.XX): Connection timed out

The message can't connect to remote host indicates that the connection failed.

Access www.aliyun.com.

wget www.aliyun.com

Expected output:

Connecting to www.aliyun.com (140.205.XX.XX:80)
Connecting to www.aliyun.com (140.205.XX.XX:443)
wget: note: TLS certificate validation not implemented
index.html           100% |***********************************************************|  462k  0:00:00 ETA

A result of 100% indicates that the connection was successful.

Scenario 4: Control the public network access permissions of pods in a namespace

Note

This operation may affect online services that are accessing the Internet. We recommend that you perform the following operations in an empty namespace.

Console

In the upper-right corner of the Network Policies page, click Create. In the Create panel, configure the network policy rule.

For more information about the parameter descriptions and operations, see Restrict access to a service to applications with specific labels. The following is an example configuration.

Configuration item	Example value
Name	deny-public-net
Pod Selector	Set Type to All.
Source	Add the following two rules: Set a rule to allow all for namespaceSelector. Set a rule to allow all for ipBlock.
Destination	Add a rule that only allows access to the internal network: Set a rule to allow All for namespaceSelector. This allows the pod to access all pods on the internal network. Set three rules for internal network segments for ipBlock, including the following three internal network segments: 10.0.0.0/8 172.16.0.0/12 192.168.0.0/16 Note You cannot add multiple network segments in one ipBlock rule.

Click Next and then click OK.
On the Basic Information tab of the cluster details page, obtain the internal IP address and port.
In the busybox terminal, run the following commands to test the pod's access to the public network and the internal network.
```
nc -vz -w 1 www.aliyunc.com 443
```
```
nc -vz -w 1 10.xx.xx.xx:<IP port> # This IP address is your internal address.
```
Note
The result shows that the pod can access only internal addresses and cannot access public addresses.

kubectl

Create a test namespace to verify the restriction capability.
Create a namespace named test-np.
```
kubectl create ns test-np
```
Expected output:
```
namespace/test-np created
```

Set a default network policy rule for this namespace that allows only outbound access to the private network.

vim default-deny.yaml

The following is an example of the default-deny.yaml template:

# The following is the content of the YAML file.
kind: NetworkPolicy
apiVersion: networking.k8s.io/v1
metadata:
  namespace: test-np
  name: deny-public-net
spec:
  podSelector: {}
  ingress:
  - from:
    - ipBlock:
        cidr: 0.0.0.0/0
  egress:
  - to:
    - ipBlock:
        cidr: 192.168.0.0/16
    - ipBlock:
        cidr: 172.16.0.0/12
    - ipBlock:
        cidr: 10.0.0.0/8

Check whether the default-deny.yaml file has been created.

kubectl apply -f default-deny.yaml

Expected output:

networkpolicy.networking.k8s.io/deny-public-net created

View the network policy:

kubectl get networkpolicy -n test-np

Expected output:

NAME                              POD-SELECTOR          AGE
deny-public-net                   <none>                1m

Configure the policy to allow public network access for pods that have specific labels.

vim allow-specify-label.yaml

The example label is set to public-network=true.

# The following is the content of the YAML file.
kind: NetworkPolicy
apiVersion: networking.k8s.io/v1
metadata:
  name: allow-public-network-for-labels
  namespace: test-np
spec:
  podSelector:
    matchLabels:
      public-network: "true"
  ingress:
  - from:
    - ipBlock:
        cidr: 0.0.0.0/0
  egress:
  - to:
    - ipBlock:
        cidr: 0.0.0.0/0
    - namespaceSelector:
        matchLabels:
          ns: kube-system

Run the following command to create the network policy:

kubectl apply -f allow-specify-label.yaml

Expected output:

networkpolicy.networking.k8s.io/allow-public-network-for-labels created

View the network policy:

kubectl get networkpolicy -n test-np

Expected output:

NAME                              POD-SELECTOR          AGE
allow-public-network-for-labels   public-network=true    1m
deny-public-net                   <none>                 3m

Verify that pods without special labels cannot access the public network.
```
kubectl run -it --namespace test-np --rm --image=registry.cn-hangzhou.aliyuncs.com/acs/busybox:v1.29.2  busybox-intranet
```
```
ping aliyun.com
```
Expected output:
```
PING aliyun.com (106.11.2xx.xxx): 56 data bytes
^C
--- aliyun.com ping statistics ---
9 packets transmitted, 0 packets received, 100% packet loss
```
The message 0 packets received indicates that the access attempt failed.
Note
Access failed because the deny-public-net network policy rule denies public network access for pods in the test-np namespace by default. Therefore, pods that are started in this namespace with default labels cannot access the public network.

Verify that pods with the public-network=true label can access the service.

kubectl run -it --namespace test-np --labels public-network=true --rm --image registry.cn-hangzhou.aliyuncs.com/acs/busybox:v1.29.2  busybox-internet

ping aliyun.com

Expected output:

PING aliyun.com (106.11.1xx.xx): 56 data bytes
64 bytes from 106.11.1xx.xx: seq=0 ttl=47 time=4.235 ms
64 bytes from 106.11.1xx.xx: seq=1 ttl=47 time=4.200 ms
64 bytes from 106.11.1xx.xx: seq=2 ttl=47 time=4.182 ms
^C
--- aliyun.com ping statistics ---
3 packets transmitted, 3 packets received, 0% packet loss
round-trip min/avg/max = 4.182/4.205/4.235 ms

The message 0% packet loss indicates that the connection was successful.

Note

Access is successful because the allow-public-network-for-labels network policy rule allows public network access for pods that have the public-network=true label. Therefore, the busybox-internet pod, which has this label, can access the public network.

Configuration item	Example value
Selector	podSelector
Type	stateless
Workload	busybox
Label	app=busybox