Use an egress traffic policy to manage egress traffic - Alibaba Cloud Service Mesh

Service Mesh (ASM) instances of version 1.16.4 and later allow you to use CustomResourceDefinition (CRD) fields to define an egress traffic policy. This topic describes how to use an egress traffic policy to manage egress traffic.

Prerequisites

An ASM instance of a commercial edition (Enterprise Edition or Ultimate Edition) is created. The version of the instance is 1.16.4 or later. For more information about how to create an ASM instance, see Create an ASM instance. For more information about how to update an ASM instance, see Update an ASM instance.
The cluster is added to the ASM instance.
Automatic sidecar proxy injection is enabled for the default namespace. For more information, see Enable automatic sidecar injection.

Feature description

ASM allows you to establish, manage, and protect communication between applications in a unified manner. ASM uses applications instead of IP addresses as the unit to which communication is allowed or denied, and you do not need to modify existing application code. An egress traffic policy defines how an egress gateway manages egress traffic. By using a combination of ASM egress gateways and authorization policies, you can control egress traffic in a flexible manner. Dingtalk_20230411112608

Preparations

1. Set an egress traffic policy for the ASM instance

Log on to the ASM console. In the left-side navigation pane, choose Service Mesh > Mesh Management.
On the Mesh Management page, click the name of the ASM instance. In the left-side navigation pane, choose Dataplane Component Management > Sidecar Proxy Setting.
On the global tab, click Outbound Traffic Policy, set the Outbound Traffic Policy parameter to REGISTEY_ONLY, and then click Update Settings.

2. Create a namespace

Create a namespace named istio-egress. For more information, see Manage global namespaces.
On the Global Namespace page, click Sync Automatic Sidecar Injection to Kubernetes Cluster to synchronize the namespace to the ACK cluster that is added to the ASM instance.

3. Create an egress gateway

Create an egress gateway named egressgateway-a in the ASM console. In the Port Mapping section, add HTTP 80, HTTPS 443, and HTTPS 444, and select Support two-way TLS authentication for the egress gateway. For more information, see Create an egress gateway.

Make sure that the following content is included in the spec section in the generated YAML file of the egress gateway. If the YAML file does not contain the following content, add the content to the file.

spec:
  podLabels:
    security.istio.io/tlsMode: istio

4. Create services

Create a namespace named mytest for the ASM instance and enable automatic sidecar proxy injection. For more information, see Manage global namespaces.

In the ACK cluster, deploy the sleep-a service in the mytest namespace and deploy the nginx service in the default namespace.

Create the test.yaml file that contains the following content:

Expand to view the YAML file

apiVersion: v1
kind: Service
metadata:
  name: sleep-a
  namespace: mytest
  labels:
    app: sleep-a
    service: sleep-a
spec:
  ports:
  - port: 80
    name: http
  selector:
    app: sleep-a
---
apiVersion: apps/v1
kind: Deployment
metadata:
  name: sleep-a
  namespace: mytest
spec:
  replicas: 1
  selector:
    matchLabels:
      app: sleep-a
  template:
    metadata:
      labels:
        app: sleep-a
    spec:
      terminationGracePeriodSeconds: 0
      containers:
      - name: sleep
        image: curlimages/curl
        command: ["/bin/sleep", "infinity"]
        imagePullPolicy: IfNotPresent
        volumeMounts:
        - mountPath: /etc/sleep/tls
          name: secret-volume
      volumes:
      - name: secret-volume
        secret:
          secretName: sleep-secret
          optional: true

---
apiVersion: apps/v1
kind: Deployment
metadata:
  labels:
    app: nginx
  name: nginx
  namespace: default
spec:
  progressDeadlineSeconds: 600
  replicas: 1
  revisionHistoryLimit: 10
  selector:
    matchLabels:
      app: nginx
  strategy:
    rollingUpdate:
      maxSurge: 25%
      maxUnavailable: 25%
    type: RollingUpdate
  template:
    metadata:
      creationTimestamp: null
      labels:
        app: nginx
    spec:
      containers:
      - image: nginx
        imagePullPolicy: Always
        name: nginx
        resources: {}
        terminationMessagePath: /dev/termination-log
        terminationMessagePolicy: File
      dnsPolicy: ClusterFirst
      restartPolicy: Always
      schedulerName: default-scheduler
      securityContext: {}
      terminationGracePeriodSeconds: 30

In the ACK cluster, run the following command to deploy the sleep-a and nginx services:
```
kubectl apply -f test.yaml
```

Run the following commands to use the sleep-a and nginx services to access http://www.httpbin.org:

kubectl -n mytest exec deployment/sleep-a -- curl -s -o /dev/null -w "%{http_code}\n" http://www.httpbin.org
kubectl -n default exec deployment/nginx -- curl -s -o /dev/null -w "%{http_code}\n" http://www.httpbin.org

The return codes are both 502, which indicates that the access fails.

Scenario 1: Allow only the sleep-a service in the mytest namespace to access http://www.httpbin.org

Create the egress-by-egressgateway-a.yaml file that contains the following content.

For more information about the fields in the following content, see CRD fields in an egress traffic policy.

apiVersion: istio.alibabacloud.com/v1
kind: ASMEgressTrafficPolicy
metadata:
  name: egress-by-egressgateway-a  # The format of this value is egress-by-{name of the egress gateway}. This value corresponds to the name of the egress gateway. 
  namespace: istio-egress         # This value is fixed to istio-egress. 
spec:
  byEgressGateway:
    name: egressgateway-a
  egressRules:
  - from:
    - namespace: mytest
      workloadSelector:
        app: sleep-a
    to:
    - name: httpbin-service-http
      hosts:
      - www.httpbin.org  # The IP addresses of multiple domain names after Domain Name System (DNS) resolution must be the same. 
      - httpbin.org      # The IP addresses of multiple domain names after DNS resolution must be the same. 
      port:
        name: http
        number: 80
        protocol: HTTP
      byEgressGateway:
        port: 80        # Traffic path: sidecar proxy > gateway (port 80) > httpbin.org service (port 80)

In the ACK cluster, run the following command to create an egress traffic policy:
```
kubectl apply -f egress-by-egressgateway-a.yaml
```
Check whether the egress traffic policy takes effect.
1. Run the following command to use the nginx service in the default namespace to access http://www.httpbin.org:
```
kubectl -n default exec deployment/nginx -- curl -s -o /dev/null -w "%{http_code}\n" http://www.httpbin.org
```
  The 502 code is returned, which indicates that the nginx service fails to access http://www.httpbin.org.
2. Run the following command to use the sleep-a service in the mytest namespace to access http://www.httpbin.org:
```
kubectl -n mytest exec deployment/sleep-a -- curl  -s -o /dev/null -w "%{http_code}\n"  http://www.httpbin.org
```
  The 200 code is returned, which indicates that the access succeeds. This meets expectations.
3. Run the following commands to delete the egress-by-egressgateway-a policy and use the sleep-a service in the mytest namespace to access http://www.httpbin.org again:
```
kubectl -n istio-egress delete ASMEgressTrafficPolicy  egress-by-egressgateway-a
kubectl -n mytest exec deployment/sleep-a -- curl -s -o /dev/null -w "%{http_code}\n" http://www.httpbin.org
```
  The 502 code is returned, which indicates that the sleep-a service fails to access http://www.httpbin.org after the egress traffic policy is deleted.
The results indicate that the egress traffic policy takes effect.

Scenario 2: Upgrade HTTP requests to HTTPS requests and allow only the sleep-a service in the mytest namespace to directly access http://www.httpbin.org and https://www.httpbin.org

Create the egress-by-egressgateway-a.yaml file that contains the following content.

In the spec section, add the httpsUpgrade fields and the CRD fields that define direct access to https://www.httpbin.org. For more information about these fields, see CRD fields in an egress traffic policy.

apiVersion: istio.alibabacloud.com/v1
kind: ASMEgressTrafficPolicy
metadata:
  name: egress-by-egressgateway-a  # The format of this value is egress-by-{name of the egress gateway}. This value corresponds to the name of the egress gateway. 
  namespace: istio-egress          # This value is fixed to istio-egress. 
spec:
  byEgressGateway:
    name: egressgateway-a
  egressRules:
  - from:
     - namespace: mytest
       workloadSelector:
          app: sleep-a
    to:
    - name: httpbin-service-http
      hosts:
      - www.httpbin.org  # The IP addresses of multiple domain names after DNS resolution must be the same. 
      - httpbin.org      # The IP addresses of multiple domain names after DNS resolution must be the same. 
      port:
        name: http
        number: 80
        protocol: HTTP
      byEgressGateway:
        port: 80        # Traffic path: sidecar proxy > gateway (port 80) > httpbin.org service (port 80)
      httpsUpgrade:
        enabled: true # If this value is set to false, the value of the port parameter under httpsUpgrade does not take effect. 
        port: 443       # Traffic path: sidecar proxy > gateway (port 80) > httpbin.org service (port 443)
    - name: httpbin-service-https
      hosts:
      - www.httpbin.org
      - httpbin.org
      port:
        name: https
        number: 443
        protocol: HTTPS
      byEgressGateway:
        port: 444 # HTTP port 444 that is configured in Step 3 in the Preparations section.

In the ACK cluster, run the following command to create an egress traffic policy:
```
kubectl apply -f egress-by-egressgateway-a.yaml
```
Check whether the egress traffic policy takes effect.
1. Check the sleep-a service in the mytest namespace.
  1. Run the following command to use the sleep-a service to access http://www.httpbin.org:
```
kubectl -n mytest exec deployment/sleep-a -- curl -s -o /dev/null -w "%{http_code}\n" http://httpbin.org
```
    The 200 code is returned, which indicates that the sleep-a service succeeds in accessing http://www.httpbin.org.
  2. Run the following command to use the sleep-a service to access httpbin.org over the anything port. Then, check whether the egress gateway upgrades HTTP requests to HTTPS requests.
```
kubectl -n mytest exec deployment/sleep-a -- sh -c "curl -s  http://httpbin.org/anything |grep url"
```
    Expected output:
```
"url": "https://httpbin.org/anything"
```
    If the url field starts with https, the requests are upgraded to HTTPS requests by the egress gateway and then forwarded to httpbin.org.
  3. Run the following command to use the sleep-a service in the mytest namespace to access https://www.httpbin.org:
```
kubectl -n mytest exec deployment/sleep-a -- curl  -s -o /dev/null -w "%{http_code}\n"  https://www.httpbin.org
```
    The 200 code is returned, which indicates that the sleep-a service succeeds in accessing https://www.httpbin.org.
2. Check the nginx service in the default namespace.
  1. Run the following command to use the nginx service to access http://www.httpbin.org:
```
kubectl -n default exec deployment/nginx -- curl -s -o /dev/null -w "%{http_code}\n" http://www.httpbin.org
```
    The 502 code is returned, which indicates that the nginx service fails to access http://www.httpbin.org.
  2. Run the following command to use the nginx service to access https://www.httpbin.org:
```
kubectl -n default exec deployment/nginx -- curl -s -o /dev/null -w "%{http_code}\n" https://www.httpbin.org
```
    The output shows that the requests are rejected, which indicates that the nginx service fails to access https://www.httpbin.org.
  3. Run the following command to view the access log of the sidecar proxy that corresponds to the nginx service workload:
```
 kubectl -n default  logs -f deployment/nginx -c istio-proxy   --tail=1 
```
    Expected output:
```
{"authority":"-","bytes_received":"0","bytes_sent":"0","downstream_local_address":"52.86.68.46:443","downstream_remote_address":"172.16.0.199:56748","duration":"0","istio_policy_status":"-","method":"-","path":"-","protocol":"-","request_id":"-","requested_server_name":"-","response_code":"0","response_flags":"UH","route_name":"-","start_time":"2023-04-11T02:00:07.409Z","trace_id":"-","upstream_cluster":"BlackHoleCluster","upstream_host":"-","upstream_local_address":"-","upstream_service_time":"-","upstream_transport_failure_reason":"-","user_agent":"-","x_forwarded_for":"-"}
```
    The output indicates that the requests are forwarded to BlackHoleCluster, and therefore the requests are rejected.
3. Run the following commands to delete the egress-by-egressgateway-a policy and use the sleep-a service in the mytest namespace to access http://www.httpbin.org and https://www.httpbin.org again.
```
kubectl -n istio-egress delete ASMEgressTrafficPolicy  egress-by-egressgateway-a
kubectl -n mytest exec deployment/sleep-a -- curl -s -o /dev/null -w "%{http_code}\n"  http://www.httpbin.org
kubectl -n mytest exec deployment/sleep-a -- curl -s -o /dev/null -w "%{http_code}\n"  https://www.httpbin.org
```
  When the service accesses http://www.httpbin.org, 502 is returned. When the service accesses https://www.httpbin.org, the requests are rejected. After the egress traffic policy is deleted, the sleep-a service cannot access http://www.httpbin.org or https://www.httpbin.org.
The results indicate that the egress traffic policy takes effect.

Related operations

Reject POST requests sent from a specified namespace

By using an egress gateway and an egress traffic policy, you can flexibly control outbound traffic in a cluster. By using an authorization policy together, you can implement more fine-grained access control. For example, you can use the following policy to reject POST requests sent from the mytest namespace:

kind: AuthorizationPolicy
apiVersion: security.istio.io/v1beta1
metadata:
  name: sleep-a-egress-www-httpbin-org
  namespace: istio-system
spec:
  action: DENY
  rules:
    - to:
        - operation:
            hosts:
            - www.httpbin.org
            - httpbin.org
            methods:
              - POST
      from:
        - source:
            namespaces: ["mytest"]
  selector:
    matchLabels:
      istio: egressgateway-a

After the preceding configuration is applied, RBAC: access is returned when POST requests are sent from the sleep-a service to access www.httpbin.org. GET requests to access www.httpbin.org are not affected.