All Products
Search
Document Center

Alibaba Cloud Service Mesh:Enable a CNI plug-in to improve security

Last Updated:Feb 28, 2024

Service Mesh (ASM) allows you to use Container Network Interface (CNI) plug-ins to remove iptables rules from pods. A CNI plug-in does not require that you have elevated Kubernetes role-based access control (RBAC) permissions. This reduces the requirements for user permissions and improves the security of ASM. This topic describes how to enable a CNI plug-in.

Prerequisites

Background information

To enable an ASM instance to work as expected, you must inject an Envoy proxy into each pod of the ASM instance. Then, you must use iptables rules to manage the traffic of each pod so that the injected Envoy proxy can redirect the traffic to the specified applications. The iptables rules of each pod belong to the network namespace of the pod. Therefore, changes to the iptables rules of a pod do not affect the other pods on the same node.

By default, the istio-init container is injected into the pods that are deployed in an ASM instance. In addition, iptables rules are configured before other containers in the pods are started. This requires that you have sufficient permissions to deploy containers, including the permissions to deploy the containers that require the NET_ADMIN capability and to reconfigure the network.

ASM allows you to use CNI plug-ins to remove iptables rules from pods. A CNI plug-in does not require that you have elevated Kubernetes RBAC permissions. You can use a CNI plug-in to configure pod traffic redirection in the network setup phase of the pod lifecycle. In this case, pods no longer need to include the istio-init container that requires the NET_ADMIN capability. After a CNI plug-in is enabled, its configuration is added to the existing CNI plug-ins of containers so that the CNI plug-in can be called when the containers are started.

A CNI plug-in identifies pods that require traffic redirection by checking whether the pods meet all of the following conditions:

  • The namespace of the pod is not contained in the value of the excludeNamespaces parameter.

  • The pod contains a container named istio-proxy.

  • The pod contains multiple containers.

  • The pod has no annotation whose key is sidecar.istio.io/inject, or the value of the sidecar.istio.io/inject annotation is not true.

Enable a CNI plug-in

You can enable a CNI plug-in for an ASM instance in the ASM console. Perform the following steps:

  1. Log on to the ASM console.

  2. On the Mesh Management page, click the name of the ASM instance. In the left-side navigation pane, choose Dataplane Component Management > ASM CNI Plug-in.

  3. On the ASM CNI Plug-in page, turn on Enable Grid CNI Plugin, select the namespaces that you want to exclude, and then click Update Settings.

    Pods in the excluded namespaces use the istio-init container rather than the CNI plug-in for network configuration. When the value in the Status column that corresponds to the ASM instance changes from Updating to Running, the CNI plug-in is enabled.

Verify iptables rules

In this example, a bookinfo application is deployed in a cluster to check whether iptables rules take effect.

  1. Create a bookinfo.yaml file that contains the following content:

    Show the bookinfo.yaml file

    ##################################################################################################
    # Details service
    ##################################################################################################
    apiVersion: v1
    kind: Service
    metadata:
      name: details
      labels:
        app: details
        service: details
    spec:
      ports:
      - port: 9080
        name: http
      selector:
        app: details
    ---
    apiVersion: v1
    kind: ServiceAccount
    metadata:
      name: bookinfo-details
      labels:
        account: details
    ---
    apiVersion: apps/v1
    kind: Deployment
    metadata:
      name: details-v1
      labels:
        app: details
        version: v1
    spec:
      replicas: 1
      selector:
        matchLabels:
          app: details
          version: v1
      template:
        metadata:
          labels:
            app: details
            version: v1
        spec:
          serviceAccountName: bookinfo-details
          containers:
          - name: details
            image: docker.io/istio/examples-bookinfo-details-v1:1.16.2
            imagePullPolicy: IfNotPresent
            ports:
            - containerPort: 9080
            securityContext:
              runAsUser: 1000
    ---
    ##################################################################################################
    # Ratings service
    ##################################################################################################
    apiVersion: v1
    kind: Service
    metadata:
      name: ratings
      labels:
        app: ratings
        service: ratings
    spec:
      ports:
      - port: 9080
        name: http
      selector:
        app: ratings
    ---
    apiVersion: v1
    kind: ServiceAccount
    metadata:
      name: bookinfo-ratings
      labels:
        account: ratings
    ---
    apiVersion: apps/v1
    kind: Deployment
    metadata:
      name: ratings-v1
      labels:
        app: ratings
        version: v1
    spec:
      replicas: 1
      selector:
        matchLabels:
          app: ratings
          version: v1
      template:
        metadata:
          labels:
            app: ratings
            version: v1
        spec:
          serviceAccountName: bookinfo-ratings
          containers:
          - name: ratings
            image: docker.io/istio/examples-bookinfo-ratings-v1:1.16.2
            imagePullPolicy: IfNotPresent
            ports:
            - containerPort: 9080
            securityContext:
              runAsUser: 1000
    ---
    ##################################################################################################
    # Reviews service
    ##################################################################################################
    apiVersion: v1
    kind: Service
    metadata:
      name: reviews
      labels:
        app: reviews
        service: reviews
    spec:
      ports:
      - port: 9080
        name: http
      selector:
        app: reviews
    ---
    apiVersion: v1
    kind: ServiceAccount
    metadata:
      name: bookinfo-reviews
      labels:
        account: reviews
    ---
    apiVersion: apps/v1
    kind: Deployment
    metadata:
      name: reviews-v1
      labels:
        app: reviews
        version: v1
    spec:
      replicas: 1
      selector:
        matchLabels:
          app: reviews
          version: v1
      template:
        metadata:
          labels:
            app: reviews
            version: v1
        spec:
          serviceAccountName: bookinfo-reviews
          containers:
          - name: reviews
            image: docker.io/istio/examples-bookinfo-reviews-v1:1.16.2
            imagePullPolicy: IfNotPresent
            env:
            - name: LOG_DIR
              value: "/tmp/logs"
            ports:
            - containerPort: 9080
            volumeMounts:
            - name: tmp
              mountPath: /tmp
            - name: wlp-output
              mountPath: /opt/ibm/wlp/output
            securityContext:
              runAsUser: 1000
          volumes:
          - name: wlp-output
            emptyDir: {}
          - name: tmp
            emptyDir: {}
    ---
    apiVersion: apps/v1
    kind: Deployment
    metadata:
      name: reviews-v2
      labels:
        app: reviews
        version: v2
    spec:
      replicas: 1
      selector:
        matchLabels:
          app: reviews
          version: v2
      template:
        metadata:
          labels:
            app: reviews
            version: v2
        spec:
          serviceAccountName: bookinfo-reviews
          containers:
          - name: reviews
            image: docker.io/istio/examples-bookinfo-reviews-v2:1.16.2
            imagePullPolicy: IfNotPresent
            env:
            - name: LOG_DIR
              value: "/tmp/logs"
            ports:
            - containerPort: 9080
            volumeMounts:
            - name: tmp
              mountPath: /tmp
            - name: wlp-output
              mountPath: /opt/ibm/wlp/output
            securityContext:
              runAsUser: 1000
          volumes:
          - name: wlp-output
            emptyDir: {}
          - name: tmp
            emptyDir: {}
    ---
    apiVersion: apps/v1
    kind: Deployment
    metadata:
      name: reviews-v3
      labels:
        app: reviews
        version: v3
    spec:
      replicas: 1
      selector:
        matchLabels:
          app: reviews
          version: v3
      template:
        metadata:
          labels:
            app: reviews
            version: v3
        spec:
          serviceAccountName: bookinfo-reviews
          containers:
          - name: reviews
            image: docker.io/istio/examples-bookinfo-reviews-v3:1.16.2
            imagePullPolicy: IfNotPresent
            env:
            - name: LOG_DIR
              value: "/tmp/logs"
            ports:
            - containerPort: 9080
            volumeMounts:
            - name: tmp
              mountPath: /tmp
            - name: wlp-output
              mountPath: /opt/ibm/wlp/output
            securityContext:
              runAsUser: 1000
          volumes:
          - name: wlp-output
            emptyDir: {}
          - name: tmp
            emptyDir: {}
    ---
    ##################################################################################################
    # Productpage services
    ##################################################################################################
    apiVersion: v1
    kind: Service
    metadata:
      name: productpage
      labels:
        app: productpage
        service: productpage
    spec:
      ports:
      - port: 9080
        name: http
      selector:
        app: productpage
    ---
    apiVersion: v1
    kind: ServiceAccount
    metadata:
      name: bookinfo-productpage
      labels:
        account: productpage
    ---
    apiVersion: apps/v1
    kind: Deployment
    metadata:
      name: productpage-v1
      labels:
        app: productpage
        version: v1
    spec:
      replicas: 1
      selector:
        matchLabels:
          app: productpage
          version: v1
      template:
        metadata:
          labels:
            app: productpage
            version: v1
        spec:
          serviceAccountName: bookinfo-productpage
          containers:
          - name: productpage
            image: docker.io/istio/examples-bookinfo-productpage-v1:1.16.2
            imagePullPolicy: IfNotPresent
            ports:
            - containerPort: 9080
            volumeMounts:
            - name: tmp
              mountPath: /tmp
            securityContext:
              runAsUser: 1000
          volumes:
          - name: tmp
            emptyDir: {}
    ---
                            
  2. Run the following command to deploy the bookinfo application:

    kubectl apply -f bookinfo.yaml
  3. Run the following commands to obtain the ID of the container and the name of the node on which the pod of the Productpage service runs:

    ns=default
    podname=kubectl get pod |grep productpage
    # Run the following command if the container runtime is Docker: 
    container_id=$(kubectl get pod -n ${ns} ${podname} -o jsonpath="{.status.containerStatuses[?(@.name=='istio-proxy')].containerID}" | sed -n 's/docker:\/\/\(.*\)/\1/p')
    # Run the following command if the container runtime is Containerd: 
    container_id=$(kubectl get pod -n ${ns} ${podname} -o jsonpath="{.status.containerStatuses[?(@.name=='istio-proxy')].containerID}" | sed -n 's/containerd:\/\/\(.*\)/\1/p')
    echo $container_id
    
    # Obtain the name of the node. 
    kubectl get pod ${podname} -o jsonpath="{.spec.nodeName}"
  4. Log on to the node on which the pod of the Productpage service runs. For example, you can run the SSH command. After logon, run the following command to obtain the process that corresponds to the ID of the container:

    # Run the following command if the container runtime is Docker: 
    docker inspect --format '{{ .State.Pid }}' $container_id
    # Run the following command if the container runtime is Containerd: 
    crictl inspect $container_id|jq ".info.pid"
  5. Run the following command to go to the network namespace of the Productpage container and obtain the current configuration:

    nsenter -t $ -n iptables -L -t nat -n -v --line-numbers -x

    Show the expected output

    Chain PREROUTING (policy ACCEPT 0 packets, 0 bytes)
    num      pkts      bytes target     prot opt in     out     source               destination
    1       34938  2096280 ISTIO_INBOUND  tcp  --  *      *       0.0.0.0/0            0.0.0.0/0
    
    Chain INPUT (policy ACCEPT 0 packets, 0 bytes)
    num      pkts      bytes target     prot opt in     out     source               destination
    
    Chain POSTROUTING (policy ACCEPT 0 packets, 0 bytes)
    num      pkts      bytes target     prot opt in     out     source               destination
    
    Chain OUTPUT (policy ACCEPT 0 packets, 0 bytes)
    num      pkts      bytes target     prot opt in     out     source               destination
    1          17     1020 ISTIO_OUTPUT  tcp  --  *      *       0.0.0.0/0            0.0.0.0/0
    
    Chain ISTIO_INBOUND (1 references)
    num      pkts      bytes target     prot opt in     out     source               destination
    1           0        0 RETURN     tcp  --  *      *       0.0.0.0/0            0.0.0.0/0            tcp dpt:15008
    2           0        0 RETURN     tcp  --  *      *       0.0.0.0/0            0.0.0.0/0            tcp dpt:22
    3           0        0 RETURN     tcp  --  *      *       0.0.0.0/0            0.0.0.0/0            tcp dpt:15020
    4       34938  2096280 RETURN     tcp  --  *      *       0.0.0.0/0            0.0.0.0/0            tcp dpt:15021
    5           0        0 RETURN     tcp  --  *      *       0.0.0.0/0            0.0.0.0/0            tcp dpt:15090
    6           0        0 ISTIO_IN_REDIRECT  tcp  --  *      *       0.0.0.0/0            0.0.0.0/0
    
    Chain ISTIO_REDIRECT (1 references)
    num      pkts      bytes target     prot opt in     out     source               destination
    1           0        0 REDIRECT   tcp  --  *      *       0.0.0.0/0            0.0.0.0/0            redir ports 15001
    
    Chain ISTIO_IN_REDIRECT (3 references)
    num      pkts      bytes target     prot opt in     out     source               destination
    1           1       60 REDIRECT   tcp  --  *      *       0.0.0.0/0            0.0.0.0/0            redir ports 15006
    
    Chain ISTIO_OUTPUT (1 references)
    num      pkts      bytes target     prot opt in     out     source               destination
    1           2      120 RETURN     tcp  --  *      *       0.0.0.0/0            0.0.0.0/0            tcp dpt:15020
    2           0        0 RETURN     all  --  *      lo      127.0.X.X            0.0.0.0/0
    3           1       60 ISTIO_IN_REDIRECT  all  --  *      lo      0.0.0.0/0           !127.0.X.X            owner UID match 1337
    4           0        0 RETURN     all  --  *      lo      0.0.0.0/0            0.0.0.0/0            ! owner UID match 1337
    5          14      840 RETURN     all  --  *      *       0.0.0.0/0            0.0.0.0/0            owner UID match 1337
    6           0        0 ISTIO_IN_REDIRECT  all  --  *      lo      0.0.0.0/0           !127.0.X.X            owner GID match 1337
    7           0        0 RETURN     all  --  *      lo      0.0.0.0/0            0.0.0.0/0            ! owner GID match 1337
    8           0        0 RETURN     all  --  *      *       0.0.0.0/0            0.0.0.0/0            owner GID match 1337
    9           0        0 RETURN     all  --  *      *       0.0.0.0/0            127.0.X.X
    10          0        0 RETURN     all  --  *      *       0.0.0.0/0            192.168.0.1
    11          0        0 ISTIO_REDIRECT  all  --  *      *       0.0.0.0/0            0.0.0.0/0

    The preceding output shows that iptables rules exist, such as ISTIO_INBOUND, ISTIO_REDIRECT, ISTIO_IN_REDIRECT, and ISTIO_OUTPUT. This indicates that the iptables rules take effect.