Alibaba Cloud Service Mesh:What can I do if the pods of a Kubernetes cluster on the data plane cannot access the IP address of the CLB instance that is configured in an ingress gateway?

Problem description

Causes

If the CLB instance whose externalTrafficPolicy parameter is set to Local is specified for the ingress gateway service of the Kubernetes cluster, only the backend pods where the service is deployed can access the IP address of the CLB instance. This is because the IP address of the CLB instance is regarded as an external IP address of the service and is used to access the ingress gateway from outside the Kubernetes cluster. If the nodes and pods in the Kubernetes cluster cannot directly access the IP address of the CLB instance, the system does not route requests to the CLB instance. Instead, the requests are forwarded by kube-proxy in iptables or IP Virtual Server (IPVS) mode.

If no backend pods of the service are deployed on the nodes of the Kubernetes cluster or the nodes where the pods that send requests reside, the IP address of the CLB instance cannot be accessed. If the backend pods of the service are deployed, the IP address of the CLB instance can be accessed. For more information, see Why kube-proxy add external-lb's address to node local iptables rule?.

Solutions

• You can use the IP address of the Kubernetes cluster or the name of the ingress gateway service to access the IP address of the CLB instance within the Kubernetes cluster. The name of the ingress gateway service is istio-ingressgateway.istio-system.

  Note

  We recommend that you use this solution.

• If you do not require source IP addresses, you can use the following solution:

  Change the value of the externalTrafficPolicy parameter of the ingress gateway to Cluster. In this case, you cannot obtain source IP addresses when you access the IP address of the CLB instance. For more information, see CRD fields for a gateway.

  apiVersion: istio.alibabacloud.com/v1beta1
  kind: IstioGateway
  metadata:
    name: ingressgateway
    namespace: istio-system
    ....
  spec:
    externalTrafficPolicy: Cluster
  ....

• If you use elastic network interfaces (ENIs) of Terway or your clusters are in inclusive ENI mode, you can use the following solution: This solution allows you to access the IP address of the CLB instance within the Kubernetes cluster without losing source IP addresses.

  Change the value of the externalTrafficPolicy parameter of the ingress gateway to Cluster and add an annotation, such as serviceAnnotations: service.beta.kubernetes.io/backend-type: "eni", to directly connect to ENIs. For more information, see CRD fields for a gateway.

  apiVersion: istio.alibabacloud.com/v1beta1
  kind: IstioGateway
  metadata:
    name: ingressgateway
    namespace: istio-system
    ....
  spec:
    externalTrafficPolicy: Cluster
    maxReplicas: 5
    minReplicas: 2
    ports:
      - name: status-port
        port: 15020
        targetPort: 15020
      - name: http2
        port: 80
        targetPort: 80
      - name: https
        port: 443
        targetPort: 443
      - name: tls
        port: 15443
        targetPort: 15443
    replicaCount: 2
    resources:
      limits:
        cpu: '2'
        memory: 2G
      requests:
        cpu: 200m
        memory: 256Mi
    runAsRoot: false
    serviceAnnotations:
      service.beta.kubernetes.io/backend-type: eni
    serviceType: LoadBalancer