All Products
Search
Document Center

Alibaba Cloud Service Mesh:What can I do if the pods of a Kubernetes cluster on the data plane cannot access the IP address of the CLB instance that is configured in an ingress gateway?

Last Updated:Oct 20, 2023

This topic shows you how to resolve the issue where the pods of a Kubernetes cluster on the data plane cannot access the IP address of the Classic Load Balancer (CLB) instance that is configured in an ingress gateway.

Problem description

A Kubernetes cluster is added to your Service Mesh (ASM) instance. A CLB instance whose externalTrafficPolicy parameter is set to Local is configured in an ingress gateway for the ASM instance. When the pods of the Kubernetes cluster access the IP address of the CLB instance that is configured in the ingress gateway, the following issue occurs:

  • The Pod on some specific nodes can access the CLB address exposed by the entry gateway.

  • The Pod on some specific nodes can not access the CLB address exposed by the entry gateway.

Causes

If the CLB instance whose externalTrafficPolicy parameter is set to Local is specified for the ingress gateway service of the Kubernetes cluster, only the backend pods where the service is deployed can access the IP address of the CLB instance. This is because the IP address of the CLB instance is regarded as an external IP address of the service and is used to access the ingress gateway from outside the Kubernetes cluster. If the nodes and pods in the Kubernetes cluster cannot directly access the IP address of the CLB instance, the system does not route requests to the CLB instance. Instead, the requests are forwarded by kube-proxy in iptables or IP Virtual Server (IPVS) mode.

If no backend pods of the service are deployed on the nodes of the Kubernetes cluster or the nodes where the pods that send requests reside, the IP address of the CLB instance cannot be accessed. If the backend pods of the service are deployed, the IP address of the CLB instance can be accessed. For more information, see Why kube-proxy add external-lb's address to node local iptables rule?.

Solutions

  • You can use the IP address of the Kubernetes cluster or the name of the ingress gateway service to access the IP address of the CLB instance within the Kubernetes cluster. The name of the ingress gateway service is istio-ingressgateway.istio-system.

    Note

    We recommend that you use this solution.

  • If you do not require source IP addresses, you can use the following solution:

    Change the value of the externalTrafficPolicy parameter of the ingress gateway to Cluster. In this case, you cannot obtain source IP addresses when you access the IP address of the CLB instance. For more information, see CRD fields for a gateway.

    apiVersion: istio.alibabacloud.com/v1beta1
    kind: IstioGateway
    metadata:
      name: ingressgateway
      namespace: istio-system
      ....
    spec:
      externalTrafficPolicy: Cluster
    ....
  • If you use elastic network interfaces (ENIs) of Terway or your clusters are in inclusive ENI mode, you can use the following solution: This solution allows you to access the IP address of the CLB instance within the Kubernetes cluster without losing source IP addresses.

    Change the value of the externalTrafficPolicy parameter of the ingress gateway to Cluster and add an annotation, such as serviceAnnotations: service.beta.kubernetes.io/backend-type: eni, to directly connect to ENIs. For more information, see CRD fields for a gateway.

    apiVersion: istio.alibabacloud.com/v1beta1
    kind: IstioGateway
    metadata:
      name: ingressgateway
      namespace: istio-system
      ....
    spec:
      externalTrafficPolicy: Cluster
      maxReplicas: 5
      minReplicas: 2
      ports:
        - name: status-port
          port: 15020
          targetPort: 15020
        - name: http2
          port: 80
          targetPort: 80
        - name: https
          port: 443
          targetPort: 443
        - name: tls
          port: 15443
          targetPort: 15443
      replicaCount: 2
      resources:
        limits:
          cpu: '2'
          memory: 2G
        requests:
          cpu: 200m
          memory: 256Mi
      runAsRoot: false
      serviceAnnotations:
        service.beta.kubernetes.io/backend-type: eni
      serviceType: LoadBalancer