Alibaba Cloud Service Mesh:How does the server application obtain the client source IP in a service mesh environment

Choose the right solution

The correct approach depends on your traffic direction and load balancer type:

If your cluster uses the Terway network plug-in, source IPs are preserved by default for north-south traffic. You can skip the externalTrafficPolicy configuration.

Deploy the sleep application

1. Create a sleep.yaml file with the following content:

      apiVersion: v1
      kind: ServiceAccount
      metadata:
        name: sleep
      ---
      apiVersion: v1
      kind: Service
      metadata:
        name: sleep
        labels:
          app: sleep
          service: sleep
      spec:
        ports:
        - port: 80
          name: http
        selector:
          app: sleep
      ---
      apiVersion: apps/v1
      kind: Deployment
      metadata:
        name: sleep
      spec:
        replicas: 1
        selector:
          matchLabels:
            app: sleep
        template:
          metadata:
            labels:
              app: sleep
          spec:
            terminationGracePeriodSeconds: 0
            serviceAccountName: sleep
            containers:
            - name: sleep
              image: curlimages/curl
              command: ["/bin/sleep", "3650d"]
              imagePullPolicy: IfNotPresent
              volumeMounts:
              - mountPath: /etc/sleep/tls
                name: secret-volume
            volumes:
            - name: secret-volume
              secret:
                secretName: sleep-secret
                optional: true

2. Apply the manifest:

      kubectl -n default apply -f sleep.yaml

Deploy the HTTPBin application

1. Create an httpbin.yaml file with the following content:

      apiVersion: v1
      kind: Service
      metadata:
        name: httpbin
        labels:
          app: httpbin
      spec:
        ports:
        - name: http
          port: 8000
        selector:
          app: httpbin
      ---
      apiVersion: apps/v1
      kind: Deployment
      metadata:
        name: httpbin
      spec:
        replicas: 1
        selector:
          matchLabels:
            app: httpbin
            version: v1
        template:
          metadata:
            labels:
              app: httpbin
              version: v1
          spec:
            containers:
            - image: docker.io/citizenstig/httpbin
              imagePullPolicy: IfNotPresent
              name: httpbin
              ports:
              - containerPort: 8000

2. Apply the manifest:

      kubectl -n default apply -f httpbin.yaml

Verify the default behavior

1. Check the pod IPs: Expected output: Note the IP address of the sleep pod (172.17.X.XXX).

      kubectl -n default get pods -o wide

      NAME                            READY   STATUS        RESTARTS   AGE     IP             NODE                     NOMINATED NODE   READINESS GATES
      httpbin-c85bdb469-4ll2m         2/2     Running       0          3m22s   172.17.X.XXX   cn-hongkong.10.0.0.XX    <none>           <none>
      sleep-8f764df66-q7dr2           2/2     Running       0          3m9s    172.17.X.XXX   cn-hongkong.10.0.0.XX    <none>           <none>

2. Send a request from the sleep pod to HTTPBin: Expected output: The reported origin is 127.0.0.6 (the Envoy loopback address), not the sleep pod IP.

      kubectl -n default exec -it deploy/sleep -c sleep -- curl http://httpbin:8000/ip

      {
        "origin": "127.0.0.6"
      }

3. Confirm at the socket level. First, log in to the HTTPBin container and install netstat: Exit the container and check port 80 connections: Expected output: The source address is 127.0.0.6.

      apt update & apt install net-tools

      kubectl -n default exec -it deploy/httpbin -c httpbin -- netstat -ntp | grep 80

      tcp        0      0 172.17.X.XXX:80         127.0.0.6:42691         TIME_WAIT   -

4. Examine the Envoy access logs for the HTTPBin pod. Key fields in the formatted log:

   Field	Value	Meaning
   downstream_remote_address	172.17.X.XXX:56160	Sleep pod IP (the actual sender)
   downstream_local_address	172.17.X.XXX:80	Address the sleep pod connected to
   upstream_local_address	127.0.0.6:42169	Address Envoy used to connect to HTTPBin -- this is why the application sees 127.0.0.6
   upstream_host	172.17.X.XXX:80	HTTPBin pod address

      {
        "downstream_remote_address": "172.17.X.XXX:56160",
        "downstream_local_address": "172.17.X.XXX:80",
        "upstream_local_address": "127.0.0.6:42169",
        "upstream_host": "172.17.X.XXX:80"
      }

Enable TPROXY mode

1. Patch the HTTPBin Deployment to use TPROXY interception mode:

      kubectl patch deployment -n default httpbin -p '{"spec":{"template":{"metadata":{"annotations":{"sidecar.istio.io/interceptionMode":"TPROXY"}}}}}'

2. After the pod restarts, send a request from the sleep pod: Expected output: HTTPBin now sees the real IP address of the sleep pod.

      kubectl -n default exec -it deploy/sleep -c sleep -- curl http://httpbin:8000/ip

      {
        "origin": "172.17.X.XXX"
      }

3. Confirm at the socket level: Expected output: The source address is now the sleep pod IP.

   After the pod restarts, reinstall netstat inside the HTTPBin container.

      kubectl -n default exec -it deploy/httpbin -c httpbin -- netstat -ntp | grep 80

      tcp        0      0 172.17.X.XXX:80         172.17.X.XXX:36728      ESTABLISHED -

4. Examine the Envoy access logs. Key fields:

   Field	Value	Meaning
   downstream_remote_address	172.17.X.XXX:39058	Sleep pod IP
   downstream_local_address	172.17.X.XXX:80	Address the sleep pod connected to
   upstream_local_address	172.17.X.XXX:46129	Now the sleep pod IP, confirming source IP preservation
   upstream_host	172.17.X.XXX:80	HTTPBin pod address

      {
        "downstream_remote_address": "172.17.X.XXX:39058",
        "downstream_local_address": "172.17.X.XXX:80",
        "upstream_local_address": "172.17.X.XXX:46129",
        "upstream_host": "172.17.X.XXX:80"
      }

HTTP requests

HTTPS requests

Source IP preservation for HTTPS traffic uses the same externalTrafficPolicy: Local setting. Complete the externalTrafficPolicy configuration above before proceeding.

1. Create an https-demo.yaml file that defines a Gateway and VirtualService for HTTPS access:

      apiVersion: networking.istio.io/v1alpha3
      kind: Gateway
      metadata:
        name: httpbin-gw-https
        namespace: default
      spec:
        selector:
          istio: ingressgateway
        servers:
          - hosts:
              - '*'
            port:
              name: https
              number: 443
              protocol: HTTPS
            tls:
              credentialName: myexample-credential
              mode: SIMPLE
      ---
      apiVersion: networking.istio.io/v1alpha3
      kind: VirtualService
      metadata:
        name: httpbin-https
        namespace: default
      spec:
        gateways:
          - httpbin-gw-https
        hosts:
          - '*'
        http:
          - route:
              - destination:
                  host: httpbin
                  port:
                    number: 8000

2. Apply the manifest:

      kubectl -n default apply -f https-demo.yaml

3. Send an HTTPS request through the ingress gateway: Expected output: The returned IP is the real client source IP address.

      export GATEWAY_URL=$(kubectl -n istio-system get service istio-ingressgateway -o jsonpath='{.status.loadBalancer.ingress[0].ip}')
      curl -k https://$GATEWAY_URL:443/ip

      {
        "origin": "120.244.XXX.XXX"
      }