Client IP-based routing on ASM ingress gateways - Alibaba Cloud Service Mesh

Client IP-based routing is commonly used. For example, if users on the internal network and the Internet need to access different versions of applications, and users in different regions need to access different types of content, client IP-based routing can be used to achieve these goals. Therefore, Service Mesh (ASM) provides custom plug-ins to convert the IP address of a client to a request header for routing. This topic describes how to use custom plug-ins to implement client IP-based routing in ASM.

Background information

ASM uses the Envoy proxy to forward traffic and uses the rich plug-ins of Envoy to expand mesh capabilities. For example, Envoy provides highly flexible extension interfaces for HTTP-based services. You can use these interfaces to read detailed request metadata and even modify request properties such as the request header and body. Envoy provides the following three types of plug-ins:

Native Envoy filters: They are developed in C++. It is difficult to develop such plug-ins but they can provide better performance.
WebAssembly (Wasm) plug-ins: They can be developed in multiple programming languages. It is easier to develop such plug-ins, and they are more secure and more flexible.
Lua plug-ins: They are developed in Lua, limiting flexibility but allowing the easiest development.

In this example, a simple Lua plug-in is used to convert the client IP address of a request to a request header. A virtual service is used to match the header. This way, different responses are returned for requests from different sources.

Prerequisites

An ASM instance is created.
An ACK managed cluster is created.
The cluster is added to the ASM instance.
A kubectl client is connected to the cluster.
The HTTPBin application is deployed in the cluster and can be accessed through an ASM ingress gateway. For more information, see Deploy the HTTPBin application.

Step 1: Confirm the source IP address of the client

First, deploy the sleep application in the cluster as a client (hereinafter referred to as the sleep client). Then, access the HTTPBin application from the sleep client and a local client to test requests from different source IP addresses.

Note

Local clients refer to user terminals on which you perform operations in a specific scenario, including personal computers (PCs) and Elastic Compute Service (ECS) instances. For example, when you log on to an ECS instance from a PC and then use kubectl to access the Container Service for Kubernetes (ACK) managed cluster, the ECS instance is the local client.

Use kubectl to connect to the ACK cluster based on the information in the kubeconfig file and create a sleep.yaml file.

Show the sleep.yaml file

apiVersion: v1
kind: ServiceAccount
metadata:
  name: sleep
---
apiVersion: v1
kind: Service
metadata:
  name: sleep
  labels:
    app: sleep
    service: sleep
spec:
  ports:
  - port: 80
    name: http
  selector:
    app: sleep
---
apiVersion: apps/v1
kind: Deployment
metadata:
  name: sleep
spec:
  replicas: 1
  selector:
    matchLabels:
      app: sleep
  template:
    metadata:
      labels:
        app: sleep
    spec:
      terminationGracePeriodSeconds: 0
      serviceAccountName: sleep
      containers:
      - name: sleep
        image: registry.cn-hangzhou.aliyuncs.com/acs/curl:8.1.2
        command: ["/bin/sleep", "infinity"]
        imagePullPolicy: IfNotPresent
        volumeMounts:
        - mountPath: /etc/sleep/tls
          name: secret-volume
      volumes:
      - name: secret-volume
        secret:
          secretName: sleep-secret
          optional: true
---

Run the following command to deploy the sleep application:
```
kubectl apply -f sleep.yaml
```
Run the following command to configure environment variables for the IP address of the ASM ingress gateway. For more information about how to query the IP address of an ingress gateway, see substep 1 of Step 3 in the Use Istio resources to route traffic to different versions of a service topic.
```
export ASM_GATEWAY_IP=112.35.xx.xx
```
Run the following command to query the IP address of the local client:
```
curl ${ASM_GATEWAY_IP}/ip
```
Run the following command to query the IP address of the sleep pod:
```
kubectl exec deploy/sleep -it -- curl ${ASM_GATEWAY_IP}/ip
```
Note
The IP address of the client is returned for requests to access the /ip path of the HTTPBin application. To use this method to obtain the IP address of the client, you must set ExternalTrafficPolicy to local in the Service configurations of the ASM ingress gateway.

Step 2: Create a Lua plug-in and use it on the ASM ingress gateway

Lua plug-ins are implemented by using Lua filters of Envoy. You can write Lua code and send the Lua code as part of the Lua filter configuration to Envoy. During request processing, Envoy calls the configured Lua code to perform the corresponding operation.

The following code block shows the Lua code required in the example in this topic:

function envoy_on_request(request_handle)
    -- Define an envoy_on_request function to process the received requests.
    -- The request_handle parameter is used to obtain and modify the request information.
  local client_address = request_handle:streamInfo():downstreamRemoteAddress()
    -- Obtain the source IP addresses of requests.
  request_handle:headers():add("x-client-address", client_address)
    -- Set the request headers. The key of the headers is x-client-address.
end

Create an Envoy filter. For more information, see Create an Envoy filter by using an Envoy filter template.

When you create an Envoy filter template, paste the following YAML code into the code editor in the Multi-version adapted EnvoyFilter templates section.

apiVersion: networking.istio.io/v1alpha3
kind: EnvoyFilter
metadata:
  name: add-header-by-ip
  namespace: istio-system
spec:
  workloadSelector:
    labels:
      istio: ingressgateway
  configPatches:
    - applyTo: HTTP_FILTER
      match:
        context: GATEWAY
        listener:
          filterChain:
            filter:
              name: "envoy.filters.network.http_connection_manager"
      patch:
        operation: INSERT_BEFORE
        value:
          name: envoy.lua
          typed_config:
            "@type": "type.googleapis.com/envoy.extensions.filters.http.lua.v3.Lua"
            inlineCode: |
              function envoy_on_request(request_handle)
                local client_address = request_handle:streamInfo():downstreamRemoteAddress()
                request_handle:headers():add("x-client-address", client_address)
              end

To bind the Envoy filter template to a workload, find the desired Envoy filter template and click Bind template to workloads in the Actions column. On the page that appears, click Bind EnvoyFilter to Workloads. On the dialog box that appears, select istio-system from the Namespace drop-down list, select Service from the Workload Type drop-down list, find the corresponding Service of the ASM ingress gateway, and then click Bind next to the desired Service. The name of the Service is in the format of istio-${Name of the ASM ingress gateway}.
After the Envoy filter template is bound to the workload, click EnvoyFilter in the left-side navigation pane. On the EnvoyFilter page, you can find that the corresponding Envoy filter is created.

Step 3: Create a client IP-based routing rule

Use kubectl to connect to the ASM instance based on the information in the kubeconfig file and modify the virtual service created for the HTTPBin application. The following code block shows the final YAML code:

apiVersion: networking.istio.io/v1beta1
kind: VirtualService
metadata:
  name: httpbin-vs
  namespace: default
spec:
  gateways:
    - httpbin
  hosts:
    - '*'
  http:
    - match:
      - headers:
          x-client-address:
            prefix: ${The obtained IP address of the sleep client}
      directResponse:
        status: 200
        body:
          string: "from sleep"
    - match:
      - headers:
          x-client-address:
            prefix: ${The obtained IP address of the local client}
      directResponse:
        status: 200
        body:
          string: "from my host"    
    - name: test
      route:
        - destination:
            host: httpbin.default.svc.cluster.local
            port:
              number: 8000

The preceding configuration shows the following information:

The from sleep string is returned for all requests sent from the sleep client.
The from my host string is returned for all requests sent from the local client.

Step 4: Verify the configuration

Run the following command on the local client:
```
curl ${ASM_GATEWAY_IP}/ip
```
Expected output:
```
from my host
```
Use kubectl to connect to the ACK cluster based on the information in the kubeconfig file and run the following command on the sleep client:
```
kubectl exec deploy/sleep -it -- curl ${ASM_GATEWAY_IP}/ip
```
Expected output:
```
from sleep
```

References

If you are familiar with the Envoy architecture, it is easier for you to develop Lua plug-ins than other Envoy plug-ins. For more information about how to develop a Lua plug-in of Envoy, see Lua script for HTTP filters.
A Wasm plug-in is an Envoy plug-in written based on the WebAssembly for Proxies specification and can be developed in multiple programming languages. You can try to use the simple Golang to develop Wasm plug-ins. For more information, see Write a Wasm plug-in in Go for an Envoy proxy.