Write WASM filters for Envoy and deploy them in ASM - Alibaba Cloud Service Mesh

Service Mesh (ASM) supports the programming language WebAssembly (WASM). You can deploy WASM filters in Envoy proxies that are used to manage clusters on the data plane. Filters help you extend Envoy proxies with new features so that you can use Envoy proxies to implement more features in ASM. This topic introduces WASM filters and describes how to write WASM filters for Envoy proxies and deploy them in ASM.

Background information

An Envoy proxy is a high-performance programmable Layer 3, Layer 4, and Layer 7 proxy. ASM places Envoy proxies on the data plane. Envoy proxies use network filters to manage collections and handle traffic. Network filters can be mixed into filter chains to implement access control, data and protocol conversion, data enhancement, and auditing. You can add filters to Envoy proxies to expand the feature set of Envoy. Use one of the following methods to add filters:

Static precompilation: Integrate additional filters into the source code of Envoy proxies and compile a new version for Envoy proxies. The drawback of this method is that you must manually maintain the version of Envoy proxies to keep them in sync with the official version. In addition, Envoy proxies are implemented in C++. Therefore, the filters must be implemented by C++, too.
Dynamic loading at runtime: Dynamically load new filters into Envoy proxies at runtime.

The second method greatly simplifies the process of extending Envoy proxies with new features. This method relies on WASM, a portable and efficient binary instruction format that provides an embeddable and isolated execution environment.

The following figure shows how ASM works with WASM filters.

Advantages of WASM filters

WASM filters provide the following advantages:

Agility: Filters can be dynamically loaded into the running Envoy proxies. You do not need to stop or recompile the Envoy proxies.
Maintainability: You do not need to change the code libraries of Envoy proxies to extend the functionality.
Diversity: You can use a popular programming language, such as C, C++, or Rust, to compile WASM filters based on your requirements.
Reliability and isolation: Filters are deployed into a sandbox virtual machine and are isolated from the Envoy process. If a WASM filter fails, it does not impact the Envoy process.
Security: Filters communicate with Envoy proxies by using the predefined API. Therefore, filters can access and modify only a limited number of connection or request properties.

Before you use WASM filters, take the following drawbacks into consideration:

Filters complied by WASM are about only 70% as fast as filters compiled by C++ in a static manner.
More memories are consumed because WASM filters rely on one or more WASM virtual machines.

Use the Envoy Proxy WASM SDK to build a filter

Envoy proxies run WASM filters in stack-based virtual machines. Memories of filters are isolated from the environment of Envoy proxies. All interactions between Envoy proxies and WASM filters are implemented by the Envoy Proxy WASM SDK. The Envoy Proxy WASM SDK supports many programming languages, including C++, Rust, AssemblyScript, and Go. Note that Go-based implementation is still in experiment. GitHub community is promoting the Application Binary Interface (ABI) specification and conventions to use between proxies and WebAssembly filters. For more information, see WebAssembly for Proxies (ABI specification).

The easiest way to build a WASM filter is using Docker. You can use the Envoy Proxy WASM SDK for C++ to create a docker image. For more information, see Docker.
Create a project. For more information, see Creating a project for use with the Docker build image.
Compile the project in the docker image. For more information, see Compiling with the Docker build image.

Go to the root directory of the project and run the following command to build a WASM filter:

docker run -v $PWD:/work -w /work  registry.cn-hangzhou.aliyuncs.com/acs/wasmsdk:v0.1 /build_wasm.sh

Deploy the WASM filter in ASM

Create a config map to hold the binary file of the WASM filter. For example, create a config map named wasm-example-filter in the default namespace and store the binary file example-filter.wasm of the WASM filter to the config map.
```
kubectl create configmap -n default wasm-example-filter --from-file=example-filter.wasm
```

Use the following two annotations to inject the binary file of the WASM filter to the Kubernetes containers of the target application:

sidecar.istio.io/userVolume: '[{"name":"wasmfilters-dir","configMap": {"name": "wasm-example-filter"}}]'
sidecar.istio.io/userVolumeMount: '[{"mountPath":"/var/local/lib/wasm-filters","name":"wasmfilters-dir"}]'

Update version 1 of the productpage service.

kubectl patch deployment productpage-v1 -p '{"spec":{"template":{"metadata":{"annotations":{"sidecar.istio.io/userVolume":"[{\"name\":\"wasmfilters-dir\",\"configMap\": {\"name\": \"wasm-example-filter\"}}]","sidecar.istio.io/userVolumeMount":"[{\"mountPath\":\"/var/local/lib/wasm-filters\",\"name\":\"wasmfilters-dir\"}]"}}}}}'

Update version 1 of the details service.

kubectl patch deployment details-v1 -p '{"spec":{"template":{"metadata":{"annotations":{"sidecar.istio.io/userVolume":"[{\"name\":\"wasmfilters-dir\",\"configMap\": {\"name\": \"wasm-example-filter\"}}]","sidecar.istio.io/userVolumeMount":"[{\"mountPath\":\"/var/local/lib/wasm-filters\",\"name\":\"wasmfilters-dir\"}]"}}}}}'

Check whether the binary file of the WASM filter is available under the /var/local/lib/wasm-filters path in the istio-proxy containers of the application.

kubectl exec -it deployment/productpage-v1 -c istio-proxy -- ls /var/local/lib/wasm-filters/
kubectl exec -it deployment/details-v1 -c istio-proxy -- ls /var/local/lib/wasm-filters/

Enable the WASM filter to keep logs at the DEBUG level when it processes traffic that targets the productpage service.
```
kubectl port-forward deployment/productpage-v1 15000
curl -XPOST "localhost:15000/logging? wasm=debug"
```
Enable the WASM filter to keep logs at the DEBUG level when it processes traffic that targets the details service.
```
kubectl port-forward deployment/details-v1 15000
curl -XPOST "localhost:15000/logging? wasm=debug"
```

Insert the WASM filter into the HTTP-level filter chain of the productpage service.

apiVersion: networking.istio.io/v1alpha3
kind: EnvoyFilter
metadata:
  name: productpage-v1-examplefilter
  labels:
    asm-system: 'true'
    provider: asm
spec:
  configPatches:
  - applyTo: HTTP_FILTER
    match:
      context: SIDECAR_INBOUND
      proxy:
        proxyVersion: '^1\.*.*'
      listener:
        filterChain:
          filter:
            name: envoy.filters.network.http_connection_manager
            subFilter:
              name: envoy.filters.http.router
    patch:
      operation: INSERT_BEFORE
      value:
        typed_config:
          "@type": type.googleapis.com/envoy.extensions.filters.http.wasm.v3.Wasm
          config:
            name: example-filter
            rootId: my_root_id
            vmConfig:
              code:
                local:
                  filename: /var/local/lib/wasm-filters/example-filter.wasm
              runtime: envoy.wasm.runtime.v8
              vmId: example-filter
              allow_precompiled: true
        name: envoy.filters.http.wasm
  workloadSelector:
    labels:
      app: productpage
      version: v1

Insert the WASM filter into the HTTP-level filter chain of the details service.

apiVersion: networking.istio.io/v1alpha3
kind: EnvoyFilter
metadata:
  name: details-v1-examplefilter
  labels:
    asm-system: 'true'
    provider: asm
spec:
  configPatches:
  - applyTo: HTTP_FILTER
    match:
      context: SIDECAR_INBOUND
      proxy:
        proxyVersion: '^1\.*.*'
      listener:
        filterChain:
          filter:
            name: envoy.filters.network.http_connection_manager
            subFilter:
              name: envoy.filters.http.router
    patch:
      operation: INSERT_BEFORE
      value:
        typed_config:
          "@type": type.googleapis.com/envoy.extensions.filters.http.wasm.v3.Wasm
          config:
            name: example-filter
            rootId: my_root_id
            vmConfig:
              code:
                local:
                  filename: /var/local/lib/wasm-filters/example-filter.wasm
              runtime: envoy.wasm.runtime.v8
              vmId: example-filter
              allow_precompiled: true
        name: envoy.filters.http.wasm
  workloadSelector:
    labels:
      app: details
      version: v1

Verify the WASM filter

Enter the ingress gateway address of the application in the address bar of your browser and send traffic to the productpage service. The response indicates that the header of the WASM filter is added to the response header, as shown in the following figure.

Run the following command to send traffic to the details service. The response indicates that the header of the WASM filter is added to the response header.

kubectl exec -ti  deploy/productpage-v1 -c istio-proxy -- curl -v http://details:9080/details/123

*   Trying 172.31.13.58...
* TCP_NODELAY set
* Connected to details (172.31.13.58) port 9080 (#0)
> GET /details/123 HTTP/1.1
> Host: details:9080
> User-Agent: curl/7.58.0
> Accept: */*
>
< HTTP/1.1 200 OK
xxxxxxx
< resp-header-demo: added by our filter
xxxxx
* Connection #0 to host details left intact
xxxxx