Alibaba Cloud Service Mesh:Build a Wasm plug-in in Go for Envoy proxies

How Wasm plug-ins work in Envoy

Wasm plug-ins execute inside a sandboxed environment with a well-defined API for communicating with the host proxy. This architecture provides several benefits:

For more details, see WebAssembly in Envoy and the proxy-wasm-go-sdk overview.

End-to-end workflow

The plug-in you build in this tutorial checks whether incoming requests contain the allow: true header. Requests without this header receive a 403 Forbidden response. Requests with the header pass through to the backend service.

1. Write plug-in logic in Go using the proxy-wasm-go-sdk.

2. Compile the Go code into a Wasm binary with TinyGo.

3. Package the binary as an OCI image and push it to Container Registry Enterprise Edition.

4. Create a WasmPlugin resource in ASM to apply the plug-in to an Envoy proxy.

Prerequisites

Before you begin, make sure that you have:

• An ASM instance (v1.18 or later) with a cluster added. See Add a cluster to an ASM instance

• Automatic sidecar proxy injection enabled. See Configure sidecar proxy injection policies

• An ingress gateway deployed. See Create an ingress gateway

• The HTTPBin application deployed and accessible. See Deploy the HTTPBin application

• A Container Registry Enterprise Edition instance created (required for OCI image support). See Create a Container Registry Enterprise Edition instance

Step 1: Set up the development environment

Install the following tools on your local machine:

The plug-in depends on the proxy-wasm-go-sdk, which contains the full Go SDK API and additional examples.

Step 2: Write the plug-in code

1. Create a project directory and add a main.go file with the following content:

   Show the main.go file

   package main
   
   import (
   	"github.com/tetratelabs/proxy-wasm-go-sdk/proxywasm"
   	"github.com/tetratelabs/proxy-wasm-go-sdk/proxywasm/types"
   )
   
   func main() {
   	proxywasm.SetVMContext(&vmContext{})
   }
   
   // vmContext implements types.VMContext.
   // Embedding DefaultVMContext provides default implementations for all methods.
   type vmContext struct {
   	types.DefaultVMContext
   }
   
   // NewPluginContext creates a new plug-in context for each Wasm VM.
   func (*vmContext) NewPluginContext(contextID uint32) types.PluginContext {
   	return &pluginContext{}
   }
   
   // pluginContext implements types.PluginContext.
   type pluginContext struct {
   	types.DefaultPluginContext
   }
   
   func (ctx *pluginContext) OnPluginStart(pluginConfigurationSize int) types.OnPluginStartStatus {
   	return types.OnPluginStartStatusOK
   }
   
   // NewHttpContext creates an HTTP context for each request.
   func (ctx *pluginContext) NewHttpContext(contextID uint32) types.HttpContext {
   	return &HeaderAuthorizationHandler{}
   }
   
   // HeaderAuthorizationHandler checks the "allow" header on each request.
   type HeaderAuthorizationHandler struct {
   	types.DefaultHttpContext
   }
   
   // OnHttpRequestHeaders inspects request headers.
   // If the "allow" header is missing or not set to "true", the request is denied with 403.
   func (ctx *HeaderAuthorizationHandler) OnHttpRequestHeaders(numHeaders int, endOfStream bool) types.Action {
   	const AuthorizationKey = "allow"
   	value, err := proxywasm.GetHttpRequestHeader(AuthorizationKey)
   	if err != nil || value != "true" {
   		proxywasm.LogDebugf("request header: 'allow' is %v, only true can passthrough", value)
   		return ctx.DenyRequest()
   	}
   	return types.ActionContinue
   }
   
   // DenyRequest sends a 403 response and pauses further processing.
   func (ctx *HeaderAuthorizationHandler) DenyRequest() types.Action {
   	proxywasm.SendHttpResponse(403, [][2]string{{"Content-Type", "text/plain"}}, []byte("Forbidden by ASM Wasm Plugin"), -1)
   	return types.ActionPause
   }

2. Initialize the Go module and download dependencies:

   go mod init
   go mod tidy

3. Compile the code into a Wasm binary: This produces a plugin.wasm file in the current directory.

   tinygo build -o plugin.wasm -scheduler=none -target=wasi main.go

Step 3: Package and push the OCI image

1. In the same project directory, create a Dockerfile:

   FROM scratch
   ADD ./plugin.wasm ./plugin.wasm

2. Build the image:

   docker build -t header-authorization:v0.0.1 .

3. Create an image repository in Container Registry Enterprise Edition. For detailed steps, see substeps 2.a and 2.b of Step 1 in Use the Coraza Wasm plug-in to implement WAF capabilities on an ASM gateway. In this example, the namespace is test-oci and the repository name is header-authorization.

   

4. Tag and push the image to your Container Registry Enterprise Edition instance. Follow the Push image to the registry instructions shown in the repository details page.

Step 4: Apply the Wasm plug-in to the ingress gateway

1. Create a Secret for image pulling. For background, see Step 2: Configure permissions to pull images. Replace the following placeholders with your actual values:

   Placeholder	Description
   <your-registry-domain>	Domain name of the Container Registry Enterprise Edition instance
   <your-username>	Registry username
   <your-password>	Registry password

   kubectl create secret docker-registry -n istio-system wasm-secret \
     --docker-server=<your-registry-domain> \
     --docker-username=<your-username> \
     --docker-password=<your-password>

2. Create a file named asm-plugin.yaml with the following content: Replace <your-registry-domain> with the domain name of your Container Registry Enterprise Edition instance.

   apiVersion: extensions.istio.io/v1alpha1
   kind: WasmPlugin
   metadata:
     name: header-authorization
     namespace: istio-system
   spec:
     imagePullPolicy: IfNotPresent
     imagePullSecret: wasm-secret
     selector:
       matchLabels:
         istio: ingressgateway
     url: oci://<your-registry-domain>/test-oci/header-authorization:v0.0.1
     phase: AUTHN

3. Connect to the ASM instance using its kubeconfig file and apply the resource:

   kubectl apply -f asm-plugin.yaml

Step 5: Verify the plug-in

1. Enable debug logging for the Wasm plug-in on the ingress gateway. Use the kubeconfig file of the data-plane cluster where the ingress gateway runs: Replace <ingress-gateway-pod> with the name of the Pod running the ingress gateway.

   kubectl -n istio-system exec <ingress-gateway-pod> -c istio-proxy -- \
     curl -XPOST "localhost:15000/logging?wasm=debug"

2. Send a request without the allow header: Expected output:

   curl <ingress-gateway-ip>/status/418

   Forbidden by ASM Wasm Plugin

3. Check the ingress gateway Pod logs. The log entry confirms the plug-in blocked the request:

   2024-03-08T08:16:46.747394Z	debug	envoy wasm external/envoy/source/extensions/common/wasm/context.cc:1168	wasm log istio-system.header-authorization: request header: 'allow' is , only true can passthrough	thread=24
   {"bytes_received":"0","bytes_sent":"28","downstream_local_address":"xxxxxxx","downstream_remote_address":"xxxxxxxx","duration":"0","istio_policy_status":"-","method":"GET","path":"/status/418","protocol":"HTTP/1.1","request_id":"780c8493-13e4-4f97-9771-486efe30347c","requested_server_name":"-","response_code":"403","response_flags":"-","route_name":"httpbin","start_time":"2024-03-08T08:16:46.747Z","trace_id":"-","upstream_cluster":"outbound|8000||httpbin.default.svc.cluster.local","upstream_host":"-","upstream_local_address":"-","upstream_service_time":"-","upstream_response_time":"-","upstream_transport_failure_reason":"-","user_agent":"curl/8.4.0","x_forwarded_for":"xxxxxx","authority_for":"xxxxxx"}

4. Send a request with the allow: true header: Expected output: The teapot response confirms that the HTTPBin application is accessible and the plug-in is working correctly.

   curl <ingress-gateway-ip>/status/418 -H "allow: true"

      -=[ teapot ]=-
   
             _...._
           .'  _ _ `.
          | ."` ^ `". _,
          \_;`"---"`|//
            |       ;/
            \_     _/
              `"""`

Fix TinyGo memory leaks with nottinygc

Wasm plug-ins compiled with TinyGo may exhibit memory leaks. The proxy-wasm-go-sdk community recommends using nottinygc as a custom garbage collector to resolve this issue.

1. Add the following import to the top of main.go: If this dependency is not yet available, run go mod tidy to download it.

   import _ "github.com/wasilibs/nottinygc"

2. Rebuild the Wasm binary with the custom GC flags: The -gc=custom flag replaces TinyGo's default garbage collector with nottinygc, and -tags='custommalloc nottinygc_envoy' enables the Envoy-optimized allocator.

   tinygo build -o plugin.wasm -gc=custom -tags='custommalloc nottinygc_envoy' \
     -target=wasi -scheduler=none main.go

3. Rebuild the OCI image and push the updated version to Container Registry Enterprise Edition, then reapply the WasmPlugin resource to roll out the fix.