Simplification and Extension of Alibaba Cloud Service Mesh Based on Wasm and ORAS

By Wang Xining – Alibaba Cloud Senior Technical Expert

This article describes how to use the ORAS client to push Wasm modules with allowed media types to the Alibaba Cloud Container Registry (ACR) (an OCI-compatible registry).

The Wasm Filter is deployed to the Pod corresponding to the specified workload through the ASM controller. The declarative mode is used in all Wasm Filter deployment procedures, which is a custom resource CRD that can describe the Wasm Filter deployment. Once the CRD is created, the ASM controller loads the Wasm module into the corresponding Envoy proxy in the data plane layer. At the same time, the corresponding Istio EnvoyFilter custom resources are also created in the control plane layer.

Envoy Filter Introduction

First, let's review the implementation mechanism of EnvoyProxy. The core of Envoy is an L3/L4 network proxy that also supports the L7 proxy. Envoy provides a pluggable filter chain mechanism for developers to write filters to execute different tasks. For example, the commonly used HTTP connection manager converts raw bytes into HTTP-level messages and events. It also processes all HTTP connections and requests with common functions, such as log access and tracing.

As shown in the figure above, Downstream is the part of the client that connects to Envoy, sends requests, and receives responses. The Listener component is bound to the IP address/port and receives connections from Downstream. Users can enable the proxy traffic management capability by configuring the Listener and use multiple filters to enhance the data stream. Multiple filters compose a Filter Chain. After being processed by these Filter Chains, requests are mapped to the corresponding Clusters. The Cluster is responsible for connecting to a set of upstream node services and forwarding these requests using the associated load balancing policy. Note: Here, the Cluster is a group of upstream hosts with the same logic to which the Envoy is connected. It is not related to the Kubernetes Cluster mentioned below.

Envoy Filters are divided into three types based on different processing tasks:

Listener Filter: Processes metadata in L4 connections
Network Filter: Processes raw data in L4 connections
HTTP Filter: Processes HTTP requests and responses in L7 connections

In addition to these built-in filters, it can develop custom filters. Filters can be built with the native C++ compilation or the Wasm technology.

Envoy provides a set of APIs, known as the xDS API, where the Envoy proxy can be dynamically configured in the control plane layer.

As shown in the preceding figure, for outbound traffic, similar to inbound traffic, a listener listens to the network traffic at the configured IP address or port. Each listener also defines a set of filters located in the data path and forms a set of Filter Chains. With this set of Filter Chains, users can configure Envoy to do specific tasks for outbound traffic, including data protocol processing, call statistics generation, and RBAC permission executions.

A real-world case is provided below to help understand these Envoy Filters and Filter Chains. The productpage service is the first service in the Istio official sample bookinfo. First, the Envoy Proxy in the productpage pod is configured with a listener that listens to Port 9080. All the traffic requests that enter port 9080 of the pod will be intercepted to this proxy. Then, the requests are processed by these Filter Chains. The details are listed below:

The first filter is evoy.filters.network.metadata_exchange. As the name implies, it exchanges metadata between filters.
The second filter is envoy.http_connection_manager. It usually has the following filters specific to http:
envoy.filters.http.wasm/envoy.wasm.metadata_exchange (for metadata interaction)
Istio_authn filter (for authorization and authentication)
envoy.filters.http.cors (filter dealing with cross-domain resource sharing)
envoy.filters.http.fault (This is a fault injection filter that can test fault tolerance in the microservice frameworks. The user can customize the error code to implement the delay injection or termination request, handling errors in different failure scenarios, such as service failure, service overload, and high latency. This is also a commonly used filter.)
envoy.filters.http.wasm/envoy.wasm.stats and envoy.filters.http.wasm/xxx-wasmfilter (filters implemented by customized Wasm)
envoy.filters.http.router (filter for HTTP forwarding that is used in almost all HTTP scenarios)

Note: Configuration information can be obtained by requesting this URL address: kubectl exec -it [productpage-xxx] -c istio-proxy curl localhost:15000/config_dump.

Add New Filters

Several built-in filters have been provided by the Envoy community. For more information, please see this link.

In service mesh, these built-in filter capabilities can be enabled through the API.

If these built-in filters cannot meet the requirements, users can also customize filters in the following two ways:

Static Precompilation

Integrate other filters into the source code of Envoy and compile a new version of Envoy.
The drawback of this method is that the Envoy version must be maintained and kept in sync with the official version.
Since Envoy is implemented by C++, the newly developed filters must be implemented by C++ as well.

Dynamic Runtime Loading

Load the new filter dynamically into the Envoy proxy at runtime
The WebAssembly technology is introduced to simplify the process of Envoy expansion. It is an effective portable binary instruction format that provides an embeddable and isolable execution environment.

Advantages and Disadvantages of Envoy Proxy Extension by Wasm

In practice, whether to extend the Envoy Filter by Wasm is decided according to the following advantages and disadvantages.

Pros

Agility: Filters can be dynamically loaded into the running Envoy proxies without ceasing or recompiling.
Maintainability: There is no need to change the base code libraries of Envoy proxies to extend the functionality.
Diversity: Popular programming languages, such as C, C++, and Rust, can be compiled into WASM. Therefore, developers can choose the programming languages to implement filters.
Reliability and Isolation: Filters are isolated from the Envoy process and deployed into a virtual machine sandbox. If a WASM Filter fails, it does not impact the Envoy process.
Security: Filters communicate with Envoy proxies through the predefined API. Therefore, the filters can only access and modify a limited number of connections or request properties.

Cons

The filter performance is only about 70% of the filter statically compiled by native C++.
More memories are consumed because one or more WASM virtual machines are enabled.
The WebAssembly ecosystem is still young.

Envoy-Wasm Operation Mechanism

As shown in the following figure, the Envoy-Wasm operation mechanism includes the following steps:

The Wasm binary code must be dynamically loaded, whether through a local file or xds remote access.
Consistency checks are needed to decide whether a Wasm Filter is allowed to be loaded: https://github.com/proxy-wasm/spec.
Once loaded, Wasm Filters become a part of the filter chain. When new requests come in, they still enter the native filter first and then go to the Proxy-Wasm extension controller.
The Proxy-Wasm extension controller calls and executes the registered and verified Wasm Filters according to the configuration information defined in the filter chain.
The built-in Wasm runtime supports LLVM-based WAVM - 20MB and V8 - 10MB.
Event-Driven Model
The native filter call method is compatible with a Wasm Filter.

The following is the configuration content of a Wasm Filter delivered to the Envoy Proxy:

The Envoy Filter and its extension through Wasm are explained above, which introduces the Wasm filter mechanism. This mechanism will be the mainstream method in the future.

In a service mesh system, cloud products need to learn how to manage the deployment and operation of Wasm Filters effectively and simply.

OPAS and Wasm Filter Registries

In the cloud-native ecosystem, when it comes to the management of an Artifact file, many people may think of the OCI specification and wonder whether these Wasm Filters can be managed in the same way as managing Docker images.

The OCI Registry As Storage (ORAS) project will solve this problem. ORAS is the reference implementation of the OCI Artifacts project, which can significantly simplify the storage of any content in the OCI registry.

The ORAS API and SDK Library can build custom tools to implement the following functions:

Push the WebAssembly module into the OCI registry
Pull the WebAssembly module from the OCI registry

The usage of oras cli is similar to that of docker cli, as shown below:

Let's take the Alibaba Cloud Container Registry Enterprise Edition (ACR EE) as an example. As an enterprise-level cloud-native application product management platform, ACR EE provides lifecycle management for container images, Helm charts, and products that conform to the OCI specification. After it is enabled, create an image repository, and an address will be allocated to it, either through a Virtual Private Cloud (VPC) or public network.

oras login --username=<login account> acree-1-registry.cn-hangzhou.cr.aliyuncs.com

Execute the following command using the oras push command:

oras push acree-1-registry.cn-hangzhou.cr.aliyuncs.com//asm-test:v0.1 --manifest-config runtime-config.json:application/vnd.module.wasm.config.v1+json example-filter.wasm:application/vnd.module.wasm.content.layer.v1+wasm

Note: the parameter – manifest-config can be configured based on the Wasm Artifact image specification.

After Wasm Filter is pushed to ACR EE registry, the related information can be checked:

Alibaba Cloud Service Mesh (ASM) Architecture

How is the Wasm technology used in Alibaba Cloud Service Mesh (ASM)? First, let's take a look at the technical architecture of ASM, as shown in the following figure. As the first fully managed Istio-compatible service mesh product in the industry, ASM focuses on building a fully managed, secure, stable, and easy-to-use service mesh that supports multiple cross-region clusters as well as unified governance for multi-cloud hybrid cloud services. The components of the control plane are hosted on the Alibaba Cloud side, which is decoupled from the user cluster on the data side. This reduces the use complexity. Users only need to focus on the development and deployment of business applications. In hosted mode, the service mesh is compatible with Istio. It supports the definition of flexible routing rules by declaration and manages traffic across multiple Kubernetes clusters uniformly.

ASM is an important link connecting the upper-layer applications to the underlying computing infrastructure. It can be perceived from three perspectives:

Downward integration with infrastructure
Capacity building of service mesh
Upward support for the application layer and capabilities to be integrated

As a managed service mesh product, from the perspective of capacity building, ASM provides a flexible architecture that supports customized Proxy of different versions for the Istio control plane and data plane.

On the hosting side, transform and host the core components of the control plane for the lifecycle management of the entire control plane and data plane components. In terms of product capabilities, ASM enhances the Mesh CA and the security audit to improve the security of Mesh instances. Diagnostic rules have been created based on common problems in customer scenarios, based on which users can run diagnostic analysis alone.
In addition to the construction of the core hosting side, ASM optimizes and integrates multiple Alibaba Cloud products and services. In terms of observability, it integrates xtrace, arms, and log services; in terms of cross-VPC network connection, it integrates cen and achieves the interconnections between multiple clusters; and in terms of throttling, it integrates the AHAS throttling service.
ASM also integrates and extends the open-source component capabilities of the community, including the support of the OPA security engine, the spiffe and spire, and the extension of the Envoy Filter. Therefore, a simple and effective method is needed to help users extend these capabilities with ease.

Use Wasm in ASM

With the optimization of the new architecture, the WebAssembly technology has been introduced into the service mesh to solve the problem of proxy extension. As such, the ASM architecture has become a combination of the hosted elastic control plane of high availability and the scalable plug-in data plane.

ASM provides support for the WebAssembly (WASM) technology. Service mesh users can deploy the extended WASM Filter to the corresponding Envoy Proxy in the data plane cluster through ASM. The easy-to-use ASMFilterDeployment Controller component supports dynamic plug-in loading and hot update.

Envoy can be extended easily through the filter extension mechanism, and its application in service mesh has reached a new high.

The following section describes how Wasm works in ASM instances.

After an ASM instance is deployed, the Wasm is not enabled by default and must be enabled by the user manually, for example, through the following aliyun cli:

aliyun servicemesh UpdateMeshFeature  --ServiceMeshId=xxxxxx --WebAssemblyFilterEnabled=true

After enabling Wasm, the ASM instance deploys the relevant components and performs the following tasks:

Deploy a DaemonSet(asmwasm-controller) to the Kubernetes cluster
The asmwasm-controller listens to a configmap where the address of the Wasm Filter to be pulled is stored. For example:acree-1-registry.cn-hangzhou.cr.aliyuncs.com/*/sample:v0.1.
If authorization and authentication are required, the asmwasm-controller will obtain the corresponding secret value according to the defined pullSecret value.
Then, call the oras API to dynamically pull the Wasm Filter from the registry.
This asmwasm-controller mounts volume by means of the HostPath. Therefore, the pulled Wasm Filter is placed to the corresponding node.

After Wasm Filter is enabled, how can you deploy a Wasm Filter and mount it to the Envoy Proxy of the corresponding workload?

ASM provides a new CRD-based ASMFilterDeployment and related controller component. This controller component listens to the ASMFilterDeployment resource object and does two things:

Create an Istio EnvoyFilter Custom Resource for the control plane and push it to the corresponding asm control plane Istio
Pull the corresponding Wasm Filter image from the OCI registry and mount it to the corresponding workload pod

The following is an ASMFilterDeployment CR example:

apiVersion: istio.alibabacloud.com/v1beta1
kind: ASMFilterDeployment
metadata:
  name: details-v1-wasmfiltersample
spec:
  workload:
    kind: Deployment
    labels:
      app: details
      version: v1
  filter:
    parameters: '{"name":"hello","value":"hello details"}'
    image: 'acree-1-registry.cn-hangzhou.cr.aliyuncs.com/asm/asm-test:v0.1'
    imagePullOptions: 
      pullSecret: 'asmwasm-cache'
    rootID: 'my_root_id'
    id: 'details-v1-wasmfiltersample.default'

The generated Istio Envoy Filter resources are listed below:

In the figure, the envoy.router filter is defined in the match segment, and the INSERT_BEFORE operation is defined in the patch segment. Insert a Wasm Filter like this:

The workload mounted with Wasm Filter is defined and updated below. Therein, the Wasm Filter file is mounted to the Proxy container through hostpath:

apiVersion: extensions/v1beta1
kind: Deployment
metadata:
.…
spec:
   ….
   template:
      metadata:
          annotations:
              sidecar.istio.io/userVolume: '[{"name":"wasmfilters-dir","hostPath":{"path":"/var/local/lib/wasm-filters"}}]'
              sidecar.istio.io/userVolumeMount: '[{"mountPath":"/var/local/lib/wasm-filters","name":"wasmfilters-dir"}]'

Check whether the Wasm Filter takes effect. Log in to the istio-proxy container of the productpage Pod and run the following command to send some traffic to the details service. The response indicates that the header of the Wasm Filter is added to the response header.

kubectl exec -ti  deploy/productpage-v1 -c istio-proxy -- curl -v http://details:9080/details/123

*   Trying 172.21.9.191...
* TCP_NODELAY set
* Connected to details (172.21.9.191) port 9080 (#0)
> GET /details/123 HTTP/1.1
> Host: details:9080
> User-Agent: curl/7.58.0
> Accept: */*
>
< HTTP/1.1 200 OK
xxxxxxx
< resp-header-demo: added by our filter
xxxxx
* Connection #0 to host details left intact
xxxxx

Summary

In the Development Stage:

Use the appropriate wasm sdk or programming languages to create a Wasm binary file and upload the file to the oci image repository using oras cli:

In the Deployment and Running Stage:

First, confirm that Wasm is enabled in ASM and then create a custom ASMFilterDeployment resource. Note: This CR is created in the apiserver corresponding to the ASM instance. Once it is created, the corresponding crd controller listens to and synchronizes the corresponding resources. On the one hand, an Istio EnvoyFilter CR is generated and sent to the apiserver on the control plane of the ASM instance. Users can check whether the generated Istio EnvoyFilter CR reaches their expectations.

On the other hand, confirm that the Workload deployment changes have taken effect, which includes:

The ability to log in to the proxy container to check whether Wasm Filter is mounted.
The ability to print relevant information by adjusting the wasm log level.

As the first fully managed Istio-compatible service mesh in the industry, Alibaba Cloud Service Mesh (ASM) is an Istio-compatible hosted platform that uniformly manages microservice application traffic. It aspires to be a fully managed, secure, stable, and easy-to-use service mesh that supports the unified governance for multiple cross-region clusters and multi-cloud and hybrid cloud services. Through traffic control, mesh observation, inter-service communication security, and other features, ASM simplifies service governance in all aspects for users. It also provides unified management for services running on a heterogeneous computing infrastructure. Being widely applicable, it fits Kubernetes clusters, Serverless Kubernetes clusters, ECS virtual machines, and user-created clusters.

You are welcome to visit the Alibaba Cloud Service Mesh (ASM) product website for a trial today!

About the Author

Wang Xining is a Senior Technical Expert of Alibaba Cloud and the Technical Director of Alibaba Cloud Service Mesh (ASM). He specializes in Kubernetes, service mesh, and other cloud-native fields. Previously, he worked in the IBM China Development Center, serving as the Chairman of the Patent Technology Review Committee. He was responsible for and participated in a series of works related to SOA middleware, cloud computing, and other fields as an architect and main developer, with more than 50 international technical patents in related fields. He has shared his insights on technologies at many technology conferences, such as Kubecon, ArchSummit, and the Computing Conference. He is also the author of Analysis and Practice on Service Mesh Technology and has published a few articles in multiple technical communities.

Community

Simplification and Extension of Alibaba Cloud Service Mesh Based on Wasm and ORAS

Envoy Filter Introduction

Add New Filters

Advantages and Disadvantages of Envoy Proxy Extension by Wasm

Pros

Cons

Envoy-Wasm Operation Mechanism

OPAS and Wasm Filter Registries

Alibaba Cloud Service Mesh (ASM) Architecture

Use Wasm in ASM

Summary

About the Author

Read previous post:

Read next post:

Xi Ning Wang(王夕宁)

You may also like

Comments

Xi Ning Wang(王夕宁)

Related Products

Container Service for Kubernetes

ACK One

Container Registry

Cloud-Native Applications Management Solution