This article introduces the latest progress of Higress in switching the runtime of Wasm plug-in from V8 to WebAssembly Micro Runtime (WAMR). By switching to WAMR and turning on the AOT mode, the Wasm plug-in performance has been greatly improved by 50% on average, and the performance of some plug-ins with complex logic has been doubled.
Higress is a cloud-native api gateway open-source project, powered by Istio and Envoy, and largely adopted by Alibaba Cloud.
Alibaba Cloud has offered online service of Wasm plug-in development and deployment for their Higress product since 2022, which has grown to the main gateway expansion method because of the unique value to users:
Based on the Istio/Envoy, Higress adds three core capabilities to the Wasm plug-in mechanism:
From the perspective of Higress's enterprise users, the adoption life cycle of Wasm plug-in technology has crossed the chasm and entered the early majority stage. The core driving force is the cost reduction brought by performance dividends. For authentication, encryption, decryption, session management, and other logic, computing resources are offloaded at the gateway without the need for back-end service processing, thereby reducing computing costs globally.
Wasm technology was born in the browser scene and V8 is the JS/WebAssembly engine of Chromium. The V8 engine with JIT mode has good performance in executing the Wasm modules. But there are also the following problems with the practices:
1. Complexity: The V8 project is very complex, and Wasm-related implementations are highly coupled with JS processing logic. For example, a bug in the early Envoy Wasm plug-in was caused by V8 introducing pointer compression to optimize JS execution memory.
2. Lack of collaboration between the V8 community and the Envoy community. Envoy’s current version dependence on V8 is still stuck in 2022, and it cannot support new features such as Wasm GC. Because the project is complex, the risk of upgrading V8 dependencies is also quite high.
3. Client-side orientation: Most V8 users and developers come from the client-side. Considering device compatibility, they prefer JIT mode. AOT mode does not have significant performance improvements and cannot fully utilize the performance advantages of Wasm.
WAMR is a popular WebAssembly runtime open-source project first developed by the Intel team and under the Bytecode Alliance (a non-profit organization for the Wasm software ecosystem). Currently, active contributors in the community include engineers from Intel, Xiaomi, Amazon, Sony, Midokura, Siemens, Ant Group, and other companies. WAMR is developed using C language and has excellent platform adaptability. It supports interpretation mode, just-in-time compilation, and ahead-of-time compilation modes to run Wasm modules. It has excellent performance and has performed well in multiple public performance evaluation reports. It also has extremely low resource overhead and can run a single Wasm instance in 100KB memory.
• Pressure measurement tool: k6
• Server CPU model: Intel(R) Xeon(R) Platinum 8369B CPU @ 2.90GHz
• Stress test method: Higress starts 2 worker threads, fixes the pressure of k6 during the stress test, and runs both threads
Some Higress plug-ins were selected for performance testing. The situation is as follows:
Note: The data in the table is the average additional delay of a single request.
Overall, the more complex the Wasm command of the plug-in, the more obvious the WAMR improvement. All the above plug-ins, except jwt-logout, which is an enterprise version plug-in and is not open-source, the other plug-ins can be viewed in the Higress open-source warehouse directory for corresponding source code implementation: https://github.com/alibaba/higress/tree/main/plugins/wasm-cpp/extensions
To compile and generate AOT files, you can use wamrc, the official compilation tool provided by WAMR: wamrc --invoke-c-api-import -o plugin.aot plugin.wasm
In order for the generated wasm file to be compatible with JIT mode, use the script under the WAMR warehouse to generate the merged file: python3 wasm-micro-runtime/test-tools/append-aot-to-wasm/append_aot_to_wasm.py --aot plugin.aot - -wasm plugin.wasm -o plugin.aot.wasm
Taking the oauth plug-in with the greatest improvement as an example, you can use the following configuration to reproduce:
k6 run --vus 300 ./script.js --duration 60s
import http from 'k6/http';
import { check } from 'k6';
export default function () {
const res = http.get('http://11.164.3.16:10000/',{headers: {'Authorization':'Bearer eyJhbGciOiJIUzI1NiIsInR5cCI6ImFwcGxpY2F0aW9uL2F0K2p3dCJ9.eyJhdWQiOiJ0ZXN0MiIsImNsaWVudF9pZCI6Ijk1MTViNTY0LTBiMWQtMTFlZS05YzRjLTAwMTYzZTEyNTBiNSIsImV4cCI6MTY2NTY3MzgyOSwiaWF0IjoxNjY1NjczODE5LCJpc3MiOiJIaWdyZXNzLUdhdGV3YXkiLCJqdGkiOiIxMDk1OWQxYi04ZDYxLTRkZWMtYmVhNy05NDgxMDM3NWI2M2MiLCJzY29wZSI6InRlc3QiLCJzdWIiOiJjb25zdW1lcjEifQ.LsZ6mlRxlaqWa0IAZgmGVuDgypRbctkTcOyoCxqLrHY'}});
check(res, { 'status was 200': (r) => r.status == 200 });
}
- name: envoy.filters.http.wasm
typed_config:
"@type": type.googleapis.com/envoy.extensions.filters.http.wasm.v3.Wasm
config:
name: "my_plugin"
configuration:
"@type": "type.googleapis.com/google.protobuf.StringValue"
value: |
{
"consumers": [
{
"name": "consumer1",
"client_id": "9515b564-0b1d-11ee-9c4c-00163e1250b5",
"client_secret": "9e55de56-0b1d-11ee-b8ec-00163e1250b5"
}
],
"clock_skew_seconds": 3153600000
}
vm_config:
runtime: envoy.wasm.runtime.wamr
#runtime: envoy.wasm.runtime.v8
code:
local:
filename: "oauth.aot.wasm"
allow_precompiled: true
The main reasons include:
With the close collaboration between the Higress community and the WAMR community, in addition to improving the Wasm plug-in performance in gateway scenarios, many practical new features will be released soon, so stay tuned:
1. Supports generating CPU flame graph. For example, the following is the CPU flame graph seen when executing Fibonacci recursion in a Wasm plug-in:
2. After a logic problem in Wasm plug-in causes a crash, the complete callstack can be printed in the plug-in log, and the specific line number in the source code can be located through the addr2line tool provided by WAMR.
3. Supports observing the CPU and memory usage of each Wasm plug-in module.
4. Supports using TypeScript to write Wasm plug-ins, complete syntax support.
More developers are welcome to participate in the Higress and WAMR open-source communities.
GitHub project addresses:
Decipher the Knative's Open-Source Serverless Framework: Traffic Perspective
Application Monitoring eBPF Edition: Non-intrusive Application Monitoring of the Golang Microservice
507 posts | 48 followers
FollowAlibaba Cloud Community - April 19, 2024
Alibaba Container Service - April 28, 2020
Alibaba Developer - November 8, 2021
Alibaba Cloud Native Community - September 12, 2023
OpenAnolis - February 2, 2023
feuyeux - May 8, 2021
507 posts | 48 followers
FollowHigh Performance Computing (HPC) and AI technology helps scientific research institutions to perform viral gene sequencing, conduct new drug research and development, and shorten the research and development cycle.
Learn MoreA HPCaaS cloud platform providing an all-in-one high-performance public computing service
Learn MoreConnect your on-premises render farm to the cloud with Alibaba Cloud Elastic High Performance Computing (E-HPC) power and continue business success in a post-pandemic world
Learn MoreAlibaba Cloud Service Mesh (ASM) is a fully managed service mesh platform that is compatible with Istio.
Learn MoreMore Posts by Alibaba Cloud Native Community