Alibaba Cloud Service Mesh (ASM) allows you to manage the traffic of microservices
in a non-intrusive manner. However, to implement end-to-end A/B testing on a microservice
in ASM without changes on the code of the microservice, you must also use WebAssembly
(Wasm). This topic shows you how to use ASM and Wasm to implement end-to-end A/B testing
in a non-intrusive manner.
Background information
Wasm is an effective and portable binary instruction format. You can use Wasm to extend
the data plane of an ASM instance with new features. For more information about non-intrusive
end-to-end A/B testing and Wasm development, see Wasm-based non-intrusive end-to-end A/B testing.
Note The image repository in this topic is for reference only. Use an image script to build
and push images to your self-managed image repository. For more information about
the image script, visit
hello-servicemesh-grpc.
Step 1: Enable Wasm-based ASM instance extension
- Create a runtime-config.json file that contains the following code:
{
"type": "envoy_proxy",
"abiVersions": [
"v0-541b2c1155fffb15ccde92b8324f3e38f7339ba6",
"v0-097b7f2e4cc1fb490cc1943d0d633655ac3c522f",
"v0-4689a30309abf31aee9ae36e73d34b1bb182685f",
"v0.2.1"
],
"config": {
"rootIds": [
"propaganda_filter_root"
]
}
}
- Run the following command to push a Wasm filter to an image repository in Container
Registry:
oras push ${WASM_REGISTRY}/propagate_header:0.0.1 \
--manifest-config \
--runtime-config.json:application/vnd.module.wasm.config.v1+json \
${WASM_IMAGE}:application/vnd.module.wasm.content.layer.v1+wasm
WASM_REGISTRY: the address of the image repository.
WASM_IMAGE: the file name of the Wasm filter under the current path.
runtime-config.json: the runtime configuration file under the current path.
- Enable Wasm-based ASM instance extension.
- Run the following command to check the version of Alibaba Cloud CLI:
The version of Alibaba Cloud CLI must be 3.0.73 or later.
aliyun version
- Run the following command to enable Wasm-based ASM instance extension:
aliyun servicemesh UpdateMeshFeature --ServiceMeshId=xxxxxx --WebAssemblyFilterEnabled=true
- Run the following command to check whether Wasm-based ASM instance extension is enabled:
aliyun servicemesh DescribeServiceMeshDetail \
--ServiceMeshId $MESH_ID |
jq '.ServiceMesh.Spec.MeshConfig.WebAssemblyFilterDeployment'
The following output is expected:
{
"Enabled": true
}
- Run the following command to check the status of the asmwasm-cache DaemonSets:
After Wasm-based ASM instance extension is enabled, a DaemonSet that is named asmwasm-cache
is created for each node of the ACK cluster.
kubectl get daemonset -n istio-system
The following output is expected:
NAME DESIRED CURRENT READY UP-TO-DATE AVAILABLE NODE SELECTOR AGE
asmwasm-cache 4 4 4 4 4 kubernetes.io/os=linux 34
Step 2: Deploy resources for implementing A/B testing
- Create a hello.yaml file that contains the following code in the kube directory:
The hello.yaml file defines the Hello1, Hello2, and Hello3 applications. Each application
has two versions, which are version 1 and version 2.
Note You can also obtain a YAML file that defines the Hello application from GitHub. For
more information, visit
Kube.
apiVersion: apps/v1
kind: Deployment
metadata:
name: hello1-deploy-v1
labels:
app: hello1-deploy-v1
service: hello1-deploy
version: v1
spec:
replicas: 1
selector:
matchLabels:
app: hello1-deploy-v1
service: hello1-deploy
version: v1
template:
metadata:
labels:
app: hello1-deploy-v1
service: hello1-deploy
version: v1
spec:
serviceAccountName: http-hello-sa
containers:
- name: hello-v1-deploy
image: registry.cn-beijing.aliyuncs.com/asm_repo/http_springboot_v1:1.0.0
env:
- name: HTTP_HELLO_BACKEND
value: "hello2-svc"
ports:
- containerPort: 8001
---
apiVersion: apps/v1
kind: Deployment
metadata:
name: hello1-deploy-v2
labels:
app: hello1-deploy-v2
service: hello1-deploy
version: v2
spec:
replicas: 1
selector:
matchLabels:
app: hello1-deploy-v2
service: hello1-deploy
version: v2
template:
metadata:
labels:
app: hello1-deploy-v2
service: hello1-deploy
version: v2
spec:
serviceAccountName: http-hello-sa
containers:
- name: hello-v2-deploy
image: registry.cn-beijing.aliyuncs.com/asm_repo/http_springboot_v2:1.0.0
env:
- name: HTTP_HELLO_BACKEND
value: "hello2-svc"
ports:
- containerPort: 8001apiVersion: v1
---
kind: Service
metadata:
name: hello1-svc
labels:
app: hello1-svc
spec:
ports:
- port: 8001
name: http
selector:
service: hello1-deployapiVersion: apps/v1
kind: Deployment
metadata:
name: hello2-deploy-v1
labels:
app: hello2-deploy-v1
service: hello2-deploy
version: v1
spec:
replicas: 1
selector:
matchLabels:
app: hello2-deploy-v1
service: hello2-deploy
version: v1
template:
metadata:
labels:
app: hello2-deploy-v1
service: hello2-deploy
version: v1
spec:
serviceAccountName: http-hello-sa
containers:
- name: hello-v1-deploy
image: registry.cn-beijing.aliyuncs.com/asm_repo/http_springboot_v1:1.0.0
env:
- name: HTTP_HELLO_BACKEND
value: "hello3-svc"
ports:
- containerPort: 8001
---
apiVersion: apps/v1
kind: Deployment
metadata:
name: hello2-deploy-v2
labels:
app: hello2-deploy-v2
service: hello2-deploy
version: v2
spec:
replicas: 1
selector:
matchLabels:
app: hello2-deploy-v2
service: hello2-deploy
version: v2
template:
metadata:
labels:
app: hello2-deploy-v2
service: hello2-deploy
version: v2
spec:
serviceAccountName: http-hello-sa
containers:
- name: hello-v2-deploy
image: registry.cn-beijing.aliyuncs.com/asm_repo/http_springboot_v2:1.0.0
env:
- name: HTTP_HELLO_BACKEND
value: "hello3-svc"
ports:
- containerPort: 8001apiVersion: v1
---
kind: Service
metadata:
name: hello2-svc
labels:
app: hello2-svc
spec:
ports:
- port: 8001
name: http
selector:
service: hello2-deployapiVersion: apps/v1
---
kind: Deployment
metadata:
name: hello3-deploy-v1
labels:
app: hello3-deploy-v1
service: hello3-deploy
version: v1
spec:
replicas: 1
selector:
matchLabels:
app: hello3-deploy-v1
service: hello3-deploy
version: v1
template:
metadata:
labels:
app: hello3-deploy-v1
service: hello3-deploy
version: v1
spec:
serviceAccountName: http-hello-sa
containers:
- name: hello-v1-deploy
image: registry.cn-beijing.aliyuncs.com/asm_repo/http_springboot_v1:1.0.0
ports:
- containerPort: 8001
---
apiVersion: apps/v1
kind: Deployment
metadata:
name: hello3-deploy-v2
labels:
app: hello3-deploy-v2
service: hello3-deploy
version: v2
spec:
replicas: 1
selector:
matchLabels:
app: hello3-deploy-v2
service: hello3-deploy
version: v2
template:
metadata:
labels:
app: hello3-deploy-v2
service: hello3-deploy
version: v2
spec:
serviceAccountName: http-hello-sa
containers:
- name: hello-v2-deploy
image: registry.cn-beijing.aliyuncs.com/asm_repo/http_springboot_v2:1.0.0
ports:
- containerPort: 8001apiVersion: v1
---
kind: Service
metadata:
name: hello3-svc
labels:
app: hello3-svc
spec:
ports:
- port: 8001
name: http
selector:
service: hello3-deployapiVersion: v1
---
kind: ServiceAccount
metadata:
name: http-hello-sa
labels:
account: http-hello-deploy
- Create a mesh.yaml file that contains the following code in the mesh directory:
Note You can also obtain a YAML file that defines ingress gateways, destination rules,
and virtual services from GitHub. For more information, visit
Mesh.
The mesh.yaml file defines an ingress gateway, three destination rules, and three
virtual services.
The following subsets are defined in the destination rules:
- hello1v1: the version 1 of the Hello1 application. hello1v2: the version 2 of the
Hello1 application.
- hello2v1: the version 1 of the Hello2 application. hello2v2: the version 2 of the
Hello2 application.
- hello3v1: the version 1 of the Hello3 application. hello3v2: the version 2 of the
Hello3 application.
The following routing rules are configured in the virtual services:
- Only requests whose headers contain route-v:v2 can be routed to hello1v2. Otherwise,
requests are routed to hello1v1.
- Only requests whose headers contain route-v:hello2v2 can be routed to hello2v2. Otherwise,
requests are routed to hello2v1.
- Only requests whose headers contain route-v:hello3v2 can be routed to hello3v2. Otherwise,
requests are routed to hello3v1.
apiVersion: networking.istio.io/v1alpha3
kind: DestinationRule
metadata:
name: hello1-dr
spec:
host: hello1-svc
subsets:
- name: hello1v1
labels:
version: v1
- name: hello1v2
labels:
version: v2
---
apiVersion: networking.istio.io/v1alpha3
kind: Gateway
metadata:
name: hello-gateway
spec:
selector:
istio: ingressgateway
servers:
- port:
number: 8001
name: http
protocol: HTTP
hosts:
- "*"
---
# https://istio.io/latest/docs/reference/config/networking/virtual-service/
apiVersion: networking.istio.io/v1alpha3
kind: VirtualService
metadata:
name: hello1-vs
spec:
hosts:
- "*"
gateways:
- hello-gateway
# - mesh
http:
- name: hello1-v1-route
match:
- headers:
route-v:
exact: v2
route:
- destination:
host: hello1-svc
subset: hello1v2
- route:
- destination:
host: hello1-svc
subset: hello1v1
---
apiVersion: networking.istio.io/v1alpha3
kind: DestinationRule
metadata:
name: hello2-dr
spec:
host: hello2-svc
subsets:
- name: hello2v1
labels:
version: v1
- name: hello2v2
labels:
version: v2
---
apiVersion: networking.istio.io/v1alpha3
kind: VirtualService
metadata:
name: hello2-vs
spec:
hosts:
- hello2-svc
http:
- name: hello2-v2-route
match:
- headers:
route-v:
exact: hello2v2
route:
- destination:
host: hello2-svc
subset: hello2v2
- route:
- destination:
host: hello2-svc
subset: hello2v1
---
apiVersion: networking.istio.io/v1alpha3
kind: DestinationRule
metadata:
name: hello3-dr
spec:
host: hello3-svc
subsets:
- name: hello3v1
labels:
version: v1
- name: hello3v1
labels:
version: v2
- name: hello3v2
labels:
version: v2
---
apiVersion: networking.istio.io/v1alpha3
kind: VirtualService
metadata:
name: hello3-vs
spec:
hosts:
- hello3-svc
http:
- match:
- headers:
route-v:
exact: hello3v2
route:
- destination:
host: hello3-svc
subset: hello3v2
- route:
- destination:
host: hello3-svc
subset: hello3v1
- Run the following command to deploy the Hello application, ingress gateway, virtual
services, and destination rules:
alias k="kubectl --kubeconfig $USER_CONFIG"
alias m="kubectl --kubeconfig $MESH_CONFIG"
k -n "$NS" apply -f kube/kube.yaml
m -n "$NS" apply -f mesh/mesh.yaml
Step 3: Deploy a custom ASMFilterDeployment resource
- Create a secret for the ACK cluster to access the image repository.
For more information about the secrets of ACK clusters, see
Secret.
- Create a myconfig.json file that contains the following code:
{
"auths":{
"**********.cn-hangzhou.cr.aliyuncs.com":{
"username":"*****username*****",
"password":"*****password*****"
}
}
}
**********.cn-hangzhou.cr.aliyuncs.com: the address of the image repository.
username: the username of the image repository.
password: the password of the image repository.
- Run the following command to create a secret:
Note The secret must be named asmwasm-cache and reside in the istio-system namespace.
kubectl create secret generic asmwasm-cache -n istio-system --from-file=.dockerconfigjson=myconfig.json --type=kubernetes.io/dockerconfigjson
- Deploy the ASMFilterDeployment resource.
- Create a hello1-afd.yaml file that contains the following code:
apiVersion: istio.alibabacloud.com/v1beta1
kind: ASMFilterDeployment
metadata:
name: hello1-propagate-header
spec:
workload:
kind: Deployment
labels:
app: hello1-deploy-v2
version: v2
filter:
patchContext: 'SIDECAR_OUTBOUND'
parameters: '{"head_tag_name": "route-v", "head_tag_value": "hello2v2"}'
image: 'wasm-repo-registry.cn-beijing.cr.aliyuncs.com/asm_wasm/propagate_header:0.0.1'
rootID: 'propaganda_filter_root'
id: 'hello1-propagate-header'
- Parameters in
workload:
kind: the type of the workload.
labels: the filter conditions.
- Parameters in
filter:
patchContext: the context that takes effect.
parameters: the parameters that are required for running the Wasm filter.
image: the address of the image repository to which the Wasm filter is pushed.
rootID: the root ID of the Wasm filter.
id: the unique ID of the Wasm filter.
- Create a hello2-afd.yaml file that contains the following code:
apiVersion: istio.alibabacloud.com/v1beta1
kind: ASMFilterDeployment
metadata:
name: hello2-propagate-header
spec:
workload:
kind: Deployment
labels:
app: hello2-deploy-v2
version: v2
filter:
patchContext: 'SIDECAR_OUTBOUND'
parameters: '{"head_tag_name": "route-v", "head_tag_value": "hello3v2"}'
image: 'wasm-repo-registry.cn-beijing.cr.aliyuncs.com/asm_wasm/propagate_header:0.0.1'
rootID: 'propaganda_filter_root'
id: 'hello2-propagate-header'
- Parameters in
workload:
kind: the type of the workload.
labels: the filter conditions.
- Parameters in
filter:
patchContext: the context that takes effect.
parameters: the parameters that are required for running the Wasm filter.
image: the address of the image repository to which the Wasm filter is pushed.
rootID: the root ID of the Wasm filter.
id: the unique ID of the Wasm filter.
- Run the following command to deploy the ASMFilterDeployment resource:
alias m="kubectl --kubeconfig $MESH_CONFIG"
m apply -f hello1-afd.yaml -n "$NS"
m apply -f hello2-afd.yaml -n "$NS"
- Run the following command to check the deployment of the ASMFilterDeployment resource:
After the ASMFilterDeployment resource is deployed, ASM automatically generates an
Envoy filter.
alias m="kubectl --kubeconfig $MESH_CONFIG"
m get envoyfilter -n "$NS"
m get ASMFilterDeployment -n "$NS"
The following output is expected:
NAME AGE
hello1-propagate-header 1s
hello2-propagate-header 0s
NAME STATUS REASON AGE
hello1-propagate-header Available 1s
hello2-propagate-header Available 1s
Implement A/B testing
Run the following command to implement A/B testing:
alias k="kubectl --kubeconfig $USER_CONFIG"
ingressGatewayIp=$(k -n istio-system get service istio-ingressgateway -o jsonpath='{.status.loadBalancer.ingress[0].ip}')
for j in {1..3}; do
curl -H "route-v:v2" "http://$ingressGatewayIp:8001/hello/eric"
echo
done
The following output is expected:
Bonjour eric@hello1:172.17.68.239<Bonjour eric@hello2:172.17.68.209<Bonjour eric@hello3:172.17.68.208
Bonjour eric@hello1:172.17.68.239<Bonjour eric@hello2:172.17.68.209<Bonjour eric@hello3:172.17.68.208
Bonjour eric@hello1:172.17.68.239<Bonjour eric@hello2:172.17.68.209<Bonjour eric@hello3:172.17.68.208
The output indicates that if the headers of the request contain route-v:v2, the request
can be routed to hello1v2, hello2v2, and hello3v2.
Troubleshooting
If the expected output is not returned, you can run the following script code to check
the logs of workloads.
- Check Envoy access logs
alias k="kubectl --kubeconfig $USER_CONFIG"
hello1_v2_pod=$(k get pod -l app=hello1-deploy-v2 -n "$NS" -o jsonpath={.items..metadata.name})
# Change the level of Envoy access logs to info.
k -n "$NS" exec "$hello1_v2_pod" -c istio-proxy -- curl -XPOST -s "http://localhost:15000/logging?level=info"
# Display Envoy access logs.
k -n "$NS" logs -f deployment/hello1-deploy-v2 -c istio-proxy
- Check the logs of the Hello application
lias k="kubectl --kubeconfig $USER_CONFIG"
k -n "$NS" logs -f deployment/hello2-deploy-v1 -c hello-v1-deploy