Alibaba Cloud Service Mesh (ASM) allows you to manage the traffic of microservices in a non-intrusive manner. However, to implement end-to-end A/B testing on a microservice in ASM without changes on the code of the microservice, you must also use WebAssembly (Wasm). This topic shows you how to use ASM and Wasm to implement end-to-end A/B testing in a non-intrusive manner.

Prerequisites

Background information

Wasm is an effective and portable binary instruction format. You can use Wasm to extend the data plane of an ASM instance with new features. For more information about non-intrusive end-to-end A/B testing and Wasm development, see Wasm-based non-intrusive end-to-end A/B testing.
Note The image repository in this topic is for reference only. Use an image script to build and push images to your self-managed image repository. For more information about the image script, visit hello-servicemesh-grpc.

Step 1: Enable Wasm-based ASM instance extension

  1. Create a runtime-config.json file that contains the following code:
    {
      "type": "envoy_proxy",
      "abiVersions": [
        "v0-541b2c1155fffb15ccde92b8324f3e38f7339ba6",
        "v0-097b7f2e4cc1fb490cc1943d0d633655ac3c522f",
        "v0-4689a30309abf31aee9ae36e73d34b1bb182685f",
        "v0.2.1"
      ],
      "config": {
        "rootIds": [
          "propaganda_filter_root"
        ]
      }
    }
  2. Run the following command to push a Wasm filter to an image repository in Container Registry:
    oras push ${WASM_REGISTRY}/propagate_header:0.0.1 \
      --manifest-config \
      --runtime-config.json:application/vnd.module.wasm.config.v1+json \
      ${WASM_IMAGE}:application/vnd.module.wasm.content.layer.v1+wasm
    • WASM_REGISTRY: the address of the image repository.
    • WASM_IMAGE: the file name of the Wasm filter under the current path.
    • runtime-config.json: the runtime configuration file under the current path.
  3. Enable Wasm-based ASM instance extension.
    1. Run the following command to check the version of Alibaba Cloud CLI:
      The version of Alibaba Cloud CLI must be 3.0.73 or later.
      aliyun version
    2. Run the following command to enable Wasm-based ASM instance extension:
      aliyun servicemesh UpdateMeshFeature --ServiceMeshId=xxxxxx --WebAssemblyFilterEnabled=true
  4. Run the following command to check whether Wasm-based ASM instance extension is enabled:
    aliyun servicemesh DescribeServiceMeshDetail \
      --ServiceMeshId $MESH_ID |
      jq '.ServiceMesh.Spec.MeshConfig.WebAssemblyFilterDeployment'

    The following output is expected:

    {
      "Enabled": true
    }
  5. Run the following command to check the status of the asmwasm-cache DaemonSets:
    After Wasm-based ASM instance extension is enabled, a DaemonSet that is named asmwasm-cache is created for each node of the ACK cluster.
    kubectl get daemonset -n istio-system

    The following output is expected:

    NAME            DESIRED   CURRENT   READY   UP-TO-DATE   AVAILABLE   NODE SELECTOR            AGE
    asmwasm-cache   4         4         4       4            4           kubernetes.io/os=linux   34

Step 2: Deploy resources for implementing A/B testing

  1. Create a hello.yaml file that contains the following code in the kube directory:
    The hello.yaml file defines the Hello1, Hello2, and Hello3 applications. Each application has two versions, which are version 1 and version 2.
    Note You can also obtain a YAML file that defines the Hello application from GitHub. For more information, visit Kube.
    apiVersion: apps/v1
    kind: Deployment
    metadata:
      name: hello1-deploy-v1
      labels:
        app: hello1-deploy-v1
        service: hello1-deploy
        version: v1
    spec:
      replicas: 1
      selector:
        matchLabels:
          app: hello1-deploy-v1
          service: hello1-deploy
          version: v1
      template:
        metadata:
          labels:
            app: hello1-deploy-v1
            service: hello1-deploy
            version: v1
        spec:
          serviceAccountName: http-hello-sa
          containers:
            - name: hello-v1-deploy
              image: registry.cn-beijing.aliyuncs.com/asm_repo/http_springboot_v1:1.0.0
              env:
                - name: HTTP_HELLO_BACKEND
                  value: "hello2-svc"
              ports:
                - containerPort: 8001
    ---
    apiVersion: apps/v1
    kind: Deployment
    metadata:
      name: hello1-deploy-v2
      labels:
        app: hello1-deploy-v2
        service: hello1-deploy
        version: v2
    spec:
      replicas: 1
      selector:
        matchLabels:
          app: hello1-deploy-v2
          service: hello1-deploy
          version: v2
      template:
        metadata:
          labels:
            app: hello1-deploy-v2
            service: hello1-deploy
            version: v2
        spec:
          serviceAccountName: http-hello-sa
          containers:
            - name: hello-v2-deploy
              image: registry.cn-beijing.aliyuncs.com/asm_repo/http_springboot_v2:1.0.0
              env:
                - name: HTTP_HELLO_BACKEND
                  value: "hello2-svc"
              ports:
                - containerPort: 8001apiVersion: v1
    ---
    kind: Service
    metadata:
      name: hello1-svc
      labels:
        app: hello1-svc
    spec:
      ports:
        - port: 8001
          name: http
      selector:
        service: hello1-deployapiVersion: apps/v1
    kind: Deployment
    metadata:
      name: hello2-deploy-v1
      labels:
        app: hello2-deploy-v1
        service: hello2-deploy
        version: v1
    spec:
      replicas: 1
      selector:
        matchLabels:
          app: hello2-deploy-v1
          service: hello2-deploy
          version: v1
      template:
        metadata:
          labels:
            app: hello2-deploy-v1
            service: hello2-deploy
            version: v1
        spec:
          serviceAccountName: http-hello-sa
          containers:
            - name: hello-v1-deploy
              image: registry.cn-beijing.aliyuncs.com/asm_repo/http_springboot_v1:1.0.0
              env:
                - name: HTTP_HELLO_BACKEND
                  value: "hello3-svc"
              ports:
                - containerPort: 8001
    ---
    apiVersion: apps/v1
    kind: Deployment
    metadata:
      name: hello2-deploy-v2
      labels:
        app: hello2-deploy-v2
        service: hello2-deploy
        version: v2
    spec:
      replicas: 1
      selector:
        matchLabels:
          app: hello2-deploy-v2
          service: hello2-deploy
          version: v2
      template:
        metadata:
          labels:
            app: hello2-deploy-v2
            service: hello2-deploy
            version: v2
        spec:
          serviceAccountName: http-hello-sa
          containers:
            - name: hello-v2-deploy
              image: registry.cn-beijing.aliyuncs.com/asm_repo/http_springboot_v2:1.0.0
              env:
                - name: HTTP_HELLO_BACKEND
                  value: "hello3-svc"
              ports:
                - containerPort: 8001apiVersion: v1
    ---
    kind: Service
    metadata:
      name: hello2-svc
      labels:
        app: hello2-svc
    spec:
      ports:
        - port: 8001
          name: http
      selector:
        service: hello2-deployapiVersion: apps/v1
    ---
    kind: Deployment
    metadata:
      name: hello3-deploy-v1
      labels:
        app: hello3-deploy-v1
        service: hello3-deploy
        version: v1
    spec:
      replicas: 1
      selector:
        matchLabels:
          app: hello3-deploy-v1
          service: hello3-deploy
          version: v1
      template:
        metadata:
          labels:
            app: hello3-deploy-v1
            service: hello3-deploy
            version: v1
        spec:
          serviceAccountName: http-hello-sa
          containers:
            - name: hello-v1-deploy
              image: registry.cn-beijing.aliyuncs.com/asm_repo/http_springboot_v1:1.0.0
              ports:
                - containerPort: 8001
    ---
    apiVersion: apps/v1
    kind: Deployment
    metadata:
      name: hello3-deploy-v2
      labels:
        app: hello3-deploy-v2
        service: hello3-deploy
        version: v2
    spec:
      replicas: 1
      selector:
        matchLabels:
          app: hello3-deploy-v2
          service: hello3-deploy
          version: v2
      template:
        metadata:
          labels:
            app: hello3-deploy-v2
            service: hello3-deploy
            version: v2
        spec:
          serviceAccountName: http-hello-sa
          containers:
            - name: hello-v2-deploy
              image: registry.cn-beijing.aliyuncs.com/asm_repo/http_springboot_v2:1.0.0
              ports:
                - containerPort: 8001apiVersion: v1
    ---
    kind: Service
    metadata:
      name: hello3-svc
      labels:
        app: hello3-svc
    spec:
      ports:
        - port: 8001
          name: http
      selector:
        service: hello3-deployapiVersion: v1
    ---
    kind: ServiceAccount
    metadata:
      name: http-hello-sa
      labels:
        account: http-hello-deploy
  2. Create a mesh.yaml file that contains the following code in the mesh directory:
    Note You can also obtain a YAML file that defines ingress gateways, destination rules, and virtual services from GitHub. For more information, visit Mesh.

    The mesh.yaml file defines an ingress gateway, three destination rules, and three virtual services.

    The following subsets are defined in the destination rules:
    • hello1v1: the version 1 of the Hello1 application. hello1v2: the version 2 of the Hello1 application.
    • hello2v1: the version 1 of the Hello2 application. hello2v2: the version 2 of the Hello2 application.
    • hello3v1: the version 1 of the Hello3 application. hello3v2: the version 2 of the Hello3 application.
    The following routing rules are configured in the virtual services:
    • Only requests whose headers contain route-v:v2 can be routed to hello1v2. Otherwise, requests are routed to hello1v1.
    • Only requests whose headers contain route-v:hello2v2 can be routed to hello2v2. Otherwise, requests are routed to hello2v1.
    • Only requests whose headers contain route-v:hello3v2 can be routed to hello3v2. Otherwise, requests are routed to hello3v1.
    apiVersion: networking.istio.io/v1alpha3
    kind: DestinationRule
    metadata:
      name: hello1-dr
    spec:
      host: hello1-svc
      subsets:
        - name: hello1v1
          labels:
            version: v1
        - name: hello1v2
          labels:
            version: v2
    ---
    apiVersion: networking.istio.io/v1alpha3
    kind: Gateway
    metadata:
      name: hello-gateway
    spec:
      selector:
        istio: ingressgateway
      servers:
        - port:
            number: 8001
            name: http
            protocol: HTTP
          hosts:
            - "*"
    ---
    # https://istio.io/latest/docs/reference/config/networking/virtual-service/
    apiVersion: networking.istio.io/v1alpha3
    kind: VirtualService
    metadata:
      name: hello1-vs
    spec:
      hosts:
        - "*"
      gateways:
        - hello-gateway
      #  - mesh
      http:
        - name: hello1-v1-route
          match:
            - headers:
                route-v:
                  exact: v2
          route:
            - destination:
                host: hello1-svc
                subset: hello1v2
        - route:
            - destination:
                host: hello1-svc
                subset: hello1v1
    ---
    apiVersion: networking.istio.io/v1alpha3
    kind: DestinationRule
    metadata:
      name: hello2-dr
    spec:
      host: hello2-svc
      subsets:
        - name: hello2v1
          labels:
            version: v1
        - name: hello2v2
          labels:
            version: v2
    ---
    apiVersion: networking.istio.io/v1alpha3
    kind: VirtualService
    metadata:
      name: hello2-vs
    spec:
      hosts:
        - hello2-svc
      http:
      - name: hello2-v2-route
        match:
        - headers:
            route-v:
              exact: hello2v2
        route:
        - destination:
            host: hello2-svc
            subset: hello2v2
      - route:
        - destination:
            host: hello2-svc
            subset: hello2v1
    ---
    apiVersion: networking.istio.io/v1alpha3
    kind: DestinationRule
    metadata:
      name: hello3-dr
    spec:
      host: hello3-svc
      subsets:
        - name: hello3v1
          labels:
            version: v1
        - name: hello3v1
          labels:
            version: v2
        - name: hello3v2
          labels:
            version: v2
    ---
    apiVersion: networking.istio.io/v1alpha3
    kind: VirtualService
    metadata:
      name: hello3-vs
    spec:
      hosts:
      - hello3-svc
      http:
      - match:
        - headers:
            route-v:
              exact: hello3v2
        route:
        - destination:
            host: hello3-svc
            subset: hello3v2
      - route:
        - destination:
            host: hello3-svc
            subset: hello3v1
  3. Run the following command to deploy the Hello application, ingress gateway, virtual services, and destination rules:
    alias k="kubectl --kubeconfig $USER_CONFIG"
    alias m="kubectl --kubeconfig $MESH_CONFIG"
    
    k -n "$NS" apply -f kube/kube.yaml
    m -n "$NS" apply -f mesh/mesh.yaml

Step 3: Deploy a custom ASMFilterDeployment resource

  1. Create a secret for the ACK cluster to access the image repository.
    For more information about the secrets of ACK clusters, see Secret.
    1. Create a myconfig.json file that contains the following code:
      {
          "auths":{
              "**********.cn-hangzhou.cr.aliyuncs.com":{
                  "username":"*****username*****",
                  "password":"*****password*****"
              }
          }
      }
      • **********.cn-hangzhou.cr.aliyuncs.com: the address of the image repository.
      • username: the username of the image repository.
      • password: the password of the image repository.
    2. Run the following command to create a secret:
      Note The secret must be named asmwasm-cache and reside in the istio-system namespace.
      kubectl create secret generic asmwasm-cache -n istio-system --from-file=.dockerconfigjson=myconfig.json --type=kubernetes.io/dockerconfigjson
  2. Deploy the ASMFilterDeployment resource.
    1. Create a hello1-afd.yaml file that contains the following code:
      apiVersion: istio.alibabacloud.com/v1beta1
      kind: ASMFilterDeployment
      metadata:
        name: hello1-propagate-header
      spec:
        workload:
          kind: Deployment
          labels:
            app: hello1-deploy-v2
            version: v2
        filter:
          patchContext: 'SIDECAR_OUTBOUND'
          parameters: '{"head_tag_name": "route-v", "head_tag_value": "hello2v2"}'
          image: 'wasm-repo-registry.cn-beijing.cr.aliyuncs.com/asm_wasm/propagate_header:0.0.1'
          rootID: 'propaganda_filter_root'
          id: 'hello1-propagate-header'
      • Parameters in workload:
        1. kind: the type of the workload.
        2. labels: the filter conditions.
      • Parameters in filter:
        1. patchContext: the context that takes effect.
        2. parameters: the parameters that are required for running the Wasm filter.
        3. image: the address of the image repository to which the Wasm filter is pushed.
        4. rootID: the root ID of the Wasm filter.
        5. id: the unique ID of the Wasm filter.
    2. Create a hello2-afd.yaml file that contains the following code:
      apiVersion: istio.alibabacloud.com/v1beta1
      kind: ASMFilterDeployment
      metadata:
        name: hello2-propagate-header
      spec:
        workload:
          kind: Deployment
          labels:
            app: hello2-deploy-v2
            version: v2
        filter:
          patchContext: 'SIDECAR_OUTBOUND'
          parameters: '{"head_tag_name": "route-v", "head_tag_value": "hello3v2"}'
          image: 'wasm-repo-registry.cn-beijing.cr.aliyuncs.com/asm_wasm/propagate_header:0.0.1'
          rootID: 'propaganda_filter_root'
          id: 'hello2-propagate-header'
      • Parameters in workload:
        1. kind: the type of the workload.
        2. labels: the filter conditions.
      • Parameters in filter:
        1. patchContext: the context that takes effect.
        2. parameters: the parameters that are required for running the Wasm filter.
        3. image: the address of the image repository to which the Wasm filter is pushed.
        4. rootID: the root ID of the Wasm filter.
        5. id: the unique ID of the Wasm filter.
    3. Run the following command to deploy the ASMFilterDeployment resource:
      alias m="kubectl --kubeconfig $MESH_CONFIG"
      
      m apply -f hello1-afd.yaml -n "$NS"
      m apply -f hello2-afd.yaml -n "$NS"
  3. Run the following command to check the deployment of the ASMFilterDeployment resource:
    After the ASMFilterDeployment resource is deployed, ASM automatically generates an Envoy filter.
    alias m="kubectl --kubeconfig $MESH_CONFIG"
    
    m get envoyfilter -n "$NS"
    m get ASMFilterDeployment -n "$NS"

    The following output is expected:

    NAME                      AGE
    hello1-propagate-header   1s
    hello2-propagate-header   0s
    
    NAME                      STATUS      REASON   AGE
    hello1-propagate-header   Available            1s
    hello2-propagate-header   Available            1s

Implement A/B testing

Run the following command to implement A/B testing:

alias k="kubectl --kubeconfig $USER_CONFIG"

ingressGatewayIp=$(k -n istio-system get service istio-ingressgateway -o jsonpath='{.status.loadBalancer.ingress[0].ip}')
for j in {1..3}; do
  curl -H "route-v:v2" "http://$ingressGatewayIp:8001/hello/eric"
  echo
done

The following output is expected:

Bonjour eric@hello1:172.17.68.239<Bonjour eric@hello2:172.17.68.209<Bonjour eric@hello3:172.17.68.208
Bonjour eric@hello1:172.17.68.239<Bonjour eric@hello2:172.17.68.209<Bonjour eric@hello3:172.17.68.208
Bonjour eric@hello1:172.17.68.239<Bonjour eric@hello2:172.17.68.209<Bonjour eric@hello3:172.17.68.208

The output indicates that if the headers of the request contain route-v:v2, the request can be routed to hello1v2, hello2v2, and hello3v2.

Troubleshooting

If the expected output is not returned, you can run the following script code to check the logs of workloads.
  • Check Envoy access logs
    alias k="kubectl --kubeconfig $USER_CONFIG"
    hello1_v2_pod=$(k get pod -l app=hello1-deploy-v2 -n "$NS" -o jsonpath={.items..metadata.name})
    # Change the level of Envoy access logs to info.
    k -n "$NS" exec "$hello1_v2_pod" -c istio-proxy -- curl -XPOST -s "http://localhost:15000/logging?level=info"
    # Display Envoy access logs.
    k -n "$NS" logs -f deployment/hello1-deploy-v2 -c istio-proxy
  • Check the logs of the Hello application
    lias k="kubectl --kubeconfig $USER_CONFIG"
    
    k -n "$NS" logs -f deployment/hello2-deploy-v1 -c hello-v1-deploy