This topic describes how to install and deploy Istio Proxy on a virtual machine. For non-containerized applications that run on virtual machines, you can use Istio Proxy to connect the data plane in which the applications reside to the control plane of an Alibaba Cloud Service Mesh (ASM) instance. This way, you can add the applications that run on the virtual machine to the ASM instance.

Prerequisites

Background information

Istio Proxy runs on virtual machines and communicates with the specified ASM instance to obtain information about discovery services (xDS), intercept the traffic of non-containerized applications, and add the non-containerized applications to the ASM instance. This way, ASM can manage traffic, authenticate requests, and report Tracing Analysis data for the non-containerized applications. Istio Proxy provides Docker image distributions to support virtual machines that run different operating systems.

Step 1: Prepare metadata

Prepare metadata that is required for adding a virtual machine to an ASM instance.

  1. Enter the URL of Cloud Shell in the address bar of a browser or open a CLI in OpenAPI Developer Portal.
  2. Obtain the certificate that is required for adding the virtual machine to the ASM instance.
    1. Log on to the ASM console.
    2. In the left-side navigation pane, choose Service Mesh > Mesh Management.
    3. On the Mesh Management page, find the ASM instance that you want to configure. Click the name of the ASM instance or click Manage in the Actions column of the ASM instance.
    4. On the details page of the ASM instance, choose Data Plane (Service Discovery) > VMs in the left-side navigation pane.
    5. On the VMs page, find the virtual machine for which you want to obtain the certificate and click Certificate in the Actions column.
    6. In the Certificate Information panel, click Copy the following certificate content: and then OK.
  3. Paste the copied certificate content to the /opt/avp/garden/root-cert.pem file of the ECS instance and save the file.
  4. Obtain the workload identity token that is required for adding the virtual machine to the ASM instance.
    1. Log on to the ASM console.
    2. In the left-side navigation pane, choose Service Mesh > Mesh Management.
    3. On the Mesh Management page, find the ASM instance that you want to configure. Click the name of the ASM instance or click Manage in the Actions column of the ASM instance.
    4. On the details page of the ASM instance, choose Data Plane (Service Discovery) > VMs in the left-side navigation pane.
    5. On the VMs page, find the virtual machine for which you want to obtain the workload identity token and click Workload Identity Token in the Actions column.
    6. In the Workload Identity Token panel, select a namespace and click OK.
      Note The namespace of the workload identity token must be the same as that of applications that run on the virtual machine.
    7. In the Workload Identity Token panel, click Copy the content of the following workload identity token: and then OK.
  5. Paste the copied workload identity token to the /var/run/secrets/kubernetes.io/serviceaccount/token file of the ECS instance and save the file.
  6. Create a /opt/avp/garden/avp.out file that contains the following content in the ECS instance and edit the file content as needed:
    XDS_ROOT_CA=/opt/avp/garden/root-cert.pem
    CA_ROOT_CA=/opt/avp/garden/root-cert.pem
    POD_NAMESPACE=default
    SERVICE_ACCOUNT=default
    CANONICAL_SERVICE=productpage
    POD_NAME=productpage-avp
    ISTIO_META_CLUSTER_ID=c88810be***
    JWT_POLICY=first-party-jwt
    ISTIO_META_DNS_CAPTURE=true
    ISTIO_META_ISTIO_VERSION=1.8.3
    SERVICE_CIDR=172.21.0.0/20,240.240.0.0/16

    Replace the first value of the SERVICE_CIDR variable with the CIDR block that is configured when you create an ACK cluster. The second value of the SERVICE_CIDR variable is fixed and is used for the Domain Name System (DNS) proxy feature.

Step 2: Configure iptables

  1. Run the following commands to load metadata to system environment variables and check whether the values of the SERVICE_CIDR variable are valid:
    . /opt/avp/garden/avp.out
    echo "SERVICE_CIDR=$SERVICE_CIDR"

    The following output is expected:

    SERVICE_CIDR=172.21.0.0/20,240.240.0.0/16
  2. Start Istio Proxy with the istio-clean-iptables parameter.
    1. Run the following commands to start Istio Proxy with the istio-clean-iptables parameter and clear the iptables configuration of the virtual machine:
      version=1.8.3
      PROXY_IMAGE=registry-vpc.cn-beijing.aliyuncs.com/acs/proxyv2:${version}
      docker run --rm \
          --name=istio-init \
          --network=host \
          --cap-add=NET_ADMIN \
          $PROXY_IMAGE istio-clean-iptables
    2. Run the following command to view the iptables configuration of the virtual machine:
      iptables -t nat -L -v

      The following output is expected:

      Chain PREROUTING (policy ACCEPT 87 packets, 4331 bytes)
       pkts bytes target     prot opt in     out     source               destination
       4865  248K DOCKER     all  --  any    any     anywhere             anywhere             ADDRTYPE match dst-type LOCAL
      
      Chain INPUT (policy ACCEPT 87 packets, 4331 bytes)
       pkts bytes target     prot opt in     out     source               destination
      
      Chain OUTPUT (policy ACCEPT 32 packets, 2241 bytes)
       pkts bytes target     prot opt in     out     source               destination
        304 18240 DOCKER     all  --  any    any     anywhere            !localhost/8          ADDRTYPE match dst-type LOCAL
      
      Chain POSTROUTING (policy ACCEPT 32 packets, 2241 bytes)
       pkts bytes target     prot opt in     out     source               destination
          0     0 MASQUERADE  all  --  any    !docker0  172.17.0.0/16        anywhere
      
      Chain DOCKER (2 references)
       pkts bytes target     prot opt in     out     source               destination
          0     0 RETURN     all  --  docker0 any     anywhere             anywhere
  3. Start Istio Proxy with the istio-iptables parameter.
    1. Run the following commands to start Istio Proxy with the istio-iptables parameter and set the iptables configuration of the virtual machine:
      version=1.8.3
      PROXY_IMAGE=registry-vpc.cn-beijing.aliyuncs.com/acs/proxyv2:${version}
      docker run --rm \
          --name=istio-init \
          --network=host \
          --cap-add=NET_ADMIN \
          $PROXY_IMAGE istio-iptables \
          -p 15001 -z 15006 -u 1337 -m REDIRECT -i "$SERVICE_CIDR" -x '' -b '*' -d 15020
    2. Run the following command to view the iptables configuration of the virtual machine:
      iptables -t nat -L -v

      The following output is expected:

      Chain PREROUTING (policy ACCEPT 2 packets, 64 bytes)
       pkts bytes target     prot opt in     out     source               destination
       4887  248K DOCKER     all  --  any    any     anywhere             anywhere             ADDRTYPE match dst-type LOCAL
          7   284 ISTIO_INBOUND  tcp  --  any    any     anywhere             anywhere
      
      Chain INPUT (policy ACCEPT 9 packets, 348 bytes)
       pkts bytes target     prot opt in     out     source               destination
      
      Chain OUTPUT (policy ACCEPT 1 packets, 76 bytes)
       pkts bytes target     prot opt in     out     source               destination
        304 18240 DOCKER     all  --  any    any     anywhere            !localhost/8          ADDRTYPE match dst-type LOCAL
          0     0 ISTIO_OUTPUT  tcp  --  any    any     anywhere             anywhere
      
      Chain POSTROUTING (policy ACCEPT 1 packets, 76 bytes)
       pkts bytes target     prot opt in     out     source               destination
          0     0 MASQUERADE  all  --  any    !docker0  172.17.0.0/16        anywhere
      
      Chain DOCKER (2 references)
       pkts bytes target     prot opt in     out     source               destination
          0     0 RETURN     all  --  docker0 any     anywhere             anywhere
      
      Chain ISTIO_INBOUND (1 references)
       pkts bytes target     prot opt in     out     source               destination
          0     0 RETURN     tcp  --  any    any     anywhere             anywhere             tcp dpt:15008
          0     0 RETURN     tcp  --  any    any     anywhere             anywhere             tcp dpt:ssh
          0     0 RETURN     tcp  --  any    any     anywhere             anywhere             tcp dpt:15020
          7   284 ISTIO_IN_REDIRECT  tcp  --  any    any     anywhere             anywhere
      
      Chain ISTIO_IN_REDIRECT (3 references)
       pkts bytes target     prot opt in     out     source               destination
          7   284 REDIRECT   tcp  --  any    any     anywhere             anywhere             redir ports 15006
      
      Chain ISTIO_OUTPUT (1 references)
       pkts bytes target     prot opt in     out     source               destination
          0     0 RETURN     all  --  any    lo      localhost            anywhere
          0     0 ISTIO_IN_REDIRECT  all  --  any    lo      anywhere            !localhost            owner UID match 1337
          0     0 RETURN     all  --  any    lo      anywhere             anywhere             ! owner UID match 1337
          0     0 RETURN     all  --  any    any     anywhere             anywhere             owner UID match 1337
          0     0 ISTIO_IN_REDIRECT  all  --  any    lo      anywhere            !localhost            owner GID match 1337
          0     0 RETURN     all  --  any    lo      anywhere             anywhere             ! owner GID match 1337
          0     0 RETURN     all  --  any    any     anywhere             anywhere             owner GID match 1337
          0     0 RETURN     all  --  any    any     anywhere             localhost
          0     0 ISTIO_REDIRECT  all  --  any    any     anywhere             172.21.0.0/20
          0     0 ISTIO_REDIRECT  all  --  any    any     anywhere             240.240.0.0/16
          0     0 RETURN     all  --  any    any     anywhere             anywhere
      
      Chain ISTIO_REDIRECT (2 references)
       pkts bytes target     prot opt in     out     source               destination
          0     0 REDIRECT   tcp  --  any    any     anywhere             anywhere         

Step 3: Start Istio Proxy

  1. View the endpoint of the control plane of the ASM instance.
    1. Log on to the ASM console.
    2. In the left-side navigation pane, choose Service Mesh > Mesh Management.
    3. On the Mesh Management page, find the ASM instance that you want to configure. Click the name of the ASM instance or click Manage in the Actions column of the ASM instance.
    4. View the endpoint of the control plane in the Basic Information section. The value of the Istio Pilot Endpoint parameter indicates the endpoint of the control plane.
  2. Run the following command to write the endpoint of the control plane of the ASM instance to the /etc/hosts file of the virtual machine:
    echo "<Control plane endpoint> istiod.istio-system.svc" > /opt/avp/garden/avp.hosts
  3. Run the following commands to start Istio Proxy.
    Note You can visit GitHub to view an example on envoy_bootstrap.json.
    version=1.8.3
    PROXY_IMAGE=registry-vpc.cn-beijing.aliyuncs.com/acs/proxyv2:${version}
    docker run -d \
        --name=istio-proxy \
        --network=host \
        --env-file /opt/avp/garden/avp.out \
        -v /opt/avp/garden:/opt/avp/garden \
        -v /var/run/secrets/kubernetes.io/serviceaccount:/var/run/secrets/kubernetes.io/serviceaccount \
        $PROXY_IMAGE proxy \
        --templateFile=/opt/avp/garden/envoy_bootstrap.json

Step 4: Enable the DNS proxy feature

Enable the DNS proxy feature. For more information, see Enable the DNS proxy feature for an ASM instance.

Note
  • If the Istio version of your ASM instance is 1.9 or later, you can enable the DNS proxy feature to simplify the DNS configuration when you add the virtual machine to the ASM instance. For example, you may want to use the helloworld.sample.svc domain name. In this case, you do not need to modify the /etc/hosts file of the virtual machine because the built-in DNS proxy of Istio Proxy automatically resolves the domain name.
  • If the Istio version of your ASM instance is earlier than 1.9, you must add the domain name to the /etc/hosts file of the virtual machine.

Verify the result

View the logs of the istio-proxy container

Run the following command to view the logs of the istio-proxy container. You can learn the processing status of the istio-proxy container from the logs.
docker logs istio-proxy
The following output is expected:
2021-03-29T08:34:33.736997Z info  sds resource:default pushed key/cert pair to proxy
2021-03-29T08:34:34.133533Z info  cache Loaded root cert from certificate ROOTCA
2021-03-29T08:34:34.133735Z info  sds resource:ROOTCA pushed root cert to proxy
View the proxy status of all workloads in the ASM instance
  1. Log on to the ASM console.
  2. In the left-side navigation pane, click Overview. On the Overview page, select the ASM instance from the ASM Instance drop-down list. Then, you can view the proxy status of all workloads in the ASM instance.
    Proxy status
Verify outbound traffic
After you start Istio Proxy, services that run on the virtual machine can access services in the ACK cluster. Outbound traffic
  1. Deploy the Hello World service in the ACK cluster.
    1. Run the following command to create a namespace that is named sample:
      kubectl create namespace sample
    2. Create a YAML file that is named helloworld. For more information about the content of the YAML file, visit GitHub.
    3. Run the following command to create the Hello World service:
      kubectl apply -n sample -f samples/helloworld/helloworld.yaml
  2. Run the following commands to deploy the sleep container on the virtual machine:
    SLEEP_REPO=governmentpaas/curl-ssl
    docker run -d \
        --name=sleep \
        --network=host \
        $SLEEP_REPO /bin/sleep 3650d
  3. Run the following commands to instruct the sleep container on the virtual machine to access the Hello World service in the ACK cluster:
    # sleep[vm] -> helloworld[ack]
    docker exec sleep curl -s helloworld.sample.svc:5000/hello

    The following output is expected:

    Hello version: v1, instance: helloworld-v1-5b75657f75-55255
    
    Hello version: v2, instance: helloworld-v2-7855866d4f-bhxqw
Verify inbound traffic
After you start Istio Proxy, services in the ACK cluster can access services that run on the virtual machine. Inbound traffic
  1. Run the following commands to deploy a container on the virtual machine:
    PRODUCTPAGE_REPO=docker.io/istio/examples-bookinfo-productpage-v1:1.16.2
    
    docker run -d \
        --name=productpage \
        -e SERVICES_DOMAIN=bookinfo \
        --network=host \
        $PRODUCTPAGE_REPO
  2. Add the productpage microservice to the ASM instance.
    1. Use the following content to create a YAML file that is named productpage-se.
      The created service entry adds the productpage microservice to the internal service registry of the ASM instance.
      apiVersion: networking.istio.io/v1beta1
      kind: ServiceEntry
      metadata:
        name: mesh-expansion-productpage
        namespace: bookinfo
      spec:
        hosts:
          - productpage.bookinfo.svc.cluster.local
          - productpage.bookinfo
        location: MESH_INTERNAL
        ports:
          - name: http-9080
            number: 9080
            protocol: HTTP
        resolution: STATIC
        workloadSelector:
          labels:
            app: productpage
    2. Use the following content to create a YAML file that is named productpage-we.
      The created workload entry defines the workload properties of the productpage microservice.
      apiVersion: networking.istio.io/v1beta1
      kind: WorkloadEntry
      metadata:
        name: mesh-expansion-productpage-1
        namespace: bookinfo
      spec:
        address: 10.0.**.**
        labels:
          app: productpage
        serviceAccount: bookinfo-productpage
    3. Run the following commands to create the service entry and workload entry:
      alias m="kubectl --kubeconfig $MESH_CONFIG"
      m apply -n bookinfo -f productpage-se.yaml
      m apply -n bookinfo -f productpage-we.yaml
  3. Deploy the Sleep service in the ACK cluster.
    1. Create a YAML file that is named sleep. For more information about the content of the YAML file, visit GitHub.
    2. Run the following commands to create the Sleep service:
      alias k="kubectl --kubeconfig $ACK_CONFIG"
      k apply -n bookinfo -f ${ISTIO_HOME}/kube/sleep.yaml
  4. Run the following commands to instruct the Sleep service in the ACK cluster to access the productpage microservice that runs on the virtual machine:
    SLEEP_POD=$(k -n bookinfo get pod -l app=sleep -o jsonpath='{.items[0].metadata.name}')
    k -n bookinfo exec $SLEEP_POD -c sleep -- curl -sIv productpage.bookinfo:9080/productpage

    The following output is expected:

    Check productpage:
    *   Trying 240.240.0.2:9080...
    * Connected to productpage.bookinfo (240.240.0.2) port 9080 (#0)
    > HEAD /productpage HTTP/1.1
    > Host: productpage.bookinfo:9080
    > User-Agent: curl/7.69.1
    > Accept: */*
    > 
    HTTP/1.1 200 OK
    content-type: text/html; charset=utf-8
    content-length: 4183
    server: envoy
    date: Wed, 31 Mar 2021 11:51:05 GMT
    x-envoy-upstream-service-time: 111
    
    * Mark bundle as not supporting multiuse
    < HTTP/1.1 200 OK
    < content-type: text/html; charset=utf-8
    < content-length: 4183
    < server: envoy
    < date: Wed, 31 Mar 2021 11:51:05 GMT
    < x-envoy-upstream-service-time: 111
    < 
    * Connection #0 to host productpage.bookinfo left intact

Appendix

The following table describes the regions in which Istio Proxy images that run on virtual machines are available.

Region ID Region
cn-hangzhou China (Hangzhou)
cn-shanghai China (Shanghai)
cn-qingdao China (Qingdao)
cn-beijing China (Beijing)
cn-zhangjiakou China (Zhangjiakou)
cn-huhehaote China (Hohhot)
cn-shenzhen China (Shenzhen)
cn-chengdu China (Chengdu)
cn-hongkong China (Hong Kong)
ap-southeast-1 Singapore (Singapore)
ap-southeast-2 Australia (Sydney)
ap-southeast-3 Malaysia (Kuala Lumpur)
ap-southeast-5 Indonesia (Jakarta)
ap-northeast-1 Japan (Tokyo)
eu-central-1 Germany (Frankfurt)
eu-west-1 UK (London)
us-west-1 US (Silicon Valley)
us-east-1 US (Virginia)
ap-south-1 India (Mumbai)
me-east-1 UAE (Dubai)