Alibaba Cloud Service Mesh:Access services with WebSocket in ASM

Deploy the WebSocket server and client

1. Deploy the WebSocket server.

   Note

   This topic uses the WebSocket server from the Python library provided by the WebSocket community as an example. To learn how to modify the deployment configuration of the WebSocket server, see Containerize an application.

   1. Create a file named websockets-server.yaml with the following content:

      apiVersion: v1
      kind: Service
      metadata:
        name: websockets-server
        labels:
          app: websockets-server
      spec:
        type: ClusterIP
        ports:
          - port: 8080
            targetPort: 80
            name: http-websocket
        selector:
          app: websockets-server
      ---
      apiVersion: apps/v1
      kind: Deployment
      metadata:
        name: websockets-server
        labels:
          app: websockets-server
      spec:
        selector:
          matchLabels:
            app: websockets-server
        template:
          metadata:
            labels:
              app: websockets-server
          spec:
            containers:
            - name: websockets-test
              image: registry.cn-beijing.aliyuncs.com/aliacs-app-catalog/istio-websockets-test:1.0
              ports:
              - containerPort: 80

   2. Run the following command to deploy the WebSocket server in the default namespace:

      kubectl apply -f websockets-server.yaml -n default

2. Deploy the WebSocket client.

   1. Create a file named websockets-client.yaml with the following content:

      apiVersion: v1
      kind: Service
      metadata:
        name: websockets-client
        labels:
          app: websockets-client
      spec:
        type: ClusterIP
        ports:
          - port: 8080
            targetPort: 80
            name: http-websockets-client
        selector:
          app: websockets-client
      ---
      apiVersion: apps/v1
      kind: Deployment
      metadata:
        name: websockets-client-sleep
        labels:
          app: websockets-client
      spec:
        selector:
          matchLabels:
            app: websockets-client
        template:
          metadata:
            labels:
              app: websockets-client
          spec:
            containers:
            - name: websockets-client
              image: registry.cn-beijing.aliyuncs.com/aliacs-app-catalog/istio-websockets-client-test:1.0
              command: ["sleep", "14d"]

   2. Run the following command to deploy the WebSocket client in the default namespace:

      kubectl apply -f websockets-client.yaml -n default

Use WebSocket over HTTP/1.1

The following steps verify WebSocket access over HTTP/1.1 in ASM. Perform them only if you use WebSocket over HTTP/1.1. If you use WebSocket over HTTP/2, go to Use WebSocket over HTTP/2 instead.

When the WebSocket client sends multiple requests to the server, the sidecar proxy log contains only one access log entry for the WebSocket connection. Envoy treats a WebSocket connection as a TCP byte stream and can interpret only the HTTP upgrade request and response. Therefore, Envoy writes the access log entry only after the connection is closed, and the entry does not show the individual messages that the connection carries.

1. Use one of the following methods to open a shell window in the WebSocket client container:

   • Method 1: Use the console

     1. Log on to the ACK console. In the left navigation pane, click Clusters.

     2. On the Clusters page, click the name of your cluster. In the left navigation pane, click Workloads > Pods.

     3. On the Pods page, find the websockets-client-sleep pod. In the Actions column, click Terminal, and then select the websockets-client container.

   • Method 2: Use kubectl

     Run the following command to open a shell window:

     kubectl exec -it -n default <pod-name> -c websockets-client -- sh

     Replace <pod-name> with the full name of the pod that the websockets-client-sleep deployment creates. The pod name starts with websockets-client-sleep and ends with a generated suffix.

2. Run the following command to access the WebSocket server:

   python3 -m websockets ws://websockets-server.default.svc.cluster.local:8080

   Expected output:

   Connected to ws://websockets-server.default.svc.cluster.local:8080.

   Enter hello and world. The server echoes the messages back.

   > hello
   < hello
   > world
   < world
   Connection closed: 1000 (OK).

3. Check the sidecar proxy logs.

   • Check the sidecar proxy log of the WebSocket client.

     1. On the Pods page, click the name of the WebSocket client pod.

     2. Click the Logs tab and set Container to istio-proxy.

        The log entry shows that the connection uses WebSocket over HTTP/1.1.

        {...."upstream_host":"10.208.0.105:80","bytes_sent":23,"protocol":"HTTP/1.1",....}

   • Check the sidecar proxy log of the WebSocket server.

     1. On the Pods page, click the name of the WebSocket server pod.

     2. Click the Logs tab and set Container to istio-proxy.

        The log entry shows that the connection uses WebSocket over HTTP/1.1.

        {...."downstream_local_address":"10.208.0.105:80","upstream_local_address":"127.0.**.**:53983","protocol":"HTTP/1.1",....}

Both log entries confirm WebSocket access over HTTP/1.1. Perform the steps in Use WebSocket over HTTP/2 only if you want the same deployment to communicate over HTTP/2 instead.

Use WebSocket over HTTP/2

WebSocket over HTTP/2 in ASM requires Istio 1.12 or later. Perform the steps in this section only if you use WebSocket over HTTP/2. If you use WebSocket over HTTP/1.1, see Use WebSocket over HTTP/1.1 instead. If your mesh runs an Istio version earlier than 1.12, see Fall back to HTTP/1.1 on Istio versions earlier than 1.12.

To use WebSocket over HTTP/2 with Istio 1.12 or later, perform the following steps:

1. Create a destination rule.

   1. Log on to the ASM console.

   2. In the left-side navigation pane, choose Service Mesh > Mesh Management.

   3. On the Mesh Management page, find the ASM instance that you want to configure. Click the name of the ASM instance or click Manage in the Actions column.

   4. On the details page of the ASM instance, choose Traffic Management Center > DestinationRule in the left-side navigation pane. On the page that appears, click Create from YAML.

   5. Set Namespaces to default, copy the following content to the text box, and then click Create.

      apiVersion: networking.istio.io/v1beta1
      kind: DestinationRule
      metadata:
        labels:
          provider: asm
        name: websockets-server
      spec:
        host: websockets-server
        trafficPolicy:
          connectionPool:
            http:
              h2UpgradePolicy: UPGRADE

      h2UpgradePolicy: Set this parameter to UPGRADE to enable HTTP/2 for websockets-server. This value is required for WebSocket over HTTP/2. Do not use DO_NOT_UPGRADE here, because that value prevents the upgrade.

2. Create an EnvoyFilter.

   By default, WebSocket does not support HTTP/2. However, Envoy can tunnel WebSocket over HTTP/2 streams, which lets you use a unified HTTP/2 network throughout the deployment. Use an EnvoyFilter to set allow_connect: true for the destination workload. The WebSocket server then accepts HTTP/2 connections.

   1. Log on to the ASM console.

   2. In the left-side navigation pane, choose Service Mesh > Mesh Management.

   3. On the Mesh Management page, find the ASM instance that you want to configure. Click the name of the ASM instance or click Manage in the Actions column.

   4. On the mesh details page, in the left-side navigation pane, choose Plugin Extension Center > Market Place.

   5. On the Market Place page, click setting allow_connect true for HTTP protocol upgrade.

   6. On the Plugin Detail page, click the Create Plug-in Instance tab. In the Plugin Effective scope section, select Workload Scope, and then click Add workloads to effective scope.

   7. In the Add workloads to effective scope dialog box, set Namespaces to default and Workload Type to Deployment. In the Select workloads section, select websockets-server, click the icon to add websockets-server to the selected section, and then click OK.

   8. In the YAML box of the Plugin Config section, enter patch_context: SIDECAR_INBOUND. Then, turn on Plugin Switch and wait until the plugin is enabled.

      After the plugin is enabled, ASM automatically creates an Envoy filter. The following YAML code provides an example of the Envoy filter:

      apiVersion: networking.istio.io/v1alpha3
      kind: EnvoyFilter
      metadata:
        name: h2-upgrade-wss
        labels:
          asm-system: 'true'
          provider: asm
      spec:
        workloadSelector:
          labels:
            app: websockets-server
        configPatches:
        - applyTo: NETWORK_FILTER
          match:
            context: SIDECAR_INBOUND
            proxy:
              proxyVersion: '^1\.*.*'
            listener:
              filterChain:
                filter:
                  name: "envoy.filters.network.http_connection_manager"
          patch:
            operation: MERGE
            value:
              typed_config:
                '@type': type.googleapis.com/envoy.extensions.filters.network.http_connection_manager.v3.HttpConnectionManager
                http2_protocol_options:
                  allow_connect: true

3. Run the following command to access the WebSocket server:

   python3 -m websockets ws://websockets-server.default.svc.cluster.local:8080

   Expected output:

   Connected to ws://websockets-server.default.svc.cluster.local:8080.

   Enter hello and world. The server echoes the messages back.

   > hello
   < hello
   > world
   < world
   Connection closed: 1000 (OK).

4. Check the sidecar proxy logs.

   • Check the sidecar proxy log of the WebSocket client.

     1. On the Pods page, click the name of the WebSocket client pod.

     2. Click the Logs tab and set Container to istio-proxy.

        The log entry shows that the connection uses WebSocket over HTTP/1.1, which indicates that the client initiates requests over HTTP/1.1.

        {...."authority":"websockets-server.default.svc.cluster.local:8080","upstream_service_time":null,"protocol":"HTTP/1.1",....}

   • Check the sidecar proxy log of the WebSocket server.

     1. On the Pods page, click the name of the WebSocket server pod.

     2. Click the Logs tab and set Container to istio-proxy.

        The log entry shows that the connection uses WebSocket over HTTP/2, which indicates that the protocol received by the WebSocket server is upgraded to HTTP/2.

        {...."method":"GET","upstream_local_address":"127.0.**.**:34477","protocol":"HTTP/2",....}

Fall back to HTTP/1.1 on Istio versions earlier than 1.12

If you use WebSocket over HTTP/2 with an Istio version earlier than 1.12, the connection to the WebSocket server fails and a 503 error is returned.

apiVersion: networking.istio.io/v1beta1
kind: DestinationRule
metadata:
  labels:
    provider: asm
  name: websockets-server
spec:
  host: websockets-server
  trafficPolicy:
    connectionPool:
      http:
        h2UpgradePolicy: DO_NOT_UPGRADE