Fix Chrome WebSocket Error 1006 & Restore Connections via EnvoyFilter - Alibaba Cloud Service Mesh

When an Istio sidecar proxy intercepts WebSocket traffic, Google Chrome may receive return code 1006 instead of the expected default (1005) or a custom return code. Firefox and Safari are not affected. To fix this, set delayed_close_timeout: 0s in an EnvoyFilter.

Cause

The Envoy sidecar proxy's HTTP connection manager uses delayed_close_timeout to drain connections gracefully. During a WebSocket close handshake, this delay causes Envoy to reset the connection before Chrome receives the close frame, resulting in "wasclean": false and return code 1006. Other browsers handle the connection timing differently and receive the correct code.

Prerequisites

Before you begin, make sure that you have:

An ACK managed cluster. For more information, see Create an ACK managed cluster
A Service Mesh (ASM) instance with the ACK cluster added. For more information, see Create an ASM instance and Add a cluster to an ASM instance
At least one ingress gateway service deployed. For more information, see Create an ingress gateway
An application deployed in the ASM instance. For more information, see Deploy an application in an ASM instance
kubectl configured to connect to both the ACK cluster and the ASM instance. For more information, see Connect to ACK clusters by using kubectl and Use kubectl on the control plane to access Istio resources

Step 1: Deploy a sample WebSocket application

Deploy a WebSocket server to reproduce the issue. Choose one of the following methods.

Option A: Use the prebuilt Alibaba Cloud image

Create a file named sample.yaml with the following content:

    apiVersion: apps/v1
    kind: Deployment
    metadata:
      name: websocket-test
      namespace: default
      labels:
        app: websocket-test
        version: current
    spec:
      replicas: 1
      selector:
        matchLabels:
          app: websocket-test
          version: current
      template:
        metadata:
          labels:
            app: websocket-test
            version: current
        spec:
          containers:
          - name: websocket-test
            image: registry.cn-hangzhou.aliyuncs.com/aliacs-app-catalog/asm-websocketsample:v1
            imagePullPolicy: Always
            command: ["node", "ws.js"]

    ---
    apiVersion: v1
    kind: Service
    metadata:
      labels:
        app: websocket-test
      name: websocket-test
      namespace: default
    spec:
      ports:
      - name: http
        port: 80
        protocol: TCP
        targetPort: 9900
      selector:
        app: websocket-test
      type: ClusterIP

Apply the manifest to the default namespace:
Automatic sidecar injection must be enabled for the default namespace. For more information, see Enable automatic sidecar proxy injection.
```
    kubectl apply -f sample.yaml
```

Option B: Build a custom Docker image

Create a package.json file for the Node.js application:

    {
      "name": "wssample",
      "version": "0.0.1",
      "main": "ws.js",
      "license": "UNLICENSED",
      "scripts": {
        "start": "node --trace-warnings ./ws.js"
      },
      "dependencies": {
        "ws": "^8.0.0"
      }
    }

Create a ws.js file with the WebSocket server logic:

    const WebSocket = require("ws");
    const http = require("http");
    const wss = new WebSocket.Server({ noServer: true });

    const server = http.createServer()

    server.on("upgrade", async (request, socket, head) => {
      const handleAuth = (ws) => {
        wss.emit("connection", ws, request);
      };
      wss.handleUpgrade(request, socket, head, handleAuth);
    })

    wss.on("connection", (conn, req) => {
      // Default close sends return code 1005.
      // With sidecar injection enabled, Chrome reports 1006 instead.
      // conn.close()

      // Custom return code 4321.
      // With sidecar injection enabled, Chrome still reports 1006.
      // Without sidecar injection, Chrome correctly reports 4321.
      conn.close(4321, "test")
    });

    server.listen({ host: '0.0.0.0', port: 9900 });

Create a Dockerfile:

    FROM node:16.7.0-alpine3.14
    WORKDIR /root/app
    COPY . .
    RUN yarn install

Build the image and deploy it by using a Deployment and Service similar to Option A. Replace the image field with your custom image registry path.

Step 2: Configure the ASM instance

Create a Gateway, DestinationRule, and VirtualService to route traffic to the WebSocket application through the ingress gateway.

Log on to the ASM console.
In the left-side navigation pane, choose Service Mesh > Mesh Management.
On the Mesh Management page, click the name of the target ASM instance, or click Manage in the Actions column.

Create a Gateway

In the left-side navigation pane, choose ASM Gateways > Gateway. Click Create from YAML.

Set Namespace to default, select a scenario template, paste the following YAML, and click Create:

    apiVersion: networking.istio.io/v1beta1
    kind: Gateway
    metadata:
      name: websocket-test
      namespace: default
    spec:
      selector:
        istio: ingressgateway
      servers:
        - hosts:
            - '*'
          port:
            name: http
            number: 80
            protocol: HTTP

Create a DestinationRule

In the left-side navigation pane, choose Traffic Management Center > DestinationRule. Click Create from YAML.

Set Namespace to default, select a scenario template, paste the following YAML, and click Create:

    apiVersion: networking.istio.io/v1alpha3
    kind: DestinationRule
    metadata:
      name: websocket-test
      namespace: default
    spec:
      host: websocket-test
      subsets:
      - name: current
        labels:
          version: current

Create a VirtualService

In the left-side navigation pane, choose Traffic Management Center > VirtualService. Click Create from YAML.

Set Namespace to default, select a scenario template, paste the following YAML, and click Create:

    apiVersion: networking.istio.io/v1alpha3
    kind: VirtualService
    metadata:
      name: websocket-test
      namespace: default
    spec:
      gateways:
      - websocket-test
      hosts:
      - '*'
      http:
      - name: default
        route:
        - destination:
            host: websocket-test
            subset: current

Step 3: Reproduce the issue

Use a simple HTML client to confirm that Chrome returns 1006 instead of the expected code.

Create a client.html file with the following content. Replace <ingress-gateway-ip> with the IP address of your ingress gateway:

    <!DOCTYPE html>
    <html>
    <head>
        <title>WebSocket example</title>
    </head>
    <body>
        <script>
            var ws = new WebSocket('ws://<ingress-gateway-ip>');

            ws.onopen = function (ev) {
                console.log(ev)
            };
            ws.onmessage = function (ev) {
                console.log("on msg", ev)
            };
            ws.onclose = function (ev) {
                console.log("on close", ev)
            };
            ws.onerror = function (ev) {
                console.log("on error", ev)
            };
        </script>
    </body>
    </html>

Open client.html in Google Chrome and press F12 to open Developer Tools.
Refresh the page and check the Console tab. The return code is 1006 instead of the expected custom return code 4321.

Step 4: Apply the EnvoyFilter fix

Set delayed_close_timeout to 0s in the Envoy HTTP connection manager to prevent the sidecar from interfering with WebSocket close frames.

Create an EnvoyFilter with the following content: This EnvoyFilter patches all Envoy sidecars (proxy version ^1.*.*) by merging a delayed_close_timeout: 0s setting into the HTTP connection manager network filter.

    apiVersion: networking.istio.io/v1alpha3
    kind: EnvoyFilter
    metadata:
      labels:
        asm-system: 'true'
        provider: asm
      name: hack-to-fix-delayedclosetimeout-istio-upper-version
      namespace: istio-system
    spec:
      configPatches:
        - applyTo: NETWORK_FILTER
          match:
            listener:
              filterChain:
                filter:
                  name: envoy.filters.network.http_connection_manager
            proxy:
              proxyVersion: ^1.*.*
          patch:
            operation: MERGE
            value:
              typed_config:
                '@type': >-
                  type.googleapis.com/envoy.extensions.filters.network.http_connection_manager.v3.HttpConnectionManager
                delayed_close_timeout: 0s

Bind the EnvoyFilter to the target workload or namespace. For more information, see Bind an Envoy filter template to a workload or namespace.

Step 5: Verify the fix

Open client.html in Google Chrome and press F12 to open Developer Tools.
Refresh the page and check the Console tab. The return code is now 4321, which matches the custom code set in the WebSocket server.