Alibaba Cloud Service Mesh:Configure Sidecar proxy

Sidecar proxy resources (istio-proxy)

Set the CPU and memory resource requests and limits for the istio-proxy container.

Configure in the console

1. On the Sidecar Proxy Setting page, select the target scope tab and expand Resource Settings.

2. At the namespace or workload level, select Configure Resources for istio-init Container if you also want to set init container resources.

3. In Resource Limits, set CPU to 2 cores and Memory to 1025 MiB. In Required Resources, set CPU to 0.1 cores and Memory to 128 MiB.

4. Click Update Settings.

5. Redeploy workloads to apply the change.

Verify

kubectl get pod -n <namespace> <pod-name> -o yaml

In the output, confirm that the istio-proxy container's resources field matches the configured values:

spec:
  containers:
    - name: istio-proxy
      resources:
        limits:
          cpu: '2'
          memory: 1025Mi
        requests:
          cpu: 100m
          memory: 128Mi

Proportional sidecar resource allocation

Instead of setting fixed values, allocate sidecar proxy resources as a percentage of the workload container's resources. This overrides the default Istio proxy resource settings.

Two computing policies are available:

• Max container resources: Uses the highest CPU or memory limit among all containers in the Pod as the baseline.

• Specified container: Uses a named container as the baseline. If the named container does not exist, the default Istio proxy resources apply. The Pod annotation scaled-resource.inject.istio.alibabacloud.com/container-ref takes priority over a manually specified container name.

Configure in the console

1. On the Sidecar Proxy Setting page, select a scope tab and expand Resource Settings.

2. Select Configure sidecar resources by ratio.

3. Set the Proportion of Resources percentage, optionally select a Computing Policy, and click Update Settings.

4. Redeploy workloads to apply the change.

Verify

kubectl get pod -n <namespace> <pod-name> -o yaml

Example outcomes with a workload container configured as requests: {cpu: 300m, memory: 512Mi} and limits: {cpu: 500m, memory: 1Gi}:

When the workload container has no requests configured and the proportion is 50%:

resources:
  requests:
    cpu: 100m       # Minimum fallback
    memory: 128Mi   # Minimum fallback
  limits:
    cpu: 250m
    memory: 512Mi

When the workload container has no limits configured and the proportion is 50%:

resources:
  requests:
    cpu: 150m
    memory: 256Mi
  # No limits set

istio-init container resources

Set resource requests and limits for the istio-init container -- the init container that configures iptables traffic interception rules before the sidecar proxy starts.

Configure in the console

1. On the Sidecar Proxy Setting page, select a scope tab and expand Resource Settings.

2. At the namespace or workload level, select Configure Resources for istio-init Container.

3. In Resource Limits, set CPU to 1 core and Memory to 512 MiB. In Required Resources, set CPU to 0.1 cores and Memory to 128 MiB.

4. Click Update Settings.

5. Redeploy workloads to apply the change.

Verify

kubectl get pod -n <namespace> <pod-name> -o yaml

Confirm the istio-init container's resources:

spec:
  initContainers:
    - name: istio-init
      resources:
        limits:
          cpu: '1'
          memory: 512Mi
        requests:
          cpu: 100m
          memory: 128Mi

ACK dynamically overcommitted resources

Allocate dynamically overcommitted resources to the sidecar proxy and istio-init containers. This setting takes effect only when the Pod has the koordinator.sh/qosClass label, which indicates that ACK dynamic resource overcommitment is enabled.

Resource units for dynamically overcommitted resources use kubernetes.io/batch-cpu (in millicores) and kubernetes.io/batch-memory (in MiB).

Configure in the console

1. On the Sidecar Proxy Setting page, select a scope tab and expand Resource Settings.

2. Select Set ACK Resources That Can Be Dynamically Overcommitted for Sidecar Proxy and configure the following:

1. Click Update Settings.

2. Redeploy workloads to apply the change.

Verify

kubectl get pod -n <namespace> <pod-name> -o yaml

Confirm that both containers use batch resource types:

spec:
  containers:
    - name: istio-proxy
      resources:
        limits:
          kubernetes.io/batch-cpu: 2k
          kubernetes.io/batch-memory: 2Gi
        requests:
          kubernetes.io/batch-cpu: '200'
          kubernetes.io/batch-memory: 256Mi
  initContainers:
    - name: istio-init
      resources:
        limits:
          kubernetes.io/batch-cpu: 1k
          kubernetes.io/batch-memory: 1Gi
        requests:
          kubernetes.io/batch-cpu: '100'
          kubernetes.io/batch-memory: 128Mi

Sidecar proxy worker threads

Set the number of Envoy worker threads for the sidecar proxy. The value must be a non-negative integer. When set to 0, the thread count is automatically determined based on the sidecar proxy's CPU resource limits (or resource requests if no limits are set).

Configure in the console

1. On the Sidecar Proxy Setting page, select a scope tab and expand Resource Settings.

2. At the namespace or workload level, set Number of Sidecar Proxy Threads to 3.

3. Click Update Settings.

4. Redeploy workloads to apply the change.

Verify

kubectl get pod -n <namespace> <pod-name> -o yaml

Confirm the --concurrency argument in the istio-proxy container:

spec:
  containers:
    - name: istio-proxy
      args:
        - proxy
        - sidecar
        - '--concurrency'
        - '3'

Outbound traffic: redirect addresses (include list)

Specify IP address ranges in CIDR notation, separated by commas. The sidecar proxy intercepts outbound traffic only to destinations within these ranges. The default value * intercepts all outbound traffic.

Configure in the console

1. Expand Enable/Disable Sidecar Proxy by Ports or IP Addresses.

2. At the namespace or workload level, select Addresses to Which External Access Is Redirected to Sidecar Proxy.

3. Enter 192.168.0.0/16,10.1.0.0/24 and click Update Settings.

4. Redeploy workloads to apply the change.

Verify

The -i parameter in the istio-init container reflects the configured CIDR ranges:

initContainers:
  - name: istio-init
    args:
      - '-i'
      - '192.168.0.0/16,10.1.0.0/24'

Outbound traffic: bypass addresses (exclude list)

Specify IP address ranges in CIDR notation, separated by commas. Outbound traffic to destinations within these ranges bypasses the sidecar proxy.

Configure in the console

1. Expand Enable/Disable Sidecar Proxy by Ports or IP Addresses.

2. At the namespace or workload level, select Addresses to Which External Access Is Not Redirected to Sidecar Proxy.

3. Enter 10.1.0.0/24 and click Update Settings.

4. Redeploy workloads to apply the change.

Verify

The -x parameter in the istio-init container includes the configured CIDR range along with the default host CIDR (192.168.0.1/32):

initContainers:
  - name: istio-init
    args:
      - '-x'
      - '192.168.0.1/32,10.1.0.0/24'

Inbound traffic: redirect ports (include list)

Specify port numbers separated by commas. The sidecar proxy intercepts inbound traffic only on these ports. The default value * intercepts all inbound traffic.

Configure in the console

1. Expand Enable/Disable Sidecar Proxy by Ports or IP Addresses.

2. At the namespace or workload level, select Ports on Which Inbound Traffic Redirected to Sidecar Proxy.

3. Enter 80,443 and click Update Settings.

4. Redeploy workloads to apply the change.

Verify

The -b parameter in the istio-init container reflects the configured ports:

initContainers:
  - name: istio-init
    args:
      - '-b'
      - '80,443'

Inbound traffic: bypass ports (exclude list)

Specify port numbers separated by commas. Inbound traffic on these ports bypasses the sidecar proxy.

By default, the sidecar proxy already excludes its own application ports: 15090, 15021, 15081, 9191, and 15020.

Configure in the console

1. Expand Enable/Disable Sidecar Proxy by Ports or IP Addresses.

2. At the namespace or workload level, select Ports on Which Inbound Traffic Not Redirected to Sidecar Proxy.

3. Enter 8000 and click Update Settings.

4. Redeploy workloads to apply the change.

Verify

The -d parameter in the istio-init container includes port 8000 alongside the default excluded ports:

initContainers:
  - name: istio-init
    args:
      - '-d'
      - '15090,15021,15081,9191,15020,8000'

Outbound traffic: redirect ports (include list)

Specify port numbers separated by commas. The sidecar proxy intercepts outbound traffic only to these destination ports.

Configure in the console

1. Expand Enable/Disable Sidecar Proxy by Ports or IP Addresses.

2. At the namespace or workload level, select Ports on Which Outbound Traffic Redirected to Sidecar Proxy.

3. Enter 80,443 and click Update Settings.

4. Redeploy workloads to apply the change.

Verify

The -q parameter in the istio-init container reflects the configured ports:

initContainers:
  - name: istio-init
    args:
      - '-q'
      - '80,443'

Outbound traffic: bypass ports (exclude list)

Specify port numbers separated by commas. Outbound traffic to these ports bypasses the sidecar proxy, regardless of any redirect address or redirect port settings.

Configure in the console

1. Expand Enable/Disable Sidecar Proxy by Ports or IP Addresses.

2. At the namespace or workload level, select Ports on Which Outbound Traffic Not Redirected to Sidecar Proxy.

3. Enter 8000 and click Update Settings.

4. Redeploy workloads to apply the change.

Verify

The -o parameter in the istio-init container reflects the configured port:

initContainers:
  - name: istio-init
    args:
      - '-o'
      - '8000'

Graceful shutdown (EXIT_ON_ZERO_ACTIVE_CONNECTIONS)

When enabled, the sidecar proxy performs the following during Pod termination:

1. The pilot-agent process stops Envoy from accepting new inbound connections.

2. After a 5-second wait, pilot-agent polls Envoy's active connection count.

3. Once active connections reach zero, pilot-agent terminates the Envoy process.

This reduces dropped requests during termination and minimizes shutdown time.

Configure in the console

1. On the Sidecar Proxy Setting page, select a scope tab and expand Manage Environment Variables for Sidecar Proxy.

2. At the namespace or workload level, select Sidecar Graceful Shutdown.

3. Turn on the switch and click Update Settings.

4. Redeploy workloads to apply the change.

Verify

kubectl get pod -n <namespace> <pod-name> -o yaml

Confirm the environment variable in the istio-proxy container:

env:
  - name: EXIT_ON_ZERO_ACTIVE_CONNECTIONS
    value: 'true'

Graceful startup

By default, the sidecar proxy starts before the application containers in a Pod. This prevents traffic loss that could occur if inbound traffic reaches the application before the sidecar proxy is ready.

Disable this setting to start the sidecar proxy and application containers simultaneously. This can speed up deployments in clusters with many Pods where the API server is under heavy load.

Configure in the console (example: global level)

1. On the Sidecar Proxy Setting page, select the global tab and expand Lifecycle Management.

2. Turn off the switch next to Sidecar Graceful Startup and click Update Settings.

3. Redeploy workloads to apply the change.

Verify

kubectl get pod -n <namespace> <pod-name> -o yaml

When disabled, the istio-proxy container's PROXY_CONFIG environment variable contains "holdApplicationUntilProxyStarts":false, and no default lifecycle field is declared.

Drain duration at Pod termination

After a Pod begins terminating, the sidecar proxy continues processing existing inbound traffic for a configurable period before closing connections. This period is the drain duration. The default is 5s.

If any API served by the workload takes longer than the drain duration to respond, increase this value to prevent in-flight requests from being dropped.

Configure in the console

1. On the Sidecar Proxy Setting page, select a scope tab and expand Lifecycle Management.

2. At the namespace or workload level, select Sidecar Proxy Drain Duration at Pod Termination.

3. Enter 10s and click Update Settings.

4. Redeploy workloads to apply the change.

Verify

kubectl get pod -n <namespace> <pod-name> -o yaml

Confirm the istio-proxy container has the drain duration configured:

env:
  - name: TERMINATION_DRAIN_DURATION_SECONDS
    value: '10'
  - name: PROXY_CONFIG
    value: >-
      {..."terminationDrainDuration":"10s"}

Custom lifecycle hooks

Customize the Kubernetes container lifecycle hooks for the sidecar proxy container by providing the lifecycle field in JSON format. This replaces the default lifecycle configuration.

Configure in the console

1. On the Sidecar Proxy Setting page, select a scope tab and expand Lifecycle Management.

2. At the namespace or workload level, select Lifecycle of Sidecar Proxy.

3. Enter the lifecycle configuration in JSON:

{
  "postStart": {
    "exec": {
      "command": [
        "pilot-agent",
        "wait"
      ]
    }
  },
  "preStop": {
    "exec": {
      "command": [
        "/bin/sh",
        "-c",
        "sleep 13"
      ]
    }
  }
}

In this example:

• postStart: Waits for pilot-agent and Envoy to fully start after the container is created.

• preStop: Sleeps 13 seconds before the container is stopped, allowing in-flight requests to complete.

1. Click Update Settings.

2. Redeploy workloads to apply the change.

Verify

kubectl get pod -n <namespace> <pod-name> -o yaml

Confirm the istio-proxy container has the expected lifecycle hooks:

lifecycle:
  postStart:
    exec:
      command:
        - pilot-agent
        - wait
  preStop:
    exec:
      command:
        - /bin/sh
        - -c
        - sleep 13

Log level

Set the Envoy proxy log level for the sidecar proxy container. The default level is info.

Available levels: debug, trace, info, warning, error, critical, off.

Configure in the console

1. On the Sidecar Proxy Setting page, select a scope tab and expand Monitoring Statistics.

2. At the namespace or workload level, select Log Level.

3. Select error from the drop-down list and click Update Settings.

4. Redeploy workloads to apply the change.

Verify

kubectl get pod -n <namespace> <pod-name> -o yaml

Confirm the --proxyLogLevel argument in the istio-proxy container:

containers:
  - name: istio-proxy
    args:
      - '--proxyLogLevel=error'

Custom Envoy metrics (proxyStatsMatcher)

By default, the sidecar proxy reports only a subset of Envoy metrics to minimize performance overhead. Use proxyStatsMatcher to collect additional metrics by matching metric names with prefixes, suffixes, or regular expressions.

Configure in the console

1. On the Sidecar Proxy Setting page, select a scope tab and expand Monitoring Statistics.

2. Select proxyStatsMatcher and Regular Expression Match.

3. Enter .*outlier_detection.* to collect circuit breaker metrics.

4. Click Update Settings.

5. Redeploy workloads to apply the change.

Verify

kubectl get pod -n <namespace> <pod-name> -o yaml

Confirm the proxyStatsMatcher in the PROXY_CONFIG environment variable:

env:
  - name: PROXY_CONFIG
    value: >-
      {..."proxyStatsMatcher":{"inclusionRegexps":[".*outlier_detection.*"]},...}

Downstream connection limits

By default, the sidecar proxy does not limit downstream connections, which can be exploited in denial-of-service scenarios. See ISTIO-SECURITY-2020-007 for details.

Set a maximum number of downstream connections to protect your workloads.

Configure in the console

1. On the Sidecar Proxy Setting page, select a scope tab.

2. In the Envoy Runtime Parameters section, enter 5000 next to Limits on Downstream Connections and click Update Settings.

3. Redeploy workloads to apply the change.

Verify

kubectl get pod -n <namespace> <pod-name> -o yaml

Confirm the runtimeValues field in the PROXY_CONFIG environment variable:

env:
  - name: PROXY_CONFIG
    value: >-
      {..."runtimeValues":{"overload.global_downstream_max_connections":"5000"},...}