Override Sidecar Proxy Settings per Pod via Kubernetes Annotations - Alibaba Cloud Service Mesh

When mesh-wide sidecar proxy defaults do not fit every workload, you can override CPU, memory, startup behavior, traffic interception, and proxy lifecycle settings for individual pods through Kubernetes annotations -- without changing the global mesh configuration.

Two annotation categories are available:

Resource annotations -- Control CPU, memory, and other container-level resource requests and limits for the sidecar proxy (for example, sidecar.istio.io/proxyMemory).
ProxyConfig annotations -- Override proxy runtime behavior such as termination drain duration, concurrency, and startup ordering through the proxy.istio.io/config annotation.

Prerequisites

Before you begin, make sure that you have:

A cluster added to the ASM instance. For details, see Add a cluster to an ASM instance
An ingress gateway deployed. For details, see Create an ingress gateway
Automatic sidecar proxy injection enabled for the target namespace. For details, see Enable automatic sidecar proxy injection
An application deployed in the target namespace. For details, see Deploy an application in an ASM instance

Add resource annotations

Resource annotations control CPU, memory, and other container-level settings for the sidecar proxy. Add these annotations under spec.template.metadata.annotations in the deployment YAML.

Apply annotations through the ACK console

Log on to the ACK console. In the left-side navigation pane, click Clusters.
On the Clusters page, click the name of the cluster that you want to manage and choose Workloads > Deployments in the left-side navigation pane.
On the Deployments page, find the target application and choose More > View in YAML in the Actions column.
In the Edit YAML dialog box, add annotations under the template parameter, then click Update. The following example sets the minimum memory of the sidecar proxy to 14 MiB:
```
annotations:
  sidecar.istio.io/proxyMemory: 14Mi
```

Verify the annotation

In the left-side navigation pane of the cluster details page, choose Workloads > Pods.
On the Pods page, find the pod and click View Details in the Actions column.
On the Container tab, click istio-proxy. The required memory for the istio-proxy container shows 14 MiB.

Set ProxyConfig fields

ProxyConfig fields control proxy runtime behavior -- termination handling, concurrency, and startup ordering. Set these fields through the proxy.istio.io/config annotation in YAML or JSON format.

The procedure for applying ProxyConfig annotations is the same as for resource annotations. See the steps in Add resource annotations.

Example 1: Set the termination drain duration

The terminationDrainDuration field controls how long the istio-proxy container waits before shutting down after the application container terminates. During this period, the proxy drains active connections -- preventing new connections while allowing existing ones to complete.

Add the following annotation to set a 3-second drain duration:

annotations:
  proxy.istio.io/config: |
    terminationDrainDuration: 3s

To verify, navigate to the pod details page: Workloads > Pods > View Details > Container tab > istio-proxy. The termination drain duration shows 3 seconds.

Example 2: Control the sidecar startup sequence

By default, ASM sets holdApplicationUntilProxyStarts to true globally. This prevents the application container from starting until the istio-proxy container is ready to receive traffic. The Istio Container Network Interface (CNI) plug-in or the sidecar injector modifies the pod spec to enforce this ordering.

Keep this set to true for most workloads -- it makes sure that traffic management, monitoring, and security policies take effect as soon as the pod starts receiving traffic. Set it to false only when the application does not depend on the proxy being ready at startup, for example, batch jobs that do not receive inbound traffic.

To verify the startup sequence, navigate to the pod details page: Workloads > Pods > View Details > Events tab. The events show that the istio-proxy container starts before the application container.

Pod events showing istio-proxy start sequence

Istio resource annotations reference

The following table lists supported Istio resource annotations for sidecar proxy configuration.

Annotation	Description
`proxy.istio.io/config`	Overrides the proxy configuration for this specific pod. Accepts YAML or JSON values.
`readiness.status.sidecar.istio.io/applicationPorts`	Comma-separated list of ports exposed by the application container. Used by the sidecar proxy readiness probe to determine whether the proxy is configured and ready to receive traffic.
`readiness.status.sidecar.istio.io/failureThreshold`	Failure threshold for the sidecar proxy readiness probe.
`readiness.status.sidecar.istio.io/initialDelaySeconds`	Initial delay for the sidecar proxy readiness probe, in seconds.
`readiness.status.sidecar.istio.io/periodSeconds`	Period for the sidecar proxy readiness probe, in seconds.
`sidecar.istio.io/componentLogLevel`	Component log level for the sidecar proxy.
`sidecar.istio.io/enableCoreDump`	Whether to enable core dump for the sidecar proxy.
`sidecar.istio.io/extraStatTags`	Comma-separated list of additional tags extracted from in-proxy Istio telemetry.
`sidecar.istio.io/inject`	Whether to enable automatic sidecar proxy injection for this pod.
`sidecar.istio.io/interceptionMode`	Traffic interception mode: `REDIRECT` or `TPROXY`.
`sidecar.istio.io/logLevel`	Log level for the sidecar proxy.
`sidecar.istio.io/proxyCPU`	Requested CPU for the sidecar proxy. Format: Kubernetes resource quantity (for example, `100m`).
`sidecar.istio.io/proxyCPULimit`	CPU limit for the sidecar proxy. Format: Kubernetes resource quantity (for example, `2000m`).
`sidecar.istio.io/proxyMemory`	Requested memory for the sidecar proxy. Format: Kubernetes resource quantity (for example, `128Mi`).
`sidecar.istio.io/proxyMemoryLimit`	Memory limit for the sidecar proxy. Format: Kubernetes resource quantity (for example, `1024Mi`).
`sidecar.istio.io/rewriteAppHTTPProbers`	Whether to rewrite HTTP readiness and liveness probes to redirect through the sidecar proxy.
`status.sidecar.istio.io/port`	HTTP status port of the sidecar proxy. Set to `0` to disable the status endpoint. Default: `15020`.
`traffic.sidecar.istio.io/excludeInboundPorts`	Comma-separated list of inbound ports to exclude from traffic redirection. The wildcard `*` matches all ports.
`traffic.sidecar.istio.io/excludeOutboundIPRanges`	Comma-separated list of CIDR blocks to exclude from outbound traffic redirection. The wildcard `*` matches all CIDR blocks.
`traffic.sidecar.istio.io/excludeOutboundPorts`	Comma-separated list of outbound ports to exclude from traffic redirection.
`traffic.sidecar.istio.io/includeInboundPorts`	Comma-separated list of inbound ports to redirect to the sidecar proxy. The wildcard `*` matches all ports. An empty value disables all inbound redirection.
`traffic.sidecar.istio.io/includeOutboundIPRanges`	Comma-separated list of CIDR blocks to redirect for outbound traffic. The wildcard `*` matches all CIDR blocks. An empty value disables all outbound redirection.
`traffic.sidecar.istio.io/includeOutboundPorts`	Comma-separated list of outbound ports to redirect to the sidecar proxy.
`traffic.sidecar.istio.io/kubevirtInterfaces`	Comma-separated list of virtual interfaces whose inbound traffic is treated as outbound traffic.

ProxyConfig fields reference

Set the following fields through the proxy.istio.io/config annotation to override proxy runtime behavior for individual pods.

Field	Type	Default	Description
`configPath`	String	--	Path to the proxy configuration file directory. The proxy agent generates and stores configuration in this directory.
`statsdUdpAddress`	String	--	IP address and port of a StatsD UDP listener (for example, `127.0.0.1:9125`).
`proxyAdminPort`	Int32	`15000`	Port on which Envoy listens for administrative commands.
`controlPlaneAuthPolicy`	AuthenticationPolicy	`MUTUAL_TLS`	Authentication policy for proxy-to-control-plane connections.
`concurrency`	Int32Value	`2`	Number of worker threads. Set to `0` to use all available cores. If unset, auto-determined based on CPU requests and limits.
`interceptionMode`	InboundInterceptionMode	--	Mode for redirecting inbound traffic to the sidecar proxy.
`tracing`	Tracing	--	Distributed tracing configuration.
`sds`	SDS	--	Secret Discovery Service (SDS) configuration.
`proxyMetadata`	Map<string, string>	--	Additional environment variables for the proxy. Variables with names starting with `ISTIO_META_` are included in the bootstrap configuration and sent to the XDS server.
`statusPort`	Int32	`15020`	Port for administrative commands such as readiness probes.
`terminationDrainDuration`	Duration	`5s`	Time allowed for active connections to complete before the proxy is killed. After receiving SIGTERM or SIGINT, istio-agent starts draining connections, then sleeps for this duration before terminating remaining processes.
`holdApplicationUntilProxyStarts`	BoolValue	`true` (ASM default)	Delays application container startup until the sidecar proxy is ready to accept traffic, preventing startup race conditions.