Override Sidecar Proxy Settings per Pod via Kubernetes Annotations - Alibaba Cloud Service Mesh - Alibaba Cloud - Alibaba Cloud Service Mesh

ASM supports modifying Sidecar proxy resources and configuration using annotations. This topic describes how to configure the Sidecar Proxy by adding resource annotations and using the ProxyConfig field.

Prerequisites

The cluster is added to the ASM instance.
An ingress gateway is deployed. For more information, see Create an ingress gateway.

Add resource annotations

ASM supports modifying Sidecar proxy resources by adding annotations to a pod. This example shows how to modify the Sidecar proxy memory request. For more information about other annotations, see Appendix A: Istio resource annotations.

Enable automatic Sidecar mesh proxy injection for the default namespace. For more information, see Enable automatic injection.
Deploy an application in the default namespace. For more information, see Deploy an application in a cluster associated with an ASM instance.
Modify the minimum memory request for the Sidecar proxy.
1. Log on to the ACK console. In the left navigation pane, click Clusters.
2. On the Clusters page, click the name of your cluster. In the left navigation pane, click Workloads > Deployments.
3. On the Deployments page, find your target application and click More > View in YAML in the Actions column.
4. In the Edit YAML dialog box, add the following content under the template parameter, then click Update.
```
annotations:
  sidecar.istio.io/proxyMemory: 14Mi
```
```
template:
    metadata:
      annotations:
        sidecar.istio.io/proxyMemory: 14Mi
      labels:
        app: details
        version: v1
```
Verify that the Sidecar proxy memory modification succeeded.
1. In the left navigation pane of the cluster management page, choose Workload > Pods.
2. On the Pods page, find your target application pod and click Details in the Actions column.
3. On the Containers tab, click istio-proxy.
  
  You can see that the istio-proxy requests 14 Mi of memory. After clicking, you can view details of the istio-proxy container, including the registry address, image pull policy (IfNotPresent), and Istio Sidecar-related environment variables. Under Requested Resources, you will see the current memory setting as Memory: 14Mi.

Set the ProxyConfig field

ASM supports defining Sidecar proxy configuration by modifying the ProxyConfig field in a pod. For more information about using the ProxyConfig field, see Appendix B: ProxyConfig fields.

Note

proxy.istio.io/config values can use either YAML or JSON format.

The procedure for setting the ProxyConfig field is the same as adding resource annotations. This section provides common ProxyConfig field examples. For steps, see the previous section.

Example 1: Set termination drain duration

Use the terminationDrainDuration field to modify the termination drain duration. This duration defines the maximum time Istio Proxy waits after an application container terminates.

Add the following content to your application YAML file to make Istio Proxy wait up to 3 seconds after the application container terminates.

annotations:
  proxy.istio.io/config: |
    terminationDrainDuration: 3s

type: RollingUpdate
  template:
    metadata:
      annotations:
        proxy.istio.io/config: |
          terminationDrainDuration: 3s
      labels:
        app: details

On the Pods page, find your target application pod and click Actions in the Details column. On the Containers tab, click istio-proxy. You will see the termination drain duration is set to 3 s. After the configuration takes effect, view the istio-proxy container details on the Pod’s container page. In the PROXY_CONFIG environment variable, terminationDrainDuration is set to 3s.

Example 2: Set Istio Proxy startup order

By default, ASM globally sets holdApplicationUntilProxyStarts to true. This ensures the Istio Proxy starts successfully before the application container starts. If the application container becomes ready before the Istio Proxy is ready, the pod fails to accept traffic.

holdApplicationUntilProxyStarts is a configuration option that controls application container startup behavior after Sidecar proxy injection. When holdApplicationUntilProxyStarts is set to true, Istio CNI or the Sidecar injector configures the Kubernetes pod to prevent the application container from starting until the proxy container is ready and begins receiving traffic. This ensures all traffic flows through the proxy before the pod receives any traffic, enabling immediate enforcement of traffic management, monitoring, and security policies. In most cases, set holdApplicationUntilProxyStarts to true to ensure proper traffic handling. Set it to false only in specific scenarios where appropriate.

On the Pods page, find your target application pod and click Actions in the Details column. Click the Events tab. You will see the application container starts first, followed by the Istio Proxy. In the Pod’s Events tab, the details-v1 pod event list shows the full container startup sequence, including Pod scheduling (Scheduled), image pulling (Pulled), container creation (Created), and container startup (Started). The started containers appear in this order: init container istio-init, application container details, and Sidecar container istio-proxy. The Started container events are highlighted, confirming the startup order enforced by the holdApplicationUntilProxyStarts configuration.

Appendix A: Istio resource annotations

Annotation	Description
proxy.istio.io/config	Override the proxy configuration for this proxy.
readiness.status.sidecar.istio.io/applicationPorts	Specify the list of ports exposed by the application container. Used by the Sidecar proxy readiness probe to determine when the proxy is configured and ready to receive traffic.
readiness.status.sidecar.istio.io/failureThreshold	Specify the failure threshold for the Sidecar proxy readiness probe.
readiness.status.sidecar.istio.io/initialDelaySeconds	Specify the initial delay (in seconds) for the Sidecar proxy readiness probe.
readiness.status.sidecar.istio.io/periodSeconds	Specify the period (in seconds) for the Sidecar proxy readiness probe.
sidecar.istio.io/componentLogLevel	Specify the component log level for the Sidecar proxy.
sidecar.istio.io/enableCoreDump	Specify whether the Sidecar proxy should enable core dump.
sidecar.istio.io/extraStatTags	List of additional tags extracted from Istio Proxy telemetry. Each additional tag must appear in this list.
sidecar.istio.io/inject	Specify whether to use Sidecar automatic injection.
sidecar.istio.io/interceptionMode	Specify the mode (REDIRECT or TPROXY) used to redirect inbound traffic to Envoy.
sidecar.istio.io/logLevel	Set the log level for the Sidecar proxy.
sidecar.istio.io/proxyCPU	Specify the requested CPU setting for the Sidecar proxy.
sidecar.istio.io/proxyCPULimit	Specify the CPU limit for the Sidecar proxy.
sidecar.istio.io/proxyMemory	Specify the requested memory setting for the Sidecar proxy.
sidecar.istio.io/proxyMemoryLimit	Specify the memory limit for the Sidecar proxy.
sidecar.istio.io/rewriteAppHTTPProbers	Redirect HTTP readiness and liveness probes to the Sidecar proxy.
status.sidecar.istio.io/port	Specify the HTTP status port for the Sidecar proxy. If set to zero, the Sidecar does not provide status.
traffic.sidecar.istio.io/excludeInboundPorts	Comma-separated list of inbound ports excluded from redirection to the Sidecar proxy. Applies only when all inbound traffic (that is, “*”) is redirected.
traffic.sidecar.istio.io/excludeOutboundIPRanges	Comma-separated list of IP ranges in CIDR format excluded from redirection. Applies only when all outbound traffic (that is, “*”) is redirected.
traffic.sidecar.istio.io/excludeOutboundPorts	Comma-separated list of string ports that prohibit outbound traffic redirection to the Sidecar proxy.
traffic.sidecar.istio.io/includeInboundPorts	Comma-separated list of inbound ports whose traffic is redirected to the Sidecar proxy. Use wildcard “*” to configure redirection for all ports. An empty list disables redirection for all inbound traffic.
traffic.sidecar.istio.io/includeOutboundIPRanges	Comma-separated list of IP ranges in CIDR format redirected to the Sidecar proxy. Use wildcard “*” to redirect all outbound traffic. An empty list disables redirection for all outbound traffic.
traffic.sidecar.istio.io/includeOutboundPorts	Comma-separated list of outbound traffic port strings whose traffic is redirected to the Sidecar proxy regardless of the destination IP.
traffic.sidecar.istio.io/kubevirtInterfaces	Comma-separated list of virtual interfaces whose inbound traffic is treated as outbound traffic.

Appendix B: ProxyConfig fields

Field	Type	Description
configPath	String	Path to the directory where generated configuration files are stored. The proxy agent generates the actual configuration and stores it in this directory.
statsdUdpAddress	String	IP address and port of the statsd UDP listener.
proxyAdminPort	Int32	Port on which Envoy listens for administrative commands. Default port is 15000.
controlPlaneAuthPolicy	AuthenticationPolicy	AuthenticationPolicy defines how the proxy authenticates when connecting to the control plane. Default is `MUTUAL_TLS`.
concurrency	Int32Value	Number of worker threads to run. If not set, this is automatically determined based on CPU requests and limits. If set to 0, all cores on the machine are used. Default is 2 worker threads.
interceptionMode	InboundInterceptionMode	Set the mode for redirecting inbound traffic to the Sidecar proxy.
tracing	Tracing	Tracing configuration used by the Sidecar proxy.
sds	SDS	Secret Discovery Service (SDS) configuration used by the proxy.
proxyMetadata	Map<string, string>	Additional environment variables for the proxy. Names starting with ISTIO_META_ are included in the generated bootstrap configuration and sent to the XDS server.
statusPort	Map<string, string>	Administrative (for example, readiness probe) port on which the proxy listens. Default is port 15020.
terminationDrainDuration	Duration	Time allowed for connections to complete during proxy shutdown. Upon receiving SIGTERM or SIGINT, istio-agent instructs the active Sidecar proxy to start draining—blocking new connections and allowing existing ones to complete. It then sleeps for termination_drain_duration before shutting down all remaining active Envoy processes. If not set, the default value of 5s is applied.
holdApplicationUntilProxyStarts	BoolValue	Boolean flag to enable or disable the holdApplicationUntilProxyStarts behavior. This feature adds hooks to delay application startup until the pod proxy is ready to accept traffic, mitigating certain startup race conditions.