Exclude Sidecar Containers from Resource Scheduling in ACS - Container Compute Service

ACS clusters don't support DaemonSets because virtual nodes can't run them. A common workaround is deploying DaemonSet-like workloads as sidecar containers. However, adding a sidecar increases a pod's total resource requests, causing the cluster to reserve more capacity than the application actually needs. To avoid this, set the __IGNORE_RESOURCE__ environment variable on the sidecar container so the scheduler excludes its resource requests when calculating pod resource consumption.

How it works

When __IGNORE_RESOURCE__ is set to true on a container, ACS excludes that container's resource requests from pod-level scheduling calculations. Resources are allocated based only on the resources.requests and resources.limits values of the remaining containers.

The sidecar still runs inside the pod and reuses the resources allocated to the other containers—it doesn't hold its own reserved capacity.

Important

Because the sidecar shares resources rather than holding a reservation, it competes with application containers under pressure. If the pod's allocated resources run low, the sidecar's CPU usage is throttled and out-of-memory (OOM) errors may occur.

Environment variable reference

Parameter	Value	Description
`__IGNORE_RESOURCE__`	`"true"`	Excludes the container's `resources.requests` from pod-level scheduling. Set this on the sidecar container.

Verify the configuration

The following steps walk you through three scenarios:

Baseline pod resource consumption with the sidecar's requests counted
Pod resource consumption after enabling __IGNORE_RESOURCE__=true
CPU share ratio between the app and sidecar containers

Prerequisites

Before you begin, ensure that you have:

An ACS cluster with kubectl configured
Permissions to create and update Deployments in the target namespace

Step 1: Create the baseline Deployment

Create a file named test-ignore.yaml with the following content:

apiVersion: apps/v1
kind: Deployment
metadata:
  name: test-ignore
  labels:
    app: test
spec:
  replicas: 1
  selector:
    matchLabels:
      app: test
  template:
    metadata:
      labels:
        app: test
    spec:
      containers:
      - name: app
        image: registry-cn-hangzhou.ack.aliyuncs.com/ack-demo/busybox:1.28
        command: ["/bin/sh", "-c", "sleep 999"]
        resources:
          requests:
            cpu: 1
            memory: 2Gi
      - name: sidecar
        image: registry-cn-hangzhou.ack.aliyuncs.com/ack-demo/busybox:1.28
        command: ["/bin/sh", "-c", "sleep 999"]
        resources:
          requests:
            cpu: 0.25
            memory: 0.5Gi

This Deployment creates one pod with two containers. Both containers have resource requests defined—the sidecar requests 0.25 vCPU and 0.5 GiB.

Apply the Deployment:
```
kubectl apply -f test-ignore.yaml
```
Check the pod's scheduled resource spec:
```
kubectl get pod <pod-name> -o yaml | grep alibabacloud.com/pod-use-spec
```
Expected output:
```
alibabacloud.com/pod-use-spec: 2-4Gi
```
The annotation shows the pod is scheduled with 2 vCPUs and 4 GiB of memory—the sum of both containers' requests.

Step 2: Enable resource ignore on the sidecar

Update the sidecar container in test-ignore.yaml to add the __IGNORE_RESOURCE__ environment variable:

apiVersion: apps/v1
kind: Deployment
metadata:
  name: test-ignore
.....
      - name: sidecar
        image: registry-cn-hangzhou.ack.aliyuncs.com/ack-demo/busybox:1.28
        command: ["/bin/sh", "-c", "sleep 999"]
        env:
        - name: __IGNORE_RESOURCE__
          value: "true"
        resources:
          requests:
            cpu: 0.25
            memory: 0.5Gi

Apply the updated Deployment:
```
kubectl apply -f test-ignore.yaml
```
Check the pod's resource spec again:
```
kubectl get pod <pod-name> -o yaml | grep alibabacloud.com/pod-use-spec
```
The pod now occupies 1 vCPU and 2 GiB of memory—only the app container's requests are counted.

Step 3: Verify CPU share ratios

CPU shares reflect each container's relative weight for CPU time. Even though the sidecar's requests are excluded from scheduling, the cgroup weights are still derived from its declared resources.requests.

Check the app container's CPU weight:

kubectl exec -ti <pod-name> -c app -- cat /sys/fs/cgroup/cpu/cpu.shares

Expected output:

Check the sidecar container's CPU weight:
```
kubectl exec -ti <pod-name> -c sidecar -- cat /sys/fs/cgroup/cpu/cpu.shares
```
Expected output:
```
256
```
The app-to-sidecar CPU share ratio is 4:1, matching the ratio of their declared requests (1 vCPU vs. 0.25 vCPU).

Configuration comparison

The following table shows how different sidecar configurations affect both pod resource consumption and CPU share ratios.

Note

Setting resources.requests on the sidecar to 0 (without using __IGNORE_RESOURCE__) also reduces the pod's scheduled resource consumption, but causes the CPU share ratio to become approximately 512:1—much higher than the intended 4:1. This makes CPU sharing between the app and sidecar containers unpredictable.

Sidecar configuration	app vs. sidecar CPU shares	Pod resource
`__IGNORE_RESOURCE__=false`, CPU request = 0.25c	4:1	2c4g
`__IGNORE_RESOURCE__=false`, CPU request = 0c	512:1	1c2g
`__IGNORE_RESOURCE__=true`, CPU request = 0.25c	4:1	1c2g