Route ACS Pod DNS to Custom Servers via Annotations - Container Compute Service

When pods need to resolve hostnames that Alibaba Cloud DNS does not serve—such as private image registry domains or internal application domains—you can add the network.alibabacloud.com/custom-dnsconfig annotation to route their DNS queries to a self-managed DNS server. The annotation merges your custom nameservers, search domains, and options into the pod's /etc/resolv.conf at startup, without replacing the cluster's default DNS behavior.

When to use this feature

Use this approach in the following scenarios:

Scenario	DNS policy	Custom DNS usage
Scenario 1	`Default`	Use `acs-profile` to apply custom DNS settings cluster-wide
Scenario 2	`ClusterFirst`	Use a self-managed DNS server to resolve image registry domain names
Scenario 3	`Default`	Use `acs-profile` for cluster-wide DNS and a self-managed DNS server for image registry domains

Do not use this approach in the following scenarios:

All domains resolved by Alibaba Cloud DNS: Use the Default DNS policy. Do not configure custom DNS servers.
Application domains on self-managed DNS, image registry domains on Alibaba Cloud DNS: Use the None DNS policy and set the dnsConfig field in the pod spec. See Customize DNS settings for a pod.

How it works

The annotation value follows the standard Kubernetes spec.dnsConfig structure. When a pod starts, ACS reads the annotation and merges the custom DNS settings into the pod's /etc/resolv.conf:

Nameservers: Custom nameservers are prepended to the list. The system also appends the Alibaba Cloud DNS server IP (100.100.2.136) so pods can still resolve public domains.
Search domains and options: Merged with the settings from the active DNS policy.

In most cases, use acs-profile to apply DNS settings cluster-wide rather than adding annotations to individual pods. See Cluster-wide DNS configuration.

Configure a custom DNS server for a pod

Prerequisites

Before you begin, ensure that you have:

An ACS cluster
kubectl configured to connect to the cluster
A self-managed DNS server and its IP address

Add the annotation to a pod

Add the network.alibabacloud.com/custom-dnsconfig annotation to the metadata section of a pod configuration. The annotation value is a JSON string:

{
    "servers": [
        "20.1.xx.xx",
        "30.1.xx.xx"
    ],
    "searches": [
        "xx.com",
        "yy.com"
    ],
    "options": [
        "ndots:2",
        "edns0"
    ]
}

Field	Type	Limit	Description
`servers`	`[]string`	Max 2 IP addresses; extras are ignored	IP addresses of custom DNS servers. The system prepends these to the nameserver list and appends the Alibaba Cloud DNS server IP automatically, so pods can still resolve public domains.
`searches`	`[]string`	Max 32 search domains	Search domains appended to incomplete hostnames before resolution. For example, with `searches: ["xx.com"]`, a query for `api` resolves as `api.xx.com`. These are merged with the search domains from the active DNS policy.
`options`	`[]string`	—	DNS resolution options as key-value pairs. Merged with options from the active DNS policy. Common values: `ndots:<n>` (minimum dots to treat a name as absolute), `edns0` (enables EDNS0 for larger UDP packets), `timeout:<n>` (query timeout in seconds), `attempts:<n>` (number of retry attempts).

Important

Set dnsPolicy to match your scenario. Scenario 2 requires dnsPolicy: ClusterFirst.

Deploy a test workload

Create a file named deploy.yaml with the following content:

apiVersion: apps/v1
kind: Deployment
metadata:
  labels:
    app: test
  name: test-default
spec:
  replicas: 1
  selector:
    matchLabels:
      app: test
  template:
    metadata:
      annotations:
        network.alibabacloud.com/custom-dnsconfig: '{"servers":["20.1.XXX.XXX","30.1.X.X"],"searches":["xx.com","yy.com"],"options":["ndots:2","edns0"]}'
      labels:
        app: test
    spec:
      containers:
      - image: registry.cn-hangzhou.aliyuncs.com/acs/busybox:v1.29.2
        command: ["sh","-c","i=1; while true; do echo $i; i=$((i=i+1)); sleep 1; done;"]
        name: busybox
        resources:
          requests:
            cpu: "2"
            memory: "8Gi"
          limits:
            cpu: "2"
            memory: "8Gi"
      dnsPolicy: Default

Apply the configuration:
```
kubectl apply -f deploy.yaml
```

Verify the DNS configuration

Check the pod's /etc/resolv.conf to confirm the settings took effect:

kubectl exec <pod-name> -- cat /etc/resolv.conf

The expected output is:

search xx.com yy.com
nameserver 20.1.XXX.XXX
nameserver 30.1.XXX.XXX
nameserver 100.100.2.136
options ndots:6 edns

100.100.2.136 is the Alibaba Cloud DNS server automatically appended by the system.

Important

If the pod enters Pending state and reports an internal error after you apply the configuration, verify that the annotation JSON is valid and that all field values are within their limits.

Cluster-wide DNS configuration

To apply custom DNS settings to all pods in a namespace without adding individual annotations, configure pod auto-injection through acs-profile.

The following acs-profile ConfigMap automatically injects the custom DNS annotation into all pods in the default namespace. After applying it, pods only need dnsPolicy: Default—no annotation is required on each pod.

apiVersion: v1
kind: ConfigMap
metadata:
  name: acs-profile
  namespace: kube-system
data:
  selectors: |
    [
      {
        "name": "selector-demo1",
        "namespaceSelector": {
          "matchLabels": {
            "kubernetes.io/metadata.name": "default"
          }
        },
        "effect": {
          "annotations": {
            "network.alibabacloud.com/custom-dnsconfig": "{\"servers\":[\"20.1.XXX.XXX\",\"30.1.X.X\"],\"searches\":[\"xx.com\",\"yy.com\"],\"options\":[\"ndots:2\",\"edns0\"]}"
          }
        }
      }
    ]

For full details on acs-profile selectors and injection rules, see Configure an acs-profile to automatically inject pod configurations.

Container Compute Service:Add annotations to configure a custom DNS server