Container Service for Kubernetes:Use ACK CoreDNS DNSTAP Analyser to diagnose DNS resolution errors

Prerequisites

Before you begin, ensure that you have:

• An ACK Pro cluster. For more information, see Create an ACK Pro cluster.

• The kubeconfig file of the cluster and kubectl connectivity. For more information, see Obtain the kubeconfig file of a cluster and use kubectl to connect to the cluster.

Background

dnstap is a flexible log format for DNS messages. CoreDNS ships with a built-in dnstap plugin that captures every DNS message passing through CoreDNS. ACK CoreDNS DNSTAP Analyser receives these messages over TCP and aggregates them into per-session diagnostic records.

For more information about the dnstap log format, see dnstap.

Step 1: Install ACK CoreDNS DNSTAP Analyser

1. Log on to the ACK console. In the left-side navigation pane, click Clusters.

2. On the Clusters page, find the cluster and click its name. In the left-side navigation pane, choose Applications > Helm.

3. On the Helm page, click Deploy. In the Basic Information step, configure the following parameters. Select ack-coredns-dnstap-analyser and click Next.

   Parameter	Value
   Application Name	ack-coredns-dnstap-analyser
   Namespace	kube-system
   Source	Marketplace (default)
   Chart — Use Scenarios	All
   Chart — Supported Architecture	amd64
   Chart — Search box	ack-coredns-dnstap-analyser

4. In the Parameters step, configure the Chart Version and click OK.

Step 2: Configure the dnstap plugin in CoreDNS

CoreDNS comes with the dnstap plugin pre-installed. Enable it and point it at ACK CoreDNS DNSTAP Analyser so that DNS messages are forwarded automatically.

1. Get the cluster IP address of the DNSTAP Analyser service. Record this address — you need it in the next step.

   kubectl -n kube-system get svc dnstap-analyser

   In the following example, the cluster IP address is 172.21.0.10.

2. Open the CoreDNS ConfigMap for editing.

   kubectl -n kube-system edit cm coredns -o yaml

3. Add the dnstap directive to the Corefile. Replace 172.21.0.10 with the actual cluster IP address from step 1.

   Corefile: |
       .:53 {
           # ... other plugins ...
           ready
   
           # Add this line. The "full" flag includes the complete wire-format DNS
           # message in each record, which provides richer diagnostic data.
           dnstap tcp://172.21.0.10:6000 full
   
           kubernetes cluster.local in-addr.arpa ip6.arpa {
           # ... other settings ...
           }
           # ... other plugins ...
           reload
           loadbalance
       }

4. Verify that CoreDNS Pods reloaded the updated configuration.

   1. List all CoreDNS Pods.

      kubectl -n kube-system get pod | grep coredns

      Expected output:

      coredns-7d56l         1/1     Running   0          30m
      coredns-s7m2t         1/1     Running   0          30m

   2. Tail the log of one CoreDNS Pod. Replace coredns-7d56l with an actual Pod name from the previous command.

      kubectl -n kube-system logs -f --tail=500 coredns-7d56l

      Wait up to one minute. A successful reload produces the following line:

      [INFO] Reloading complete

Step 3: View the log of ACK CoreDNS DNSTAP Analyser

1. List all ACK CoreDNS DNSTAP Analyser Pods.

   kubectl -n kube-system get pod | grep dnstap-analyser

   Expected output:

   dnstap-analyser-bbdf879-g****         1/1     Running   0          30m

2. Stream the log. Replace dnstap-analyser-bbdf879-g**** with the actual Pod name from the previous command.

   kubectl -n kube-system logs -f dnstap-analyser-bbdf879-g****

Step 4: Analyze the log

Each log line represents one DNS session — the complete set of requests and responses exchanged among a client, CoreDNS, and the upstream DNS server.

How to read a session entry: First check the Status field to confirm whether the session succeeded or failed. Then check BitMap to identify which stage in the resolution chain failed. Finally, inspect Messages for the raw DNS payloads.

(Optional) Step 5: Uninstall ACK CoreDNS DNSTAP Analyser

After completing your investigation, uninstall ACK CoreDNS DNSTAP Analyser to avoid incurring unnecessary costs.

1. Remove the dnstap directive from the CoreDNS ConfigMap.

   1. Open the ConfigMap for editing.

      kubectl -n kube-system edit cm coredns -o yaml

   2. Delete the line containing dnstap.

      Corefile: |
          .:53 {
              # ... other plugins ...
              ready
      
              # Delete the following line.
              dnstap tcp://10.10.10.10:6000 full
      
              kubernetes cluster.local in-addr.arpa ip6.arpa {
              # ... other settings ...
              }
              # ... other plugins ...
              reload
              loadbalance
          }

   3. Verify that CoreDNS Pods reloaded the updated configuration. List all CoreDNS Pods.

      kubectl -n kube-system get pod | grep coredns

      Expected output:

      coredns-7d56l         1/1     Running   0          30m
      coredns-s7m2t         1/1     Running   0          30m

   4. Tail the log of one CoreDNS Pod. Replace coredns-7d56l with an actual Pod name from the previous command.

      kubectl -n kube-system logs -f --tail=500 coredns-7d56l

      Wait up to one minute. A successful reload produces:

      [INFO] Reloading complete

2. Delete the ACK CoreDNS DNSTAP Analyser Helm release.

   1. Log on to the ACK console. In the left-side navigation pane, click Clusters.

   2. On the Clusters page, find the cluster and click its name. In the left-side navigation pane, choose Applications > Helm.

   3. On the Helm page, find ack-coredns-dnstap-analyser and click Delete in the Actions column. Follow the on-screen instructions to complete the deletion.

What's next

• DNS overview

• DNS policies and domain name resolution

• DNS troubleshooting