We recommend that you upgrade the CoreDNS version to 1.6.2 or later to improve the stability of domain name resolution in your Container Service for Kubernetes (ACK) cluster. This topic describes how to manually upgrade CoreDNS.

Background information

CoreDNS 1.4 and earlier have the following two issues that affect the stability of domain name resolution in clusters of ACK versions earlier than 1.16:
  • CoreDNS must send watch requests to the API server to query service data. Therefore, the health check plug-in checks the connectivity between CoreDNS and the API server. If connectivity exceptions occur, such as network jitters or the restart of the API server, the health check plug-in reports an error. CoreDNS restarts if the connectivity fails to pass the health check for three times. Services are unavailable during the CoreDNS restart process. For more information, see coredns.
  • CoreDNS is affected by the bug of klog. After CoreDNS is disconnected from the API server, klog attempts to write an error log to a directory named /tmp. However, this directory does not exist. As a result, the logging operation fails and leads to a CoreDNS crash. For more information, see add mount of /tmp #137.
We recommend that you upgrade CoreDNS to improve the stability of domain name resolution in your ACK cluster. You can use one of the following solutions to upgrade CoreDNS:
  • Long-term solution: Upgrade the ACK cluster to V1.16 or later. During the cluster upgrade, CoreDNS is automatically upgraded to 1.6.2 or later. For information about how to upgrade an ACK cluster in the console, see Upgrade a cluster. For information about how to upgrade an ACK cluster by calling an API operation, see Upgrade a cluster.
  • Short-term solution: If you do not want to upgrade the cluster, you can manually upgrade the CoreDNS version to 1.6.2 or later. For information about how to manually upgrade the CoreDNS version, see Upgrade procedure in this topic.
    Notice You need to manually upgrade CoreDNS only if the CoreDNS version is 1.4 or earlier.

Upgrade procedure

  1. Check whether the cluster version is compatible with CoreDNS 1.6.2.
    Check the version of the ACK cluster. Make sure that the cluster version is compatible with CoreDNS 1.6.2. The following table lists the ACK cluster versions that are compatible with CoreDNS 1.6.2. Clusters of ACK V1.11, V1.12, V1.14, and V1.16 are compatible with CoreDNS 1.6.2.
    Product Compatible version
    ACK cluster 1.11 1.12 1.14 1.16
    CoreDNS 1.6.2 1.6.2 1.6.2 1.6.2

    Perform the following steps to check the version of the ACK cluster:

    1. Log on to the ACK console.
    2. In the left-side navigation pane, click Clusters.
    3. On the Clusters page, find the cluster of which you want to check the version, and check the cluster version in the Version column.version
  2. Check the version of CoreDNS.
    • Check the version of CoreDNS in the ACK console.
      1. Log on to the ACK console.

      2. In the left-side navigation pane, click Clusters. On the Clusters page, find the cluster of which you want to check the CoreDNS version, and click Applications in the Actions column.
      3. On the Deployments tab, select kube-system from the Namespace drop-down list and check the version of CoreDNS in the Image column.dns
    • You can also run the following kubectl command to query the version of CoreDNS.
      kubectl get deployment coredns -n kube-system -o jsonpath="{.spec.template.spec.containers[0].image}"
      The following code block shows a sample output:
      registry-vpc.cn-hangzhou.aliyuncs.com/acs/coredns:1.3.1
  3. Modify the CoreDNS configuration item.

    CoreDNS 1.6.2 uses forward to replace the proxy plug-in. You must use the forward field to replace the proxy field of the coredns configuration item in the kube-system namespace.

    • You can update the coredns configuration item in the ACK console.
      1. Log on to the ACK console.

      2. In the left-side navigation pane, click Clusters, and then click the name of the cluster for which you want to upgrade CoreDNS.
      3. In the left-side navigation pane, click Configurations.
      4. On the ConfigMap page, click YAML next to coredns.
      5. On the Edit page, change proxy to forward.forward
    • You can also run the kubectl command to modify the CoreDNS configuration item.
      # Open the window where you can edit the configurations of CoreDNS.
      kubectl edit configmap/coredns -n kube-system
      
      # Use forward to replace proxy.
      # Save the change and exit.
    After the configuration item is modified, check whether the configuration is reloaded to the stdout log of a CoreDNS pod. A hot reloading takes 30 seconds.
    1. Run the following command to view the running status of the CoreDNS pods in the cluster:
      kubectl get pods -n kube-system | grep coredns
      The following code block shows a sample output:
      coredns-78d4b8bd88-6g62w                           1/1     Running   0          9d
      coredns-78d4b8bd88-n6wjm                           1/1     Running   0          9d
    2. Run the following command to view the logs of a running CoreDNS pod:
      kubectl logs coredns-78d4b8bd88-n6wjm -n kube-system
      The following code block shows a sample output:
      .:53
      [INFO] plugin/reload: Running configuration MD5 = 71c5f1ff539d304c630521f315dc2ac2
      CoreDNS-1.6.7
      linux/amd64, go1.13.6, da7f65b
      [INFO] 127.0.0.1:48329 - 42313 "HINFO IN 1108347002237365533.4506541768939609094. udp 57 false 512" NXDOMAIN qr,rd,ra 132 0.008874794s

      If plugin/reload appears in the output, the CoreDNS configuration is reloaded.

  4. Upgrade the image version of CoreDNS.
    Upgrade the image version of CoreDNS to 1.6.2.
    • Upgrade the image version in the ACK console
      1. Log on to the ACK console.

      2. In the left-side navigation pane, click Clusters. On the Clusters page, find the cluster of which you want to check the CoreDNS version, and click Applications in the Actions column.
      3. On the Deployments tab, find coredns and choose More > View in YAML.
      4. On the Edit YAML page, change the version of image to 1.6.2.image
    • Upgrade the image version by running the kubectl command

      # Open the window where you can edit the image version.
      kubectl edit deployment/coredns -n kube-system
      
      # Change the image version to 1.6.2.
      # Save the change and exit.
  5. Verify the result.
    Run the following command to check whether all CoreDNS pods in the cluster are in the Running state:
    kubectl get pods -n kube-system | grep coredns
    The following code block shows a sample output:
    coredns-78d4b8bd88-6g62w                           1/1     Running   0          9d
    coredns-78d4b8bd88-n6wjm                           1/1     Running   0          9d