Container Service for Kubernetes:Manually update unmanaged CoreDNS

Update notes

• If the load balancing mode of kube-proxy is set to IP Virtual Server (IPVS), all DNS queries within the cluster may fail or time out after CoreDNS is updated. This situation lasts about 5 minutes. You can use one of the following methods to avoid this issue:

  • Modify the timeout period for IPVS UDP sessions in the kube-proxy configuration. For more information, see Configure the UDP timeout for an IPVS cluster.

  • Use NodeLocal DNSCache to optimize DNS resolution. For more information, see Configure NodeLocal DNSCache.

• The update takes approximately 2 minutes, though the actual duration may vary depending on the number of CoreDNS replicas in your cluster. Existing replicas are not terminated during the process, so there will be no impact on service resolution.

Query the CoreDNS version

kubectl get deployment coredns -n kube-system -o jsonpath="{.spec.template.spec.containers[0].image}"

registry-vpc.cn-hangzhou.aliyuncs.com/acs/coredns:1.6.2 # Version 1.6.2 is used in this example.

Confirm the update version

Confirm the CoreDNS version to which you want to update. The following table describes the Kubernetes versions that are compatible with CoreDNS. We recommend that you use the latest CoreDNS version that is compatible with the Kubernetes version.

Manually update

Configure the UDP timeout for an IPVS cluster

For Kubernetes 1.18 or later clusters

Using the console

1. Log on to the ACK console. In the left navigation pane, click Clusters.

2. On the Clusters page, click the name of the one you want to change. In the left-side navigation pane, choose Configurations > ConfigMaps.

3. On the ConfigMaps page, select the kube-system namespace. Find the kube-proxy-worker ConfigMap and click Edit YAML in the Actions column.

4. In the Edit YAML panel, add udpTimeout: 10s under the ipvs field and click OK.

   apiVersion: v1
   data:
     config.conf: |
       apiVersion: kubeproxy.config.k8s.io/v1alpha1
       kind: KubeProxyConfiguration
       # Other irrelevant fields are omitted.
       mode: ipvs
       # If the ipvs key does not exist, add it.
       ipvs:
         udpTimeout: 10s

5. Recreate all pods named kube-proxy-worker.

   1. On the cluster details page, in the navigation pane on the left, select Workloads > DaemonSets.

   2. In the DaemonSet list, find and click kube-proxy-worker.

   3. On the kube-proxy-worker page, click the Pods tab. In the row of a pod, choose More > Delete, and then click OK.

      Repeat this step to delete all pods. After the pods are deleted, the system automatically recreates them.

6. Verify that the UDP timeout is configured.

   1. Run the following command to install ipvsadm.

      ipvsadm is a management tool for the IPVS module. For more information, see ipvsadm.

      sudo yum install -y ipvsadm

   2. Run the following command on any ECS node in the cluster and check the third number in the output.

      sudo ipvsadm -L --timeout

      If the third number in the output is 10, the UDP timeout for the IPVS cluster is successfully changed.

      After the change is successful, wait at least five minutes before you proceed with the next step.

Using the command line

1. Run the following command to edit the kube-proxy-worker configuration file.

   kubectl -n kube-system edit configmap kube-proxy-worker

2. In the kube-proxy configuration file, add udpTimeout: 10s under the ipvs field. Then, save the file and exit.

   apiVersion: v1
   data:
     config.conf: |
       apiVersion: kubeproxy.config.k8s.io/v1alpha1
       kind: KubeProxyConfiguration
       # Other irrelevant fields are omitted.
       mode: ipvs
       # If the ipvs key does not exist, add it.
       ipvs:
         udpTimeout: 10s

3. Run the following commands to recreate all pods named kube-proxy-worker.

   1. Run the following command to view information about the existing pods.

      kubectl -n kube-system get pod -o wide | grep kube-proxy-worker

   2. Run the following command to delete the pods that you found in the previous step. The system automatically recreates the pods named kube-proxy-worker.

      kubectl -n kube-system delete pod <kube-proxy-worker-****>

      Replace <kube-proxy-worker-****> with the names of the pods that you found in the previous step.

4. Verify that the UDP timeout is configured.

   1. Run the following command to install ipvsadm.

      ipvsadm is a management tool for the IPVS module. For more information, see ipvsadm.

      sudo yum install -y ipvsadm

   2. Run the following command on any ECS node in the cluster and check the third number in the output.

      sudo ipvsadm -L --timeout

      If the third number in the output is 10, the UDP timeout for the IPVS cluster is successfully changed.

      After the change is successful, wait at least five minutes before you proceed with the next step.

For Kubernetes 1.16 or earlier clusters

sudo yum install -y ipvsadm
sudo ipvsadm -L --timeout > /tmp/ipvsadm_timeout_old
sudo ipvsadm --set 900 120 10
sudo ipvsadm -L --timeout > /tmp/ipvsadm_timeout_new
diff /tmp/ipvsadm_timeout_old /tmp/ipvsadm_timeout_new

For more information about batch operations in OOS, see Batch operation instances.