In a cluster of Container Service for Kubernetes (ACK), Service names are preferably used to access Services. Therefore, a cluster-wide DNS service is required to resolve Service names to cluster IPs (IP addresses assigned to Services). CoreDNS is deployed in ACK clusters and serves as a DNS server to provide DNS resolution services for workloads deployed in ACK clusters. This topic describes how DNS resolution works in ACK clusters and how to configure CoreDNS and DNS policies in different scenarios.
Prerequisites
Background information
kubectl get svc kube-dns -n kube-system
The following output is returned:NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE
kube-dns ClusterIP 172.24.0.10 <none> 53/UDP,53/TCP,9153/TCP 27d
kubectl get deployment coredns -n kube-system
The following output is returned:NAME READY UP-TO-DATE AVAILABLE AGE
coredns 2/2 2 2 27d
How DNS resolution works in ACK clusters
The startup parameters of kubelet in an ACK cluster contain --cluster-dns=<dns-service-ip>
and --cluster-domain=<default-local-domain>
. These parameters are used to configure the IP addresses and suffixes of the base
domain name for the DNS servers in the ACK cluster.
nameserver xx.xx.0.10
search kube-system.svc.cluster.local svc.cluster.local cluster.local
options ndots:5
Parameter | Description |
---|---|
nameserver | Specifies the IP addresses of the DNS servers. |
search | Specifies the suffixes that are used for DNS queries. More suffixes indicate more
DNS queries. In ACK clusters, suffixes are kube-system.svc.cluster.local , svc.cluster.local , and cluster.local . Therefore, up to eight queries (four for an IPv4 address and four for an IPv6 address)
are generated for a request sent to an ACK cluster.
|
options | Specifies the options for the DNS configuration file. You can specify multiple key-value pairs. For example, ndots:5 specifies that if the number of dots in the domain name string is greater than 5, the domain name is a fully qualified domain name and is directly resolved. If the number of dots in the domain name string is less than 5, the domain name is appended with the suffixes specified by the search parameter before it is resolved. |
Based on the preceding DNS resolution settings on the pods, queries of internal domain names (Services) and external domain names are sent to the DNS servers of an ACK cluster for DNS resolution.
Configure DNS policies for an ACK cluster in different scenarios
- ClusterFirst: This policy indicates that a pod uses CoreDNS to resolve domain names from inside or outside of the ACK cluster. The /etc/resolv.conf file contains the address of the DNS server that is provided by CoreDNS, which is kube-dns. This is the default DNS policy for workloads in an ACK cluster.
- None: This policy indicates that a pod ignores the DNS settings of the ACK cluster. You must customize the DNS settings by using the dnsConfig field.
- Default: This policy indicates that a pod inherits the DNS resolution settings from the node where the pod is deployed. In an ACK cluster, nodes are created based on Elastic Compute Service (ECS) instances. Therefore, a pod directly uses the /etc/resolv.conf file of the ECS instance-based node where the pod is deployed. This file contains the address of a DNS server that is provided by Alibaba Cloud DNS.
- ClusterFirstWithHostNetwork: This policy indicates that a pod in the host network uses the ClusterFirst policy. By default, a pod in the host network uses the Default policy.
-
Scenario 1: Use CoreDNS provided by ACK clusters to resolve domain names
In this scenario, you must specifydnsPolicy: ClusterFirst
for the DNS policy settings. The following code block is an example:apiVersion: v1 kind: Pod metadata: name: alpine namespace: default spec: containers: - image: alpine command: - sleep - "10000" imagePullPolicy: Always name: alpine dnsPolicy: ClusterFirst
-
Scenario 2: Customize DNS settings for a pod
To customize DNS settings for a Deployment, you must specifydnsPolicy: None
for the DNS policy settings. The following code block is an example:apiVersion: v1 kind: Pod metadata: name: alpine namespace: default spec: containers: - image: alpine command: - sleep - "10000" imagePullPolicy: Always name: alpine dnsPolicy: None dnsConfig: nameservers: ["169.254.xx.xx"] searches: - default.svc.cluster.local - svc.cluster.local - cluster.local options: - name: ndots value: "2"
Descriptions of parameters in the dnsConfig section.Parameter Description nameservers A list of IP addresses of DNS servers for a pod. You can specify up to three IP addresses. If you specify dnsPolicy as None
for a pod, you must specify at least one IP address. If you do not specify dnsPolicy as None for a pod, this parameter is optional. The listed DNS server IP addresses will be added to the nameserver parameter of the DNS configuration file that is generated based on the specified DNS policy. Duplicate IP addresses are removed.searches A list of DNS search domains for hostname lookup in a pod. This parameter is optional. The listed DNS search domains will be added to the list of base search domains that are generated based on the specified DNS policy. Duplicate domain names are removed. You can specify up to six search domains. options A list of optional items. Each item can contain a name (required) and a value (optional). The specified items will be added to the list of optional items that are generated based on the specified DNS policy. Duplicate items are removed. For more information, see DNS for Services and Pods.
- Scenario 3: Use the DNS settings of an ECS instance that is provided by Alibaba CloudIf your application pods do not need to access other services deployed in the ACK cluster, you can specify
dnsPolicy: Default
for the DNS policy settings. In this scenario, the DNS resolution is performed by Alibaba Cloud DNS and CoreDNS is not required. The following code block is an example:apiVersion: v1 kind: Pod metadata: name: alpine namespace: default spec: containers: - image: alpine command: - sleep - "10000" imagePullPolicy: Always name: alpine dnsPolicy: Default
- Scenario 4: Enable pods in the host network to access services of an ACK clusterIf you specify hostNetwork:true for the network settings of your application pods, your application pods can directly use the host network. In this case, the default DNS policy for a pod is Default. As a result, your application pods cannot access services deployed in the ACK cluster. If you want to enable pods that run in the host network to access services deployed in the ACK cluster, you must specify
dnsPolicy: ClusterFirstWithHostNet
for the DNS policy settings. The following code block is an example:apiVersion: v1 kind: Pod metadata: name: alpine namespace: default spec: hostNetwork: true dnsPolicy: ClusterFirstWithHostNet containers: - image: alpine command: - sleep - "10000" imagePullPolicy: Always name: alpine
Configure CoreDNS for an ACK cluster
Default configurations of CoreDNS
Corefile: |
.:53 {
errors
health {
lameduck 5s
}
ready
kubernetes cluster.local in-addr.arpa ip6.arpa {
pods insecure
upstream
fallthrough in-addr.arpa ip6.arpa
ttl 30
}
prometheus :9153
forward . /etc/resolv.conf
cache 30
loop
reload
loadbalance
}
Parameter | Description |
---|---|
errors | Errors are logged to stdout. |
health | The health check report of CoreDNS. The default listening port is 8080. This parameter
is used to evaluate the health status of CoreDNS. You can visit http://localhost:8080/health to view the health check report of CoreDNS.
|
ready | The health check report of the CoreDNS plug-ins. The default listening port number
is 8181. This parameter is used to evaluate the readiness of the CoreDNS plug-ins.
You can visit http://localhost:8181/ready to view the readiness of the CoreDNS plug-ins. After all plug-ins are in the running
state, 200 is returned for the readiness of CoreDNS plug-ins.
|
kubernetes | The CoreDNS kubernetes plug-in that is used to provide DNS resolution for Services inside an ACK cluster. |
prometheus | Metrics of CoreDNS. You can visit http://localhost:9153/metrics to view metrics of CoreDNS in Prometheus format.
|
forward or proxy | DNS queries are forwarded to the predefined DNS server. By default, DNS queries of domain names that are not within the cluster domain of Kubernetes are forwarded to the predefined resolver (/etc/resolv.conf). The default configurations are based on the /etc/resolv.conf file on the host. |
cache | DNS cache. |
loop | Loop detection. If a loop is detected, CoreDNS is suspended. |
reload | Allows automatic reload of a changed Corefile. After you edit the ConfigMap configuration, wait two minutes for the changes to take effect. |
loadbalance | A round-robin DNS loadbalancer that randomizes the order of A, AAAA, and MX records in the answer. |
Configure extended features based on CoreDNS
- Scenario 1: Enable Log ServiceTo log each DNS resolution performed by CoreDNS, enable the log plug-in by adding log to Corefile. The following code block is an example:
Corefile: | .:53 { errors log health { lameduck 5s } ready kubernetes cluster.local in-addr.arpa ip6.arpa { pods insecure upstream fallthrough in-addr.arpa ip6.arpa ttl 30 } prometheus :9153 forward . /etc/resolv.conf cache 30 loop reload loadbalance }
- Scenario 2: Customize DNS servers for specified domain namesIf domain names with a suffix of example.com need to be resolved by a user-defined DNS server (for example, DNS server 10.10.0.10), you must add a custom resolution setting for the domain names. The following code block is an example:
The following code block shows all configurations:example.com:53 { errors cache 30 forward . 10.10.0.10 }
Corefile: | .:53 { errors health { lameduck 5s } ready kubernetes cluster.local in-addr.arpa ip6.arpa { pods insecure upstream fallthrough in-addr.arpa ip6.arpa ttl 30 } prometheus :9153 forward . /etc/resolv.conf cache 30 loop reload loadbalance } example.com:53 { errors cache 30 forward . 10.10.0.10 }
- Scenario 3: Customize DNS servers for external domain namesIf the domain names that need to be resolved by user-defined DNS servers do not contain the same suffix, you can use user-defined DNS servers to perform DNS resolution for all external domain names. In this scenario, you must forward the domain names that cannot be resolved by the on-premises DNS servers to Alibaba Cloud DNS. Do not modify the /etc/resolv.conf files on ECS instances of the ACK cluster. Assume that the IP addresses of the user-defined DNS servers are 10.10.0.10 and 10.10.0.20, you can modify the forward parameter. The following code block is an example:
Corefile: | .:53 { errors health { lameduck 5s } ready kubernetes cluster.local in-addr.arpa ip6.arpa { pods insecure upstream fallthrough in-addr.arpa ip6.arpa ttl 30 } prometheus :9153 forward . 10.10.0.10 10.10.0.20 cache 30 loop reload loadbalance }
- Scenario 4: Customize hosts for specified domain namesYou can configure the hosts plug-in if you want to customize hosts for specified domain names, for example, you may need to point www.example.com to 127.0.0.1. The following code block is an example:
Corefile: | .:53 { errors health { lameduck 5s } ready hosts { 127.0.0.1 www.example.com fallthrough } kubernetes cluster.local in-addr.arpa ip6.arpa { pods insecure upstream fallthrough in-addr.arpa ip6.arpa ttl 30 } prometheus :9153 forward . /etc/resolv.conf cache 30 loop reload loadbalance }
Notice You must specify fallthrough. Otherwise, domain names other than the specified one may fail to be resolved. - Scenario 5: Enable external access to services in an ACK cluster
If you want to enable a process on an ECS instance to access services in the ACK cluster where the ECS instance is deployed, you can add the ClusterIP of kube-dns in the ACK cluster to the nameserver parameter of the /etc/resolv.conf file on the ECS instance. Do not modify other settings in the /etc/resolv.conf file.
In an internal network, you can use an internal Server Load Balancer (SLB) instance to allow internal access to services in the ACK cluster. Then, log on to the Alibaba Cloud DNS console and add an A record that points to the private IP address of the SLB instance.
- Scenario 6: Use a domain name to allow access to your service in an ACK cluster or
enable CNAME resolution for an ACK clusterYou can use foo.example.com to allow all access to your service in an ACK cluster from the Internet, internal networks, and inside an ACK cluster. The following section describes how to enable this feature:
- Your service foo.default.svc.cluster.local is exposed to external access through an public-facing SLB instance. The domain name foo.example.com is resolved to the IP address of the public-facing SLB instance.
- Your service foo.default.svc.cluster.local is exposed to internal access through an public-facing SLB instance. Log on to the Alibaba Cloud DNS console to point foo.example.com to the IP address of the internal SLB instance in the virtual private cloud (VPC) where the ACK cluster is deployed. For more information about the procedure, see the preceding section Scenario 4: Customize hosts for specified domain names.
- Inside the ACK cluster, you can use the Rewrite plug-in to add a Canonical Name (CNAME)
record to point foo.example.com to foo.default.svc.cluster.local. The following code
block is an example:
Corefile: | .:53 { errors health { lameduck 5s } ready rewrite stop { name regex foo.example.com foo.default.svc.cluster.local answer name foo.default.svc.cluster.local foo.example.com } kubernetes cluster.local in-addr.arpa ip6.arpa { pods insecure upstream fallthrough in-addr.arpa ip6.arpa ttl 30 } prometheus :9153 forward . /etc/resolv.conf cache 30 loop reload loadbalance }
- Scenario 7: Monitor failed DNS resolutions of CoreDNSYou can query the CoreDNS log to troubleshoot failed DNS resolutions of CoreDNS. For more information, see the preceding section Scenario 1: Enable Log Service. In this case, you must enable autopath for CoreDNS. For more information about autopath, see Use the autopath add-on. The following code block is an example:
Corefile: | .:53 { errors health ready log kubernetes cluster.local in-addr.arpa ip6.arpa { pods verified fallthrough in-addr.arpa ip6.arpa } autopath @kubernetes prometheus :9153 forward . /etc/resolv.conf cache 30 loop reload loadbalance }
After the template is implemented, run thekubectl get pods -n kube-system | grep coredns
command to query information about the CoreDNS pods. Then, run thekubectl logs coredns-{pod id} -n kube-system
command to query the log of each CoreDNS pod. If response codes of the NXDOMAIN or SERVFAIL type appear in the log of CoreDNS, it indicates failed DNS resolutions of CoreDNS.