DNS in Kubernetes (CoreDNS)

Pod IPs are ephemeral — they change on every restart, scale, or reschedule. Hard-coding them is impossible. Kubernetes runs an in-cluster DNS (CoreDNS) so workloads can reach each other by stable names like payments.prod.svc.cluster.local instead of chasing IPs.

Solution: Run CoreDNS in-cluster so pods reach Services by stable DNS names (payments.prod.svc.cluster.local) via the resolver in their /etc/resolv.conf.

A big port runs an internal directory desk where crew can ask for a ship by name and get back the pier number it is docked at, and the desk keeps its answers current by reading the port's live berth ledger. CoreDNS is that desk for the cluster: a pod asks for a Service name and gets back its ClusterIP, and CoreDNS stays accurate by watching Services and Endpoints on the api-server.

The same cluster entities appear in every day's notes; the <== marks what this day touches.

CoreDNS is a DNS server that runs as pods (a Deployment in kube-system), fronted by a Service called kube-dns. It watches the api-server for Services and Endpoints and answers cluster queries from that live data.

kubectl -n kube-system get deploy,pods -l k8s-app=kube-dns
kubectl -n kube-system get svc kube-dns          # the ClusterIP pods point at
kubectl -n kube-system get configmap coredns -o yaml   # the Corefile

Every Service gets a deterministic DNS name:

   <service>.<namespace>.svc.cluster.local        ->  Service ClusterIP
   <service>.<namespace>                            ->  same (search domain fills the rest)

   # headless Service (clusterIP: None) returns the POD IPs directly:
   <pod-hostname>.<service>.<namespace>.svc.cluster.local

   examples:
     payments.prod.svc.cluster.local
     kubernetes.default.svc.cluster.local   (the api-server itself)

The kubelet writes each pod's /etc/resolv.conf to point at the CoreDNS ClusterIP, with a search list and an ndots setting.

kubectl exec -it mypod -- cat /etc/resolv.conf
# nameserver 10.96.0.10
# search default.svc.cluster.local svc.cluster.local cluster.local
# options ndots:5

ndots:5 means names with fewer than 5 dots are tried against each search domain first. That is why payments resolves to payments.<ns>.svc.cluster.local — but it also causes extra lookups for external names (see pitfalls).

Here is the full path when one pod looks up another workload by its Service name and then connects to it.

The quickest way to confirm DNS works is to resolve a known name from inside the cluster, either with a throwaway debug pod or from an existing pod.

# one-off debug pod
kubectl run dns-test --image=busybox:1.36 --restart=Never -it --rm -- \
  nslookup kubernetes.default

# from an existing pod
kubectl exec -it mypod -- nslookup payments.prod
kubectl exec -it mypod -- nslookup payments.prod.svc.cluster.local

A handful of DNS problems come up again and again; these are the usual suspects and what causes each one.

   - CoreDNS pods CrashLooping  -> often a bad Corefile or a node-resolv.conf loop
   - slow external lookups      -> ndots:5 tries search domains first; use a
                                    trailing dot ("github.com.") or set ndots:1
                                    via dnsConfig for chatty external clients
   - NXDOMAIN for a Service     -> wrong namespace, or no Endpoints (no ready pods)
   - headless Service confusion -> clusterIP:None returns POD IPs, not one VIP
   - NetworkPolicy too strict   -> blocks egress to kube-dns (allow UDP/TCP 53)

A pod reaches another workload by stable name, resolved by CoreDNS from live cluster state.

• CoreDNS (pods in kube-system, Service kube-dns) is the cluster resolver.
• It watches the api-server for Services/Endpoints — DNS reflects live state.
• Service name = <svc>.<ns>.svc.cluster.local; headless returns pod IPs.
• Pods point at CoreDNS via /etc/resolv.conf with a search list + ndots:5.
• Debug with nslookup/dig from a pod; check Endpoints when a name fails.

• [ ] Found CoreDNS pods, the kube-dns Service, and the coredns ConfigMap
• [ ] Read a pod's /etc/resolv.conf and explained search + ndots
• [ ] Resolved a Service by short and FQDN name from inside a pod
• [ ] Explained headless Service DNS (pod IPs) vs ClusterIP
• [ ] Know to allow port 53 egress when writing NetworkPolicies