Kubernetes DNS Debugging with dnsutils
Every Service you create in Kubernetes automatically gets a DNS name. When a Pod wants to talk to another Service, it resolves that name through the cluster's built-in DNS server — CoreDNS. Most of the time this happens invisibly and everything just works.
But what happens when DNS resolution fails? Your application logs might show "could not resolve host" or connections time out. Without a way to look inside the cluster's DNS system, you are debugging blind.
This is where the dnsutils Pod comes in. It is a lightweight debugging container that ships with DNS tools — nslookup, dig, and host — so you can run queries from inside the cluster, exactly the way your application Pods do. Think of it as your DNS troubleshooting toolkit that lives right next to your workloads.
Core Concepts
How DNS Works in Kubernetes
Before diving into dnsutils, you need to understand the DNS machinery running inside every Kubernetes cluster:
CoreDNS runs as a Deployment in the kube-system namespace. It watches the Kubernetes API for new Services and Pods and creates DNS records for them automatically.
Every Pod gets a /etc/resolv.conf file configured by the kubelet. This file points the Pod's DNS resolver to the CoreDNS Service IP.
When a Pod looks up my-service, the resolver appends the search domains listed in /etc/resolv.conf (like .default.svc.cluster.local) and queries CoreDNS.
CoreDNS returns the ClusterIP of the matching Service, and the Pod connects.
DNS Record Formats
Kubernetes creates different DNS records depending on the resource type:
Resource DNS Format Example
ClusterIP Service <service>.<namespace>.svc.cluster.local order-api.production.svc.cluster.local
Headless Service <pod>.<service>.<namespace>.svc.cluster.local db-0.postgres-headless.data.svc.cluster.local
SRV Record (named port) _<port-name>._<protocol>.<service>.<namespace>.svc.cluster.local _http._tcp.order-api.production.svc.cluster.local
ExternalName Service Returns a CNAME record pointing to the external host my-db.default.svc.cluster.local → db.example.com
What Is the dnsutils Image?
The Kubernetes project provides an official debugging image as part of its end-to-end test suite:
registry.k8s.io/e2e-test-images/agnhost:2.39
This image is based on Debian and includes essential DNS tools:
nslookup — quick name-to-IP resolution checks.
dig — detailed DNS queries showing full response headers, flags, and TTLs.
host — simplified DNS lookups.
cat — useful for inspecting /etc/resolv.conf.
There is also a legacy community image on Docker Hub called tutum/dnsutils. It is smaller and simpler, but it has not been updated in over 10 years. For production debugging, always prefer the official Kubernetes image.
Understanding /etc/resolv.conf
Every Pod in Kubernetes gets a /etc/resolv.conf file that controls how DNS resolution works. A typical file looks like this:
nameserver 10.96.0.10
search default.svc.cluster.local svc.cluster.local cluster.local
options ndots:5
Here is what each line means:
nameserver — the IP address of the CoreDNS Service. All DNS queries go here first.
search — a list of domain suffixes. When you look up my-service, the resolver tries my-service.default.svc.cluster.local, my-service.svc.cluster.local, my-service.cluster.local, and finally the bare name.
ndots:5 — if the name you query has fewer than 5 dots, the resolver appends the search domains before trying the absolute name. This is why short names like my-service resolve correctly inside the cluster.
DNS Policies
You can control how a Pod resolves DNS by setting the dnsPolicy field in the Pod spec:
Policy Behaviour
ClusterFirst (default) Queries go to CoreDNS first. If they do not match a cluster domain, CoreDNS forwards them to the upstream nameserver.
Default The Pod inherits the DNS configuration from the Node it runs on. Cluster Services are not resolvable.
ClusterFirstWithHostNet Same as ClusterFirst, but for Pods running with hostNetwork: true.
None Kubernetes ignores all DNS settings. You must provide DNS config through the dnsConfig field.
Hands-On: Kubernetes Commands
Deploying the dnsutils Pod
Apply the dnsutils Pod manifest to create a long-running debugging container:
kubectl apply -f dns-debug-pod.yaml
Wait for the Pod to be ready:
kubectl get pod dns-debug-pod
Looking Up a Service with nslookup
The most common debugging command. This queries the cluster DNS for the kubernetes Service in the default namespace:
kubectl exec -it dns-debug-pod -- nslookup kubernetes.default
A successful response looks like:
Server: 10.96.0.10
Address: 10.96.0.10#53
Name: kubernetes.default.svc.cluster.local
Address: 10.96.0.1
If this fails, your cluster DNS is broken and you need to check CoreDNS.
Detailed Query with dig
Use dig for more detail — it shows query time, TTL, response flags, and the full answer section:
kubectl exec -it dns-debug-pod -- dig kubernetes.default.svc.cluster.local
You can also query for SRV records to discover named ports:
kubectl exec -it dns-debug-pod -- dig SRV _https._tcp.kubernetes.default.svc.cluster.local
Simple Lookup with host
The host command gives a one-line answer — useful for quick checks:
kubectl exec -it dns-debug-pod -- host kubernetes.default
Inspecting resolv.conf
See exactly what DNS configuration the kubelet injected into the Pod:
kubectl exec -it dns-debug-pod -- cat /etc/resolv.conf
Checking CoreDNS Health
If DNS is not working, verify that CoreDNS Pods are running:
kubectl get pods -n kube-system -l k8s-app=kube-dns
Check CoreDNS logs for errors:
kubectl logs -n kube-system -l k8s-app=kube-dns
Verify the CoreDNS Service has endpoints:
kubectl get endpoints kube-dns -n kube-system
Testing Cross-Namespace Resolution
To resolve a Service in a different namespace, use the fully qualified domain name (FQDN):
kubectl exec -it dns-debug-pod -- nslookup payment-api.finance.svc.cluster.local
Testing External DNS Resolution
Verify that the cluster can resolve external domain names (forwarded through CoreDNS to upstream resolvers):
kubectl exec -it dns-debug-pod -- nslookup www.google.com
Step-by-Step Example
In this example, you will deploy a .NET 10 Web API application, expose it with a Service, then use the dnsutils Pod to verify that DNS resolution works end to end.
Step 1 — Create the Dockerfile
Below is a multi-stage Dockerfile for a .NET 10 ASP.NET Core application called Weather Forecast API:
FROM mcr.microsoft.com/dotnet/sdk:10.0 AS build
WORKDIR /src
COPY ["WeatherForecastApi.csproj", "."]
RUN dotnet restore
COPY . .
RUN dotnet publish -c Release -o /app/publish
FROM mcr.microsoft.com/dotnet/aspnet:10.0
WORKDIR /app
COPY --from=build /app/publish .
EXPOSE 8080
ENTRYPOINT ["dotnet", "WeatherForecastApi.dll"]
Step 2 — Deploy the Application
Apply the Deployment manifest that runs two replicas of the Weather Forecast API:
apiVersion: apps/v1
kind: Deployment
metadata:
name: weather-api-deployment
labels:
app: weather-api
spec:
replicas: 2
selector:
matchLabels:
app: weather-api
template:
metadata:
labels:
app: weather-api
spec:
containers:
- name: weather-api
image: myregistry.azurecr.io/weather-forecast-api:1.0
ports:
- containerPort: 8080
resources:
requests:
cpu: "100m"
memory: "128Mi"
limits:
cpu: "500m"
memory: "256Mi"
readinessProbe:
httpGet:
path: /healthz
port: 8080
initialDelaySeconds: 5
periodSeconds: 10
kubectl apply -f weather-api-deployment.yaml
Step 3 — Expose the Application with a Service
Create a ClusterIP Service so other Pods can reach the Weather Forecast API by name:
apiVersion: v1
kind: Service
metadata:
name: weather-api-service
spec:
type: ClusterIP
selector:
app: weather-api
ports:
- name: http
port: 80
targetPort: 8080
kubectl apply -f weather-api-service.yaml
Step 4 — Deploy the dnsutils Pod
Deploy the DNS debugging Pod so you can run DNS queries from inside the cluster:
apiVersion: v1
kind: Pod
metadata:
name: dns-debug-pod
labels:
app: dns-debug
spec:
containers:
- name: dnsutils
image: registry.k8s.io/e2e-test-images/agnhost:2.39
command: ["sleep", "infinity"]
resources:
requests:
cpu: "50m"
memory: "64Mi"
limits:
cpu: "200m"
memory: "128Mi"
restartPolicy: Always
kubectl apply -f dns-debug-pod.yaml
Wait until the Pod is running:
kubectl wait --for=condition=Ready pod/dns-debug-pod --timeout=60s
Step 5 — Verify DNS Resolution with nslookup
Use nslookup to check that the weather API Service is resolvable:
kubectl exec -it dns-debug-pod -- nslookup weather-api-service
You should see the ClusterIP address of the Service in the response:
Server: 10.96.0.10
Address: 10.96.0.10#53
Name: weather-api-service.default.svc.cluster.local
Address: 10.96.73.42
Step 6 — Get Full Details with dig
Run a dig query for the complete DNS response including TTL and flags:
kubectl exec -it dns-debug-pod -- dig weather-api-service.default.svc.cluster.local
Look at the ANSWER SECTION to see the A record. The Query time at the bottom tells you how long the resolution took (it should be under a few milliseconds inside the cluster).
Step 7 — Inspect the Pod's resolv.conf
Verify the DNS configuration that the kubelet injected:
kubectl exec -it dns-debug-pod -- cat /etc/resolv.conf
Confirm that the nameserver points to your CoreDNS Service IP and that the search domains include your namespace.
Step 8 — Test Cross-Namespace and External Resolution
Resolve a Service in another namespace:
kubectl exec -it dns-debug-pod -- nslookup kube-dns.kube-system
Resolve an external domain to confirm CoreDNS forwarding works:
kubectl exec -it dns-debug-pod -- nslookup www.microsoft.com
Step 9 — Troubleshooting Checklist
If DNS resolution fails, follow these steps in order:
Check that the CoreDNS Pods are running:
kubectl get pods -n kube-system -l k8s-app=kube-dns
Check CoreDNS logs for errors:
kubectl logs -n kube-system -l k8s-app=kube-dns
Verify the kube-dns Service exists and has endpoints:
kubectl get svc kube-dns -n kube-system
kubectl get endpoints kube-dns -n kube-system
Check that the Pod's /etc/resolv.conf points to the correct nameserver:
kubectl exec -it dns-debug-pod -- cat /etc/resolv.conf
Check the CoreDNS ConfigMap for misconfigurations:
kubectl get configmap coredns -n kube-system -o yaml
Verify that the target Service and its endpoints exist:
kubectl get svc weather-api-service
kubectl get endpoints weather-api-service
Step 10 — Clean Up
Remove the resources you created:
kubectl delete -f dns-debug-pod.yaml
kubectl delete -f weather-api-service.yaml
kubectl delete -f weather-api-deployment.yaml
Summary
DNS is the backbone of service-to-service communication in Kubernetes. When it breaks, nothing works. Here is what you should remember:
Every Service automatically gets a DNS name in the format <service>.<namespace>.svc.cluster.local.
CoreDNS is the cluster DNS server. It runs in kube-system and watches the API server for changes.
The dnsutils Pod (using registry.k8s.io/e2e-test-images/agnhost:2.39) gives you nslookup, dig, and host for debugging DNS from inside the cluster.
Always check /etc/resolv.conf first — it tells you where the Pod sends its DNS queries and which search domains are in use.
The ndots:5 option means short names (fewer than 5 dots) are resolved using the search domain list before trying the absolute name.