Kubernetes Service Discovery

Overview

Kubernetes is a dynamic system. It schedules Pods onto nodes, restarts them when they crash, and moves them around as cluster conditions change. Each time a Pod is rescheduled, it gets a new IP address. This creates a fundamental problem: how does one piece of your application find another when its address keeps changing?

This problem — finding which process is listening at which address for which service — is called service discovery. Traditional DNS works well on the internet where addresses are stable, but it falls short inside Kubernetes where IP addresses change constantly and you may have dozens of replicas behind a single logical name.

Kubernetes solves this with the Service object: a stable virtual IP address (called a cluster IP) that automatically tracks which Pods are ready and load-balances traffic across them. In this article, you will learn how Services work, how Kubernetes DNS resolves them, and how to expose your application inside the cluster, to your private network, and to the public internet.

Core Concepts

Step 1: Why DNS Alone Is Not Enough

Your first instinct might be to just use DNS to find Pods. Kubernetes does provide DNS, but raw DNS has several problems in a dynamic cluster:

1. Caching. Many runtimes (Java, for example) look up a DNS name once and cache the result forever. If that Pod is replaced, the cached IP is stale, and traffic goes to a dead address.
2. TTL delays. Even well-behaved clients take time to notice when a DNS record changes. During that window, some requests fail.
3. Scale limits. DNS breaks down beyond 20–30 A records for a single name, and it has no concept of health.
4. No health awareness. DNS returns all addresses, including addresses of Pods that are starting up or overloaded.

The Kubernetes Service object solves all of these problems by introducing a stable virtual IP that never changes even as the backing Pods do.

Step 2: The Service Object

A Service is a Kubernetes object that gives a stable name and IP address to a dynamic set of Pods. It does this by pairing a label selector with a cluster IP. The selector matches Pods; the cluster IP stays constant.

The cluster IP is a virtual IP — it does not belong to any real network interface. A component called kube-proxy runs on every node and programs iptables rules that intercept traffic sent to the cluster IP and redirect it to one of the healthy, ready Pod IPs behind the service.

Key things a Service provides:

1. A stable cluster IP that never changes while the Service exists.
2. A DNS name automatically registered by the cluster DNS (CoreDNS).
3. Health filtering — traffic is only routed to Pods that pass their readiness check.
4. Load balancing across all healthy Pod replicas.

Step 3: Creating a Service with a Manifest

A Service is defined with a spec.selector that matches Pod labels and a spec.ports block that maps a service port to a container port. Here is a Service that targets our .NET Catalog API pods:

The port field is what other services use to call this service (catalog-api:80). The targetPort is the port the container is actually listening on (8080 for ASP.NET Core). The two can differ — this is very useful for standardizing all your services on port 80 without changing container port configurations.

Step 4: The Deployment Behind the Service

A Service needs Pods to route traffic to. The Pods are typically managed by a Deployment. The Deployment's pod template must include the labels that the Service's selector is looking for.

Here is the Deployment for the Catalog API. Notice how its pod template labels (app: catalog-api, env: prod) match the Service selector exactly:

The readiness probe is important here. The Service will only forward traffic to Pods where the readiness probe is currently passing. A Pod that is still initializing, or is overloaded and returning errors, will be automatically removed from the Service's active backend list until it recovers.

Step 5: Service DNS

The cluster's DNS server (CoreDNS) automatically creates a DNS entry for every Service. Because the cluster IP is a stable virtual address — unlike Pod IPs — DNS caching is no longer a problem. The IP that DNS returns for a Service will never change unless you delete and recreate the Service.

The full DNS name of a Service follows this pattern:

Breaking down catalog-api.default.svc.cluster.local:

Pods within the same namespace can use just the short name: catalog-api. Pods in other namespaces use catalog-api.default or the fully qualified name. This means your ASP.NET application running in the same namespace can call the Catalog API with:

Step 6: Readiness Checks and Service Traffic

Pods go through phases: they start up, initialize caches, open database connections — and only then are they ready to serve requests. Without readiness checking, Kubernetes would forward traffic to a Pod the instant its container process starts, which would cause errors during the startup window.

The Service object continuously monitors the readiness state of its backing Pods via the Endpoints object. Every Service has a companion Endpoints object that Kubernetes keeps up to date with the IPs of all currently-ready Pods:

When a Pod fails its readiness probe, its IP is removed from the Endpoints list and no new traffic reaches it. When the probe passes again, the IP is re-added automatically. This enables graceful handling of overloaded or restarting Pods with zero manual intervention.

Step 7: Service Types

Kubernetes provides four service types. Each one builds on the previous:

Step 8: NodePort — Exposing Outside the Cluster

A NodePort service allocates a port (in the range 30000–32767 by default) on every node in the cluster. Any traffic arriving on that node port is forwarded to the service and then to one of its backing Pods — regardless of which node the Pod is running on.

You can specify the port you want or let Kubernetes pick one automatically:

After applying this, you can reach the service at <any-node-ip>:31080. This is useful during development on a local cluster (like kind or kubeadm) or when you have a hardware load balancer that you manage separately.

Step 9: LoadBalancer — Cloud Load Balancer Integration

On cloud providers (GKE, AKS, EKS), a LoadBalancer service automatically provisions a cloud load balancer with a public IP address and points it at your NodePorts. You do not need to manually configure anything — Kubernetes communicates with the cloud provider API on your behalf.

After applying this, check the assigned external IP:

It may take a minute for the cloud to provision the load balancer. Once the EXTERNAL-IP appears, your service is publicly reachable at that IP on port 80.

Security consideration: A LoadBalancer service exposes your application to the public internet. Ensure your application has proper authentication and that you have reviewed what endpoints are accessible before creating a public LoadBalancer.

Step 10: Internal Load Balancers

Often you want a load balancer that is accessible from your private virtual network but not from the public internet. Cloud providers support this via annotations on the Service object. The annotation tells the cloud provider to create an internal (private) load balancer instead of a public-facing one.

Example for an AKS internal load balancer:

This is the correct way to expose a service to other internal systems (such as on-premise services connecting over a VPN) without exposing it to the public internet.

Step 11: Selector-less Services for External Resources

Sometimes your Kubernetes application needs to call a resource that lives outside the cluster — for example, a legacy database, an on-premise API, or a managed cloud service not provisioned by Kubernetes. You want to give it a Kubernetes DNS name so your code doesn't need to know whether the backing resource is inside or outside the cluster.

The solution is a selector-less Service paired with a manually managed Endpoints object. The Service has no spec.selector, so Kubernetes will not automatically populate its endpoints. You supply the backend IP address manually:

After applying this, any Pod in the cluster can reach the external database at legacy-db:1433 through normal Kubernetes DNS. The application code does not need to know the real IP address. If the database is ever migrated to a new IP, you update only the Endpoints object — the application configuration and code stay unchanged.

Step 12: How kube-proxy Makes Cluster IPs Work

You might wonder: who actually routes traffic from the cluster IP to the backend Pods? The answer is kube-proxy, a component that runs as a DaemonSet on every node.

When you create a Service, the Kubernetes API server assigns it a cluster IP from the service IP range. kube-proxy watches the API for new Services and Endpoints. For each one, it programs iptables rules on the node that intercept packets destined for the cluster IP and redirect them to one of the ready Pod IP addresses.

This means:

1. The cluster IP never appears on any real network interface — it is a virtual address that only exists in iptables rules.
2. Load balancing happens at the kernel packet level — very fast, with no userspace proxy overhead.
3. When Pod readiness changes, kube-proxy updates the iptables rules within seconds.

Hands-On: Kubernetes Commands

Create a Service Imperatively

The quickest way to expose a Deployment as a Service:

List All Services

Get Service Details with Selector Info

Describe a Service

Look for the Endpoints: line — it shows the current live Pod IPs behind the service.

Inspect the Endpoints Object

Watch Endpoints Change in Real Time

In one terminal, watch the endpoints:

In another terminal, scale the deployment up or down and observe the endpoints list update live:

Port-Forward to a Pod for Local Testing

Grab the name of a running Pod and forward a local port to it:

Then test the API from your machine:

Test DNS Resolution from Inside a Pod

Start a temporary container in the same namespace and resolve the service name:

Test Cross-Namespace DNS

Delete a Service

Step-by-Step Example

Let us deploy a .NET Catalog API with three replicas, expose it as a ClusterIP Service, verify DNS resolution and readiness probe integration, then change the service type to NodePort and LoadBalancer.

1. Build the container image. Save this Dockerfile in your ASP.NET Core project root:

1. Your ASP.NET Core application must register the health check endpoints. In Program.cs:

1. Deploy the Catalog API. Apply the deployment manifest:

1. Watch the Pods start up:

1. Notice the Pods start in 0/1 Ready state. The readiness probe runs every 10 seconds starting 5 seconds after the container starts. Only after the probe passes does the Pod become 1/1 Ready.
2. Create the ClusterIP Service.

1. Verify that the Endpoints object lists the ready Pods.

1. All three Pod IPs appear because all three readiness probes are passing.
2. Verify DNS resolution from inside the cluster.

1. The short name catalog-api resolved to the stable cluster IP. Any other Pod in the default namespace can reach the Catalog API at http://catalog-api/products — no hardcoded IPs needed.
2. Watch readiness probe behavior in action. Open two terminals. In terminal 1, watch the endpoints:

1. In terminal 2, scale the deployment down to 1 replica:

1. Back in terminal 1, you will see the endpoints list shrink as Pods terminate and their IPs are removed from the active backend list. Scale back up when done:

1. Change to NodePort for external access.

1. You can now reach the service on any cluster node at port 31080. On a local cluster:

1. Create a LoadBalancer service for external cloud access.

1. Once the EXTERNAL-IP is assigned, your API is publicly accessible:

1. Register an external database as a Kubernetes Service. If your application connects to an on-premise SQL Server at 10.0.1.50:1433, apply the selector-less Service and Endpoints:

1. Verify the manually managed endpoint is visible:

1. Your application Pods can now connect to the database using legacy-db:1433 as the connection string host — no code change is needed if the database IP changes in the future, only the Endpoints object needs updating.
2. Clean up everything created in this example:

Summary

Kubernetes is a dynamic system — Pod IPs change constantly as Pods are created, moved, and restarted. The Service object is the solution: it provides a stable cluster IP and a DNS name that persists across Pod lifecycles, and it automatically tracks which Pods are healthy and ready to receive traffic via the Endpoints object.

There are four service types to match your exposure needs: ClusterIP for internal cluster traffic, NodePort for accessing services on a fixed node port, LoadBalancer for cloud-provisioned public or private load balancers, and selector-less services for bridging Kubernetes DNS to external resources. Combined with readiness probes, Services enable zero-configuration load balancing and graceful rollouts with no traffic interruption.