Skip to main content
RunBook Academy

KubernetesCXXI · Service TroubleshootingService troubleshooting

Headless and ExternalName — the DNS-only Services

Advanced⏱ ~14 minkubectl

What you'll learn

  • Apply the 11-step methodology to headless and ExternalName Services
  • Distinguish a headless Service from a regular ClusterIP
  • Diagnose ExternalName DNS failures
  • Identify the production failure modes of headless and ExternalName Services

Prerequisites

Verified against Kubernetes 1.34.x · kubeadm 1.34.x · kubectl 1.34.x · etcd 3.6.x · CoreDNS 1.11.x · containerd 1.7.x / 2.x · 2026-08-16

Not yet marked complete on this device.

A headless Service has no ClusterIP and an ExternalName Service has no endpoints at all, so neither of them produces a kube-proxy rule anywhere in the cluster. Everything they do happens in DNS: the answer is a list of Pod IPs in one case and a CNAME in the other. That makes the failure surface small and unusual — an empty answer, a stale answer, or a CNAME that resolves nowhere — and this lesson walks each of them.

The headless Service

A headless Service is one with clusterIP: None. The EndpointSlice controller publishes the Pod IPs directly to the Service’s DNS. The application’s DNS query returns the Pod IPs, not a ClusterIP.

flowchart LR
    A[Application] --> B[DNS query]
    B --> C[Pod IPs]
    C --> D[Direct routing]

The kube-proxy is not in the path. The application connects directly to the Pod IPs.

A real headless Service:

apiVersion: v1
kind: Service
metadata:
  name: billing
  namespace: prod
spec:
  clusterIP: None
  selector:
    app: billing
  ports:
  - port: 80
    targetPort: 8080

The diagnostic is the DNS:

# Substitute your own values before running:
SOURCE_POD=checkout-6b9f4c7d55-2xk9p

kubectl exec -it "$SOURCE_POD" -- nslookup billing.prod.svc.cluster.local

The response returns the Pod IPs:

Server:    10.96.0.10
Address 1: 10.96.0.10 kube-dns.svc.cluster.local

Name:      billing.prod.svc.cluster.local
Address 1: 10.244.5.23
Address 2: 10.244.5.24
Address 3: 10.244.7.10
Address 4: 10.244.7.11

The headless Service’s DNS returns the Pod IPs directly.

The ExternalName Service

An ExternalName Service is one with kind: ExternalName. The Service’s DNS is a CNAME to an external DNS name. The application’s DNS query returns the external name.

flowchart LR
    A[Application] --> B[DNS query]
    B --> C[CNAME: external-db.example.com]
    C --> D[External resolution]

The kube-proxy is not in the path. The application connects to the external service.

A real ExternalName Service:

apiVersion: v1
kind: Service
metadata:
  name: external-db
  namespace: prod
spec:
  type: ExternalName
  externalName: db.example.com

The diagnostic is the DNS:

# Substitute your own values before running:
SOURCE_POD=checkout-6b9f4c7d55-2xk9p

kubectl exec -it "$SOURCE_POD" -- nslookup external-db.prod.svc.cluster.local

The response returns the CNAME:

Server:    10.96.0.10
Address 1: 10.96.0.10 kube-dns.svc.cluster.local

Name:      external-db.prod.svc.cluster.local
external-db.prod.svc.cluster.local  canonicalName = db.example.com.

The ExternalName Service’s DNS returns the CNAME.

Headless failures

The most common headless failures:

  • No Pods Ready. The EndpointSlice is empty because no Pods match the selector and are Ready.
  • Selector mismatch. The Service’s selector does not match the Pods’ labels.
  • DNS cache. The application’s DNS cache is stale.
# Substitute your own values before running:
SOURCE_POD=checkout-6b9f4c7d55-2xk9p

# Diagnose the headless Service
kubectl get endpointslices -n prod -l kubernetes.io/service-name=billing

# Test the DNS
kubectl exec -it "$SOURCE_POD" -- nslookup billing.prod.svc.cluster.local

# Clear the DNS cache (in the application)
kubectl exec -it "$SOURCE_POD" -- nscd -i hosts

ExternalName failures

The most common ExternalName failures:

  • External DNS not reachable. The CoreDNS Pod cannot reach the external DNS server.
  • External service is down. The external service is down.
  • DNS cache. The application cached the CNAME’s old target and keeps dialing the previous host until the TTL expires.
# Substitute your own values before running:
SOURCE_POD=checkout-6b9f4c7d55-2xk9p

# Test the DNS
kubectl exec -it "$SOURCE_POD" -- nslookup external-db.prod.svc.cluster.local

# Test the external DNS
kubectl exec -it "$SOURCE_POD" -- nslookup db.example.com 8.8.8.8

# Test the external service
kubectl exec -it "$SOURCE_POD" -- curl -v http://db.example.com

Production discipline

A headless or ExternalName Service failure is the cluster’s DNS hypothesis. The discipline is to walk the DNS-only path, identify the failure mode, apply the remediation. The flow is shorter; the discipline is the same.

  • Walk the DNS-only path. Headless and ExternalName Services bypass the kube-proxy.
  • Test the Pod readiness for headless. The EndpointSlice is the source of truth for the Pod IPs.
  • Test the external DNS for ExternalName. The CNAME is the source of truth.

Quiz

Knowledge check · 4 questions

  1. Q1. What is the difference between a headless Service and a regular ClusterIP?

  2. Q2. An ExternalName Service's DNS returns a CNAME to an external DNS name.

  3. Q3. An operator runs `kubectl exec -it <source-pod> -- nslookup billing.prod.svc.cluster.local`. The response is NXDOMAIN. The Service is headless with selector `app=billing`. The Pods are Running with `app=billing`. What is the diagnostic and remediation?

    The Service is `billing` in namespace `prod`. The Service is headless (clusterIP: None). The Pods are `billing-7d8f-abcde` and `billing-7d8f-def01` in namespace `prod`. The Pods are Running with `app=billing`. The DNS query returns NXDOMAIN.

  4. Q4. Name the diagnostic command for a headless Service and an ExternalName Service.

Passing score: 75%. Answers are checked in this browser.