Search domains and ndots — the lookups you did not ask for

An application connects to api. The container is on a user-defined network, another container on that network is called api, and it still gets the wrong address — the address of a staging box in another datacentre. Nobody typed that address anywhere. The resolver invented the question that produced it.

That is the search list doing its job. This lesson is about the two resolver options that control it, because between them they cause more misdirected traffic and more mysterious latency than any other part of container DNS.

The search list

A search line in /etc/resolv.conf is a list of suffixes the resolver may append to a name before giving up:

$ docker exec web cat /etc/resolv.conf

nameserver 127.0.0.11
search corp.example.com svc.corp.example.com
options ndots:0

With that file, a lookup for api can become up to three separate questions:

• api. (the literal name)
• api.corp.example.com.
• api.svc.corp.example.com.

Docker copies the host’s search line into the container. You did not ask for it; the host had it, so the container has it. On a corporate laptop or a cloud VM that line is rarely empty.

resolv.conf(5) caps the list: six domains, 256 characters total. Anything past that is silently dropped, which is its own quiet failure mode on hosts with a generous DHCP-supplied search list.

ndots decides the order

ndots:n sets a threshold for how many dots a name must contain before the resolver tries it as an absolute name first. The man page states the default is 1: a name containing any dot is tried literally before the search list is applied; a name with no dots goes through the search list first.

Docker writes options ndots:0 into the generated file for containers on user-defined networks. Zero means always try the literal name first, which is exactly right for container DNS — api, db, redis are single-label names that the embedded DNS can answer directly, and trying the literal form first means one query and one answer.

The Kubernetes row is there because engineers arriving from Kubernetes often carry ndots:5 habits and configuration into Compose. On Docker it is almost always the wrong setting: it turns every two-label name into a burst of pointless queries before the correct one is asked.

Counting the queries

One connect() to a name is not one DNS query. A dual-stack resolver asks for A and AAAA, and it does that for every element of the search path it tries. With two search domains and a name that needs the full expansion, a single connection attempt costs six queries.

CTR=web
PID=$(docker inspect --format '{{.State.Pid}}' "$CTR")

# Watch DNS from inside the container network namespace
nsenter -t "$PID" -n tcpdump -i any -n port 53 &

docker exec "$CTR" getent hosts api
wait

Each failed expansion is a round trip. When the upstream is a corporate resolver two hops away and the name genuinely does not exist, the failure that the application reports as “DNS timeout” is often six sequential timeouts stacked end to end.

The wildcard trap

The defences, in order of preference:

1. Do not let the host’s search list reach the container. Set dns_search explicitly on the service — to the domains the container genuinely needs and nothing else — rather than inheriting whatever DHCP handed the host.
2. Use the trailing dot. api. is a fully qualified name and is never expanded. Most application config parsers accept it; some do not, which is why it is second on this list.
3. Keep ndots at 0 so the literal name is always tried first. The wildcard then only answers after the correct answer has already failed.

Forcing the question you meant to ask

dig will show you both halves of the story:

# Ask exactly the name given, no expansion (dig default)
docker exec web dig +noall +answer api

# Apply the search list the way getaddrinfo would
docker exec web dig +search +noall +answer api

# Ask an absolute name explicitly
docker exec web dig +noall +answer api.corp.example.com.

If line one returns the container address and line two returns something else, the search list is winning and ndots is not 0. That is a two-command diagnosis for a class of bug that otherwise eats an afternoon.

Sanity check