Network troubleshooting — packet flow and tools · Docker & Containers

“Container cannot be reached from outside” is the most common Docker networking incident. The diagnostic tree is short but you must follow it in order. Skip a step and you will spend hours on the wrong layer.

The troubleshooting tree

flowchart TD
  Start[Container unreachable] --> Q1{Host can ping container?}
  Q1 -- no --> P1[Check bridge / veth / iptables]
  Q1 -- yes --> Q2{Container can ping external host?}
  Q2 -- no --> P2[Check default gateway / DNS / outbound iptables]
  Q2 -- yes --> Q3{External host can reach host:published port?}
  Q3 -- no --> P3[Check host firewall / port publishing / proxy]
  Q3 -- yes --> Q4{External host can reach container?}
  Q4 -- no --> P4[Check DNAT rules / conntrack]
  Q4 -- yes --> Done[Working - investigate further if flaky]

Always start with docker inspect to confirm the container’s network configuration, then move outward.

Step 0 — Confirm the basics

docker ps
docker inspect CONTAINER --format '{{json .NetworkSettings}}'
docker network ls
docker network inspect NETWORK

Is the container on the network you expect? Does it have the IP you expect? Is the right port published?

Step 1 — From the host, can you reach the container?

# Ping the container's IP
PIP=$(docker inspect --format '{{range .NetworkSettings.Networks}}{{.IPAddress}}{{end}}' CONTAINER)
ping -c 3 $PIP

# TCP connect to a published port
curl -v http://localhost:8080

# Or from inside a peer container
docker run --rm --network NETWORK alpine nc -zv web 80

If this fails:

The container is not running (docker ps shows Exited).
The container is in a different network namespace than you think.
The bridge or iptables rules are broken (a daemon misconfig).

Step 2 — Inspect the host’s networking

# Show bridges
ip link show type bridge

# Show veth interfaces
ip -o link show | grep veth

# Show iptables DNAT
iptables -t nat -L DOCKER -n -v

# Show conntrack entries
conntrack -L | grep 172.17.0

A working DNAT chain has a rule for the published port.

Step 3 — Trace the packet

# On the host, watch for incoming traffic to the published port
tcpdump -i any -n 'port 8080'

# Inside the container, watch for traffic on the application port
PID=$(docker inspect --format '{{.State.Pid}}' CONTAINER)
nsenter -t $PID -n tcpdump -i eth0 -n 'port 80'

You should see:

The SYN arriving on the host’s external interface.
The SYN appearing inside the container’s eth0.
The SYN-ACK returning from the container to the host.
The SYN-ACK leaving the host’s external interface.

If the SYN arrives on the host but not inside the container, the DNAT or bridge is broken. If the SYN arrives inside the container but no SYN-ACK is returned, the application is not listening or is hung.

The port is published, the DNAT rule is right, and nothing answers

docker ps shows 0.0.0.0:8080->8080/tcp. iptables -t nat -L DOCKER -n has the DNAT rule. tcpdump on the host shows the SYN arriving. The container is Up, its logs say listening on port 8080, and curl http://localhost:8080 from the host gives you one of these:

$ curl -v http://localhost:8080
*   Trying 127.0.0.1:8080...
* Connected to localhost (127.0.0.1) port 8080
> GET / HTTP/1.1
* Empty reply from server

or, with the userland proxy disabled, a flat Connection refused — even though ss -tln on the host clearly shows something listening on 8080.

Everything on the checklist passes, so the search widens: the firewall, the reverse proxy, the health check, the image. Hours go into the network. The network is fine.

What is actually happening. The application is bound to 127.0.0.1:8080 inside the container. That is the container’s own loopback interface, in its own network namespace, and nothing outside that namespace can reach it — not the host, not the bridge, not another container, not the DNAT rule. Publishing a port cannot help: the DNAT rewrites the destination to the container’s bridge address, and there is nothing listening there.

The two error messages tell you which path you are on and neither one points at the cause. Empty reply from server is docker-proxy accepting your connection on the host and then failing to dial the container. Connection refused is the container’s kernel sending an RST because the DNAT’d packet arrived at an address with no listener. Both read as “the application is broken”, which is nearly true and completely unactionable.

Why it survives testing. Binding to loopback is the correct, security-conscious default on a developer laptop, and it is what a lot of frameworks ship: HOST=localhost, bind-address = 127.0.0.1, app.listen(8080, 'localhost'), rails s -b localhost. It works perfectly outside a container, so nothing flags it until the same config runs inside one.

How to tell, in one command. Look at the listening socket from inside the container’s network namespace. 0.0.0.0:8080 or [::]:8080 is reachable. 127.0.0.1:8080 is not, and no amount of network configuration will make it so.

The fix is in the application, not in Docker. Bind to 0.0.0.0 (or ::) and control exposure at the publish instead — -p 127.0.0.1:8080:8080 gives you the same “only reachable from this host” property, enforced at a layer that actually works.

Read-only / Safewhat is the application really bound to?— Ask inside the container's network namespace. This single line separates 'the network is broken' from 'the application is listening in the wrong place', and it is the first thing to run, not the last.

CONTAINER=web
PID=$(docker inspect --format '{{.State.Pid}}' "$CONTAINER")
sudo nsenter -t "$PID" -n ss -tlnp

State  Recv-Q Send-Q Local Address:Port Peer Address:Port Process
LISTEN 0      511        127.0.0.1:8080      0.0.0.0:*         users:(("node",pid=4127,fd=20))

Illustrative output

Read the Local Address column and nothing else:

Local Address	Reachable from the host via a published port?
`0.0.0.0:8080`	Yes — bound to every interface in the namespace.
`[::]:8080`	Yes, and on IPv4 too on most hosts, via the v4-mapped path.
`172.19.0.4:8080`	Yes — bound to the container’s bridge address specifically.
`127.0.0.1:8080`	No. Container loopback only.
nothing listening	No — the application never bound at all; read its logs.

If you cannot use nsenter, the same answer is available with docker exec "$CONTAINER" ss -tlnp on images that ship ss, or docker exec "$CONTAINER" cat /proc/net/tcp on ones that do not — in that file the local address is hex and little-endian, so loopback appears as 0100007F.

Read-only / Safeconfirm the mapping separately from the listener— docker port reports what Docker published. It says nothing about whether anything is listening on the container side, which is exactly why the two checks have to be separate.

CONTAINER=web
docker port "$CONTAINER"
PID=$(docker inspect --format '{{.State.Pid}}' "$CONTAINER")
CIP=$(docker inspect --format '{{range .NetworkSettings.Networks}}{{.IPAddress}}{{end}}' "$CONTAINER")
curl -sS -o /dev/null -w 'direct to container: %{http_code}\n' "http://$CIP:8080/" || true
curl -sS -o /dev/null -w 'via published port: %{http_code}\n' 'http://127.0.0.1:8080/' || true

8080/tcp -> 0.0.0.0:8080
curl: (7) Failed to connect to 172.19.0.4 port 8080: Connection refused
via published port: 000

Illustrative output

Connecting directly to the container’s IP is the decisive test, because it takes DNAT, docker-proxy and the host firewall entirely out of the path. If that fails too, the problem is inside the container and no networking change will fix it. If it succeeds while the published port fails, then — and only then — the problem really is in the publishing path.

Step 4 — Inspect from inside the container

# Inside the container's network namespace
PID=$(docker inspect --format '{{.State.Pid}}' CONTAINER)
nsenter -t $PID -n ss -tlnp
nsenter -t $PID -n ss -tnp  # established connections
nsenter -t $PID -n ip route
nsenter -t $PID -n ip addr
nsenter -t $PID -n iptables -t nat -L

What is the application actually listening on? What does the container think its IP and gateway are?

Step 5 — DNS

docker exec CONTAINER cat /etc/resolv.conf
docker exec CONTAINER nslookup web
docker exec CONTAINER getent hosts web

DNS failures look like TCP failures but are diagnosed by nslookup.

Tools you should know

Tool	Purpose
`ip link`	Show interfaces
`ip addr`	Show addresses
`ip route`	Show routing table
`ss -tlnp`	Listening TCP sockets
`ss -tnp`	Established TCP sockets
`tcpdump`	Packet capture
`nsenter -t PID -n CMD`	Run a command in the container’s network namespace
`nft list ruleset`	nftables ruleset
`iptables -t nat -L`	iptables NAT table
`conntrack -L`	Connection tracking entries
`bridge link show`	Linux bridge port list