Network segmentation and management access

A production Docker host carries traffic with at least three different trust levels. Mixing them is how a single compromised container becomes a data breach, and the mixing usually happens by accident rather than by design.

The three planes

• Management. SSH, the monitoring exporters, the daemon itself. Reachable only from a jump host or a VPN, never from the internet, and never from a container.
• Application. The frontend and the API. Reachable from the internet, but only through the reverse proxy, and only on the ports the proxy owns.
• Data. The database, the cache, the message broker. Reachable from the application and from nothing else, with no route out to the internet.

flowchart TB
  Internet((Internet)) -->|443 only| EXT[eth0 203.0.113.10]
  VPN((VPN / jump host)) -->|22, 9100| MGMT[eth1 192.0.2.10]
  EXT --> Proxy[caddy<br/>published on eth0 only]
  MGMT --> Host[sshd, node_exporter,<br/>dockerd]
  Proxy --> AppNet[app-net bridge<br/>br-app]
  AppNet --> API[api]
  API --> DataNet[data-net bridge<br/>br-data<br/>internal: true]
  DataNet --> DB[(postgres)]
  DataNet --> Cache[(redis)]

The important edge in that diagram is the one that is missing: there is no arrow from DataNet back out to EXT. That is what internal: true buys you, and it is enforced by the absence of a route rather than by a rule somebody has to maintain.

The failure that makes firewalls useless

Before any ruleset, understand why the obvious one does not work.

Publishing to the interface you meant

The short ports: ["5432:5432"] form binds 0.0.0.0 — every address on every interface, including the public one. On a host with a management NIC and a public NIC, that is almost never what you want.

services:
  db:
    image: postgres:16
    # No ports: at all. api reaches it as db:5432 over data-net.
    networks: [data-net]

  api:
    image: myorg/api:1.4.0
    networks: [app-net, data-net]

  caddy:
    image: caddy:2.9
    ports:
      # Public service: bind explicitly to the external address
      - "203.0.113.10:443:443"
      - "203.0.113.10:80:80"
    networks: [app-net]

  metrics:
    image: prom/node-exporter:v1.8.2
    ports:
      # Management-only service: bind to the management address
      - "192.0.2.10:9100:9100"
    networks: [app-net]

networks:
  app-net:
    driver_opts:
      com.docker.network.bridge.name: br-app
  data-net:
    internal: true
    driver_opts:
      com.docker.network.bridge.name: br-data

Two things in that file are doing real work beyond documentation.

internal: true on data-net means the daemon creates no gateway route and no masquerade rule for it. A compromised database container cannot reach the internet to exfiltrate anything or pull a second-stage payload — not because a rule denies it, but because there is no path.

com.docker.network.bridge.name gives the bridge a stable, predictable interface name, which matters more than it looks.

Filtering in the right chain

When a port must be published but only to some sources, the rule goes in DOCKER-USER, and it is inserted rather than appended so it runs before Docker’s accepts.

MGMT_NET=192.0.2.0/24

# Let established traffic back through first, or you break every reply.
iptables -I DOCKER-USER 1 -m conntrack --ctstate ESTABLISHED,RELATED -j ACCEPT

# Allow the management network to reach the exporter's ORIGINAL destination
# port. After DNAT the port is the container's, so match on conntrack's
# record of the original packet instead.
iptables -I DOCKER-USER 2 -s "$MGMT_NET" \
-m conntrack --ctorigdstport 9100 -j ACCEPT

# Drop everything else aimed at that port.
iptables -I DOCKER-USER 3 -m conntrack --ctorigdstport 9100 -j DROP

The conntrack --ctorigdstport match is the detail that trips everyone up and it follows directly from the traversal order. By the time a packet reaches DOCKER-USER it has already been DNATed, so --dport 9100 matches the container’s port, which for 9100:9100 happens to be the same and for 8443:443 is not. --ctorigdstport matches what the client actually dialled. The Docker documentation notes that conntrack matching costs performance; on a management port that is irrelevant, on a high-traffic edge port it is worth measuring.

Rules in DOCKER-USER do not survive a reboot on their own. Persist them the way your distribution does — iptables-persistent, an nftables.service include, or a systemd unit ordered After=docker.service so the chain exists when the rules load.

The management plane

Management access is the plane people segment last and should segment first, because everything else on the host depends on it.

• SSH on the management address only. ListenAddress 192.0.2.10 in sshd_config, not a firewall rule — the socket never binds the public interface at all, so there is nothing to filter.
• Never expose the daemon on TCP. -H tcp://0.0.0.0:2375 is unauthenticated root on the host. If you need remote Docker, use DOCKER_HOST=ssh://user@host, which reuses SSH auth and needs no new listener.
• Never mount the socket into a container that does not have a compelling, reviewed reason. A container with /var/run/docker.sock can start a privileged container that mounts /. Read-only does not help meaningfully — reading is enough to enumerate every secret mount and environment variable on the host.
• Exporters bind the management address. node_exporter, cAdvisor and the daemon’s own metrics endpoint expose an inventory of your entire estate to anyone who can reach them.

Verification that can fail

Everything above is a claim until something outside the host confirms it.

$ ss -lntp | grep -E 'docker-proxy|dockerd|sshd'

LISTEN 0 4096  203.0.113.10:443   0.0.0.0:*  users:(("docker-proxy",pid=2841,fd=4))
LISTEN 0 4096  192.0.2.10:9100    0.0.0.0:*  users:(("docker-proxy",pid=2903,fd=4))
LISTEN 0 4096    192.0.2.10:22    0.0.0.0:*  users:(("sshd",pid=1102,fd=3))
LISTEN 0 4096       0.0.0.0:5432  0.0.0.0:*  users:(("docker-proxy",pid=2977,fd=4))

The last line is the finding. 0.0.0.0:5432 means every interface, including the public one, regardless of what the firewall says. Grepping that output for a bind address of 0.0.0.0 is a genuinely useful nightly check, and one of the few that catches a mistake before an attacker does.

TARGET=203.0.113.10

# Expected: 443 open, everything else refused or filtered
for port in 22 443 5432 9100 2375; do
timeout 3 bash -c "echo > /dev/tcp/$TARGET/$port" 2>/dev/null \
  && echo "$port OPEN" || echo "$port closed"
done

Anything but 443 OPEN and the rest closed is a finding. Note this must run from another host: from the host itself every one of them will appear open, which is precisely how the database in the callout above stayed exposed for four months.

Knowledge check