nginx as a Docker reverse proxy · Docker & Containers

nginx is the workhorse reverse proxy: stable, fast, and older than containers by a decade. That last part is the one that matters here. nginx has no idea Docker exists. It reads a text file, resolves the names in it, and serves. Every operational surprise in this lesson comes from that single fact.

The configuration below is the one to start from. The sections after it are the four things it does that a three-line example does not, and the one failure that will find you anyway.

Where nginx runs, and what changes

	nginx in a container	nginx on the host
Reaches backends by	Compose service name on a shared network	published port on `127.0.0.1`
Backend sees client as	the nginx container’s address	the bridge gateway, e.g. `172.17.0.1`
Certificate lives on	a mounted volume or bind mount	the host filesystem
Restarted by	`docker compose restart proxy`	`systemctl reload nginx`
Fails when	the backend container is recreated	the published port moves

Both are legitimate. In a container is the common choice because the proxy then travels with the stack; on the host is the choice when nginx fronts things that are not containers as well. This lesson assumes the container case and flags where the host case differs.

A configuration that survives production

Configuration change/etc/nginx/conf.d/app.conf— Annotated. Every directive here earns its place; the reasons are below.

# Docker's embedded DNS. Only reachable from a container attached to a
# user-defined network - it does not exist on the host.
resolver 127.0.0.11 valid=10s ipv6=off;

upstream app_backend {
  # 'zone' puts the upstream state in shared memory, which is what
  # makes 'resolve' and runtime state possible. Open source since 1.9.0.
  zone app_backend 64k;

  # 'resolve' re-resolves the name in the background on the TTL above,
  # instead of once at startup. Open source since nginx 1.27.3.
  server web:8080 resolve max_fails=3 fail_timeout=10s;

  # Reuse upstream connections. Requires HTTP/1.1 and an empty
  # Connection header on the proxy_pass side, set below.
  keepalive 16;
}

server {
  listen 443 ssl;
  http2 on;
  server_name app.example.com;

  ssl_certificate     /etc/nginx/certs/app.example.com/fullchain.pem;
  ssl_certificate_key /etc/nginx/certs/app.example.com/privkey.pem;
  ssl_protocols       TLSv1.2 TLSv1.3;
  ssl_session_cache   shared:SSL:10m;

  # Trust the forwarded address only from the addresses that are
  # genuinely in front of us. See the client-identity lesson.
  set_real_ip_from 192.0.2.0/24;
  real_ip_header   X-Forwarded-For;
  real_ip_recursive on;

  # Bound the time nginx will wait on the client, separately from
  # the time it will wait on the backend.
  client_max_body_size 20m;
  client_body_timeout  30s;

  location / {
      proxy_pass http://app_backend;

      proxy_http_version 1.1;
      proxy_set_header Connection "";

      proxy_set_header Host              $host;
      proxy_set_header X-Real-IP         $remote_addr;
      proxy_set_header X-Forwarded-For   $proxy_add_x_forwarded_for;
      proxy_set_header X-Forwarded-Proto $scheme;
      proxy_set_header X-Forwarded-Host  $host;

      proxy_connect_timeout 5s;
      proxy_send_timeout    30s;
      proxy_read_timeout    30s;

      # Do not silently replay a POST against a second backend.
      proxy_next_upstream error timeout http_502 http_503;
      proxy_next_upstream_tries 2;
  }

  # The proxy's own liveness, not the backend's. Answering this
  # proves nginx is up and says nothing about the application.
  location = /nginx-health {
      access_log off;
      return 200 "nginx ok\n";
  }
}

server {
  listen 80;
  server_name app.example.com;
  return 308 https://$host$request_uri;
}

Three of those lines are the difference between this and the example everyone copies: resolve on the server line, the explicit timeouts, and proxy_next_upstream_tries 2. Each has a section below.

The failure that defines nginx-in-Docker

The proxy 502s after every third deploy and comes back on restart

You redeploy the application. Every request to it returns 502 Bad Gateway. The application container is running, healthy, and docker compose exec proxy curl http://web:8080/ from inside the proxy container works perfectly. Restarting nginx fixes it. A week later it happens again.

The first diagnosis — “the app is broken” — is wrong, and so is the second one, “DNS is broken”, because DNS demonstrably works when you test it by hand.

What is broken is when nginx asked. With a plain hostname in proxy_pass or on an upstream server line, nginx resolves the name once, at configuration load, and caches the address for the life of the configuration. docker compose up -d destroys the old container and creates a new one, which gets a new address on the bridge. nginx keeps connecting to the address of a container that no longer exists, gets ECONNREFUSED, and returns 502. Your manual curl resolves fresh every time, which is exactly why it works.

It is intermittent because it only bites when the new container happens to land on a different address — which depends on what else was created and destroyed in between.

The fix is to make nginx re-resolve. There are two forms, and which one you can use depends on your nginx version.

Configuration changetwo ways to re-resolve— Prefer the first. Use the second only on nginx older than 1.27.3.

# 1. nginx 1.27.3 and later: 'resolve' on the upstream server line.
#    Keeps load balancing, keepalive and passive health checks.
resolver 127.0.0.11 valid=10s ipv6=off;

upstream app_backend {
  zone app_backend 64k;
  server web:8080 resolve;
  keepalive 16;
}

# 2. Older nginx: put the name in a variable. Using any variable in
#    proxy_pass defers resolution to request time.
#    Cost: no upstream block, so no keepalive and no passive health
#    checking. A per-request DNS lookup instead.
resolver 127.0.0.11 valid=10s ipv6=off;

location / {
  set $backend "web:8080";
  proxy_pass http://$backend;
}

Form 2 has a second, quieter cost. Because the address is a variable, nginx passes the request URI through unchanged rather than performing the location-prefix substitution — which is what you want for location /, and a behaviour change you must account for anywhere you were relying on proxy_pass rewriting the path.

Why 127.0.0.11 and why it is not in resolv.conf on the host

127.0.0.11 is the address of Docker’s embedded DNS server. It is not a real host; it is a listener the daemon sets up inside each container’s network namespace for containers attached to a user-defined network. The daemon writes it into that container’s /etc/resolv.conf, answers queries for container and service names from the network’s own record set, and forwards everything else to the host’s configured resolvers.

Two consequences that cost people time:

It only exists inside the namespace. dig @127.0.0.11 web on the host returns nothing, because on the host 127.0.0.11 is just loopback with nothing listening. Run the query with docker compose exec proxy and it works.
Containers on the default bridge network do not get it. They receive a copy of the host’s /etc/resolv.conf instead, which is why service-name resolution “does not work” on the default bridge. Any Compose project creates a user-defined network, so you normally have it without asking.

Confirm which case you are in before you debug anything else:

Read-only / Safeprove the resolver— What the proxy container actually resolves, and to what.

PROXY=proxy
BACKEND=web

# Is the embedded DNS present in this container at all?
docker compose exec "$PROXY" cat /etc/resolv.conf

# What does the name resolve to right now?
docker compose exec "$PROXY" getent hosts "$BACKEND"

# What address is the container actually on?
docker inspect -f '{{range .NetworkSettings.Networks}}{{.IPAddress}} {{end}}' "$BACKEND"

If the last two disagree, you have found the 502.

Health checking upstreams: what nginx OSS will not do

This is the most commonly misstated thing about nginx, and the correction is worth stating plainly.

Active health checking — nginx polling a backend on a timer, independently of client traffic — is a commercial (nginx Plus) feature. Open-source nginx has passive health checking only: it learns a backend is bad by failing real client requests against it.

Configuration changepassive health checking— Defaults are max_fails=1 fail_timeout=10s, which is usually too twitchy.

upstream app_backend {
  zone app_backend 64k;
  resolver 127.0.0.11 valid=10s;

  # 3 failures inside 10s take this server out for 10s.
  server web-a:8080 resolve max_fails=3 fail_timeout=10s;
  server web-b:8080 resolve max_fails=3 fail_timeout=10s;

  # A server marked 'backup' only receives traffic when every
  # primary is marked down.
  server web-standby:8080 resolve backup;

  keepalive 16;
}

The practical meaning: the first max_fails requests after a backend dies are served an error to a real user. Passive checking cannot be otherwise — the failure is the detection. Sizing max_fails is a trade between how many users see the error and how easily a single slow request evicts a healthy backend.

If you need genuine active checks without a subscription, the answer is not an nginx directive. It is either a different proxy — HAProxy and Traefik both do active checks in their open-source builds — or an external agent that rewrites the upstream file and reloads nginx.

502 versus 504, and the timeout that causes each

These two status codes are nginx telling you two very different things, and reading them correctly saves the first twenty minutes of an incident.

Response	nginx saw	Usual cause
`502 Bad Gateway`	connection refused, reset, or an empty/invalid response	backend down, backend crashed mid-response, stale upstream address, keepalive race
`504 Gateway Time-out`	nothing at all, until a timeout fired	backend alive but too slow, or blocked on its own dependency
`503 Service Unavailable`	no upstream server available to try	every backend marked down by passive checks, or `limit_req` rejected it

The timeout defaults are all 60s and all three measure different things:

proxy_connect_timeout (60s) — establishing the TCP connection. Should be small; a backend on the same bridge either accepts in milliseconds or is not there. 5s is generous.
proxy_send_timeout (60s) — between two successive write operations while sending the request body.
proxy_read_timeout (60s) — between two successive read operations, not the total response time. A backend that streams a byte every 20 seconds for an hour never trips a 30-second read timeout. This is the directive people believe is a total-response deadline; it is not.

Sporadic 502s on a low-traffic site with keepalive enabled

A handful of requests a day fail with 502. There is no pattern in the URL, no error in the application log at that timestamp, and the rate is far too low to be capacity. Retrying always works.

This is the keepalive race, and it is a timeout mismatch rather than a fault. nginx holds an idle upstream connection open. The application’s own keepalive timeout is shorter than nginx’s, so the application closes that connection first. nginx picks the same connection off its pool microseconds later and writes a request into a socket the other end has already closed — an empty response, which is a 502.

The fix is arithmetic, not a bug fix: the backend’s idle keepalive timeout must be longer than the proxy’s. Node’s default server.keepAliveTimeout of 5 seconds against an nginx pool is the classic instance. Raise the application’s value above nginx’s keepalive_timeout, or set proxy_next_upstream error timeout and accept a retry — but never on a non-idempotent request, which is what proxy_next_upstream_tries 2 and the absence of non_idempotent in the list are protecting.

Draining a backend on deploy

nginx OSS has no runtime API, so draining means editing the upstream and reloading. The reload is the graceful part.

Service impact possibledrain one backend— Mark it down, reload, wait for in-flight work, then redeploy it.

set -euo pipefail
UPSTREAM=/etc/nginx/conf.d/upstream.conf
TARGET=web-a

# 1. Mark it down. 'down' stops new requests; it does not cut existing ones.
sudo sed -i "s|^\(\s*server $TARGET:8080\)[^;]*;|\1 down;|" "$UPSTREAM"

# 2. Validate, then reload. Old workers finish their current requests.
sudo nginx -t && sudo nginx -s reload

# 3. Prove it is idle before you touch it.
docker compose exec "$TARGET" sh -c 'ss -tn state established | wc -l'

A reload is genuinely graceful: the master process starts new worker processes with the new configuration, stops sending them new connections, and lets the old workers finish what they already accepted before exiting. worker_shutdown_timeout bounds how long it will wait for a worker that will not finish — the default is no limit, which means one WebSocket can keep an old worker alive indefinitely. Set it if you have long-lived connections.

TLS termination and where the certificate lives

nginx does not speak ACME. It reads two files. Something else has to put them there and reload nginx when they change — certbot, a cert-manager-style job, or a sidecar container.

Configuration changecert placement for a containerised nginx— Read-only mount, and a reload the renewal hook can call.

cat <<'YAML' > compose.override.yaml
services:
proxy:
  volumes:
    - /etc/letsencrypt:/etc/nginx/certs:ro
YAML

# The renewal hook. certbot runs this after a successful renewal.
cat <<'SH' | sudo tee /etc/letsencrypt/renewal-hooks/deploy/reload-nginx.sh
#!/usr/bin/env bash
set -euo pipefail
docker compose -f /srv/app/compose.yaml exec -T proxy nginx -t
docker compose -f /srv/app/compose.yaml exec -T proxy nginx -s reload
SH
sudo chmod 0755 /etc/letsencrypt/renewal-hooks/deploy/reload-nginx.sh

The :ro matters. A proxy that can write to your certificate store is a proxy that can replace your certificate if it is compromised, and it has no reason to.

Verification that can fail

Read-only / Safewhat is actually being served— Compare the certificate on the wire with the one on disk. A mismatch means a missed reload.

$ openssl s_client -connect app.example.com:443 -servername app.example.com </dev/null 2>/dev/null | openssl x509 -noout -dates -fingerprint -sha256

notBefore=Jul 22 04:11:07 2026 GMT
notAfter=Oct 20 04:11:06 2026 GMT
sha256 Fingerprint=A1:B2:C3:D4:E5:F6:07:18:29:3A:4B:5C:6D:7E:8F:90:A1:B2:C3:D4:E5:F6:07:18:29:3A:4B:5C:6D:7E:8F:90

Illustrative output

Read-only / Safefour checks that distinguish working from broken— Each of these can fail. 'It returned something' is not verification.

HOST=app.example.com
PROXY=proxy
BACKEND=web

# 1. Configuration parses. Non-zero exit is the answer.
docker compose exec -T "$PROXY" nginx -t

# 2. Fingerprint on disk matches fingerprint on the wire.
docker compose exec -T "$PROXY" openssl x509 -in /etc/nginx/certs/live/"$HOST"/fullchain.pem -noout -fingerprint -sha256

# 3. The name the proxy has cached resolves to the container that exists.
docker compose exec -T "$PROXY" getent hosts "$BACKEND"
docker inspect -f '{{range .NetworkSettings.Networks}}{{.IPAddress}} {{end}}' "$BACKEND"

# 4. The backend sees a forwarded address, not the proxy's own.
curl -sS -o /dev/null -w '%{http_code}\n' "https://$HOST/"
docker compose logs --since 30s --no-log-prefix "$BACKEND" | tail -n 5

Check 4 is the one people skip. If the backend’s access log shows the proxy’s container address on every line rather than a real client address, the forwarded headers are not being read — which means your rate limiting, your allowlists and your audit trail are all keyed on a constant.

When nginx is the right choice

	nginx	HAProxy	Traefik	Caddy
Discovery	static file	static file	Docker labels	static file
Active health checks	commercial only	yes	yes	yes
ACME built in	no	no	yes	yes
L4 (TCP)	`stream` module	first-class	limited	`layer4` plugin
Runtime state change without reload	no	yes (Runtime API)	n/a	yes (admin API)
Also a web server / static files	yes	no	no	yes

Choose nginx when the backend set is stable, when you need it to serve static content or do caching as well as proxy, or when the team already reads nginx configuration fluently — which is a real operational property and not a soft one. Choose against nginx when containers are created and destroyed frequently, because that is the case its static model fits worst.

What if you reload nginx while a 40-minute file upload is in flight?

The upload completes. That is the good news and it is also the problem.

A reload starts new workers with the new configuration and asks the old workers to shut down gracefully — they stop accepting new connections but keep serving the ones they already have. The worker handling that upload stays alive, holding the old configuration, until the upload finishes.

Three consequences worth knowing before you rely on a reload:

Old workers hold old certificates and old upstreams. For as long as they live, some of your traffic is being served by the configuration you thought you had replaced. During a certificate rotation this is a window in which two different certificates are in use.
Repeated reloads accumulate workers. nginx -s reload in a deploy loop against long-lived connections leaves a pile of shutting-down workers and multiplies memory use. ps inside the container shows them as worker process is shutting down.
worker_shutdown_timeout is unset by default, meaning no limit. Setting it to something like 30s converts “waits forever” into “forcibly closes stragglers”, which is a decision you should make deliberately rather than inherit.

The undo, if a reload went wrong, is to put the previous configuration back and reload again — nginx -t first. There is no state to unwind, because nginx keeps none.

Knowledge check

Where next

HAProxy is the next lesson, and it is the same job with the opposite trade: a runtime API and real active health checks, in exchange for a configuration language that assumes you already know load balancing.