Host hardening — the production baseline · Docker & Containers

Hardening is defence in depth, and the phrase gets used as if it meant “apply many controls”. It means something more specific: an attacker has to complete a sequence of steps, and each control makes one step of that sequence fail.

That framing is the difference between a checklist people apply and one they argue with. “Drop all capabilities” invites a debate. “Drop all capabilities, because CAP_SYS_ADMIN in a container is a mount syscall away from the host filesystem” ends it.

The sequence a control interrupts

Almost every container compromise follows the same shape:

flowchart LR
  A[Code execution<br/>in the container] --> B[Escalate to root<br/>inside the container]
  B --> C[Break out to<br/>the host]
  C --> D[Reach other<br/>containers or hosts]

The controls map onto it directly, and that mapping is the whole lesson.

Control	Step it breaks	The attack it stops
Non-root `USER` in the image	A to B	Attacker starts unprivileged; most local escalations need root in the namespace first
`--security-opt no-new-privileges=true`	A to B	A setuid binary inside the image cannot raise privileges
`--cap-drop=ALL`	B to C	Root in the container holds no capability that reaches the host
Default seccomp profile	B to C	Blocks the syscalls used by most kernel exploits
AppArmor or SELinux	B to C	Confines file and syscall access even for a container root
`--read-only` root filesystem	A to B	Attacker cannot write a payload, a cron file or a modified binary
`userns-remap` or rootless mode	C	Container root maps to an unprivileged host UID; a breakout lands as nobody
Not mounting `/var/run/docker.sock`	B to C	The socket is the daemon API, and the daemon is root
User-defined networks, `icc: false`	C to D	Lateral movement between containers
Host firewall	C to D	Lateral movement between hosts

Read down the “step it breaks” column. Three controls guard the A-to-B transition and four guard B-to-C, which is deliberate — those are the two transitions where a defence actually costs the attacker something.

Layer 1: the host

Configuration change/etc/sysctl.d/60-docker-hardening.conf— One file, written once. Writing several settings with separate redirections to the same path overwrites the file each time and leaves only the last line.

sudo tee /etc/sysctl.d/60-docker-hardening.conf >/dev/null <<'EOF'
# Hide kernel pointers from unprivileged users. Defeats the address leaks
# that kernel exploits use to defeat KASLR.
kernel.kptr_restrict = 1

# Only privileged users can read the kernel ring buffer. dmesg leaks
# addresses, driver versions and host details useful for targeting.
kernel.dmesg_restrict = 1

# Restrict ptrace to parent-child only. Blocks a compromised process from
# reading another process memory in the same namespace.
kernel.yama.ptrace_scope = 1

# Refuse to follow symlinks and hardlinks into files you do not own in
# world-writable sticky directories. Kills a whole class of /tmp races.
fs.protected_symlinks = 1
fs.protected_hardlinks = 1
fs.protected_fifos = 1
fs.protected_regular = 2

# Bound the number of user namespaces an unprivileged user can create.
# Zero would be stronger and breaks rootless Docker and userns-remap;
# a bound keeps the feature while limiting exploit surface.
user.max_user_namespaces = 15000
EOF

sudo sysctl --system

# Evidence: read the values back from the running kernel, not the file.
sysctl kernel.kptr_restrict kernel.dmesg_restrict kernel.yama.ptrace_scope

Three corrections to advice that circulates widely:

user_namespace.enable=1 is not a sysctl. It was a kernel boot parameter on RHEL 7, where user namespaces were disabled by default. On a current kernel it does nothing at all, and writing it into /etc/sysctl.d/ produces an error at boot that most people never see. The relevant modern knob is user.max_user_namespaces.
kernel.yama.ptrace_scope = 3 is not simply “more secure”. Level 3 disables ptrace entirely and cannot be lowered without a reboot. You lose gdb, strace, perf attach and any crash dumper, permanently, including during the incident where you need them. Level 1 stops cross-process reads and keeps debugging a parent’s own children. Choose 3 deliberately, not by reflex.
Blocking filesystem modules (cramfs, squashfs, udf) with install <module> /bin/true in /etc/modprobe.d/ is a genuine CIS control, but name the file after what it does. A file called disable-cracklib.conf containing a cramfs directive is how a control gets removed by someone tidying up. And check first: squashfs is used by snap, so blocking it on Ubuntu can break packages you rely on.

Generic hardening guides break Docker in two specific ways

Docker needs the host to do things a hardening guide will switch off, because the guide was written for a host that does not route packets or manage bridges.

net.ipv4.ip_forward = 0. This is on nearly every Linux hardening checklist, and it is correct for a host that is not a router. A Docker host is a router: every packet from a container to the outside world is forwarded from the bridge to the physical interface. Set it to 0 and containers can reach each other and nothing else — DNS fails, pulls fail, and the symptom is “the network is broken” with no error mentioning forwarding.

Docker sets ip_forward=1 when the daemon starts, so this often appears to work until a sysctl --system run by configuration management resets it under a running daemon. The failure then arrives hours after the change, which makes it very hard to attribute.

Firewall rules placed above Docker’s chains. Docker programs its own DOCKER and DOCKER-USER chains and inserts rules at daemon start and on every container start. A firewall managed independently — a ufw or firewalld reload, or a flushed iptables — removes them, and containers lose connectivity until the daemon restarts.

Neither is a reason to skip the control. DOCKER-USER exists precisely so you have a chain Docker will not overwrite; put your rules there. The point is that a hardening baseline for a Docker host is not a general Linux baseline plus Docker, and applying one blind produces an outage that looks like a network fault.

Layer 2: the daemon

Service impact possible/etc/docker/daemon.json— A baseline that is safe to apply to an existing host. userns-remap is deliberately absent; see the callout below.

{
"icc": false,
"live-restore": true,
"userland-proxy": false,
"no-new-privileges": true,
"log-driver": "json-file",
"log-opts": {
  "max-size": "50m",
  "max-file": "3"
},
"default-ulimits": {
  "nofile": { "Name": "nofile", "Hard": 8192, "Soft": 4096 }
},
"builder": {
  "gc": { "enabled": true, "defaultKeepStorage": "20GB" }
}
}

Two entries commonly found in baselines are absent here, and their absence is the point.

"features": {"buildkit": true} is obsolete. BuildKit has been the default builder since Engine 23, and the flag is no longer part of the documented features block — which today carries keys such as containerd-snapshotter and cdi. Leaving a stale key in daemon.json is harmless in itself and corrosive in aggregate: it teaches readers of the file that its contents are not maintained.
"selinux-enabled": true only applies where SELinux exists. On a Debian or Ubuntu host running AppArmor it is not the control you want, and a baseline that carries it unconditionally produces a FAIL in every audit on half your fleet. Confirm which mandatory access control system the host actually runs before configuring for one.

`userns-remap` is the strongest control here and the one that must not be applied in place

Remapping container UIDs into a subordinate range means container root is an unprivileged host UID — it turns a successful breakout into a session as nobody. It is the single most valuable daemon control available.

Enabling it on a running host is disruptive in ways that look like data loss:

Every existing image, container and volume disappears from docker ps -a and docker images. The daemon uses a separate storage root per remapped user. Nothing is deleted — it is in the old tree and returns if you disable remapping — but the first ten minutes are indistinguishable from having lost everything.
Bind mounts break. Files owned by host UID 1000 are not owned by the remapped container user, so applications get permission denied on paths that worked an hour ago.
--network host, --pid host and --privileged become incompatible, and containers using them fail to start.

Enable it when building a host, or plan a migration with volume re-ownership scripted and rehearsed. Do not add it to a baseline that is applied to existing production hosts by configuration management, which is how it usually gets deployed and how it usually causes an incident.

Rootless mode (dockerd-rootless-setuptool.sh install) achieves a similar goal more thoroughly, by running the daemon itself as a normal user. It also has real constraints — it is a per-user daemon, and several networking and storage features behave differently — so treat it as a deployment model rather than a setting to toggle.

Layer 3: the image

# Build stage omitted. The runtime stage is where the controls live.
FROM gcr.io/distroless/static-debian12:nonroot

# Non-root by UID, not by name. A numeric UID is what the kernel enforces,
# and it survives an image rebuild that reorders /etc/passwd.
USER 65532:65532

WORKDIR /app
COPY --from=build --chown=65532:65532 /app/binary /app/binary

# EXPOSE is documentation. It publishes nothing and enforces nothing.
EXPOSE 8080

ENTRYPOINT ["/app/binary"]

A distroless base has no shell, no package manager, no curl, no wget. That matters at a precise point in the attack chain: an attacker who has achieved code execution through a deserialisation bug or an SSRF usually needs to stage — download a second-stage payload, or spawn a shell to run a series of commands. In an image with no shell and no download tool, that step has to be done entirely within the exploited process, which is a significantly higher bar.

It also has a real operational cost that should be stated: you cannot docker exec into it to debug. The answer is docker debug on Docker Desktop, or attaching an ephemeral debug container that shares the target’s namespaces — not adding a shell back to the production image.

Layer 4: the container runtime

Configuration changea hardened docker run— Every flag maps to a row of the table at the top of this lesson.

docker run -d --name api \
--user 65532:65532 \
--read-only \
--cap-drop ALL \
--cap-add NET_BIND_SERVICE \
--security-opt no-new-privileges=true \
--pids-limit 256 \
--memory 512m \
--tmpfs /tmp:rw,noexec,nosuid,size=64m \
--publish 127.0.0.1:8080:8080 \
--network app-net \
registry.example.com/myorg/api:1.0.0

--cap-add NET_BIND_SERVICE only if the process binds below port 1024. If your application listens on 8080, drop this too — the container needs no capabilities at all, which is the goal.
--pids-limit 256 bounds the process table for this container. A fork bomb inside a container without it exhausts the host’s PID space and takes down every other container on the machine, including ones that are otherwise perfectly hardened. It is the cheapest control on this list and the most often omitted.
--tmpfs /tmp:rw,noexec,nosuid. A read-only root filesystem needs somewhere writable; noexec means the attacker cannot write a binary there and run it, which is exactly the staging step --read-only was meant to block.
--publish 127.0.0.1:8080:8080. Publishing without an address binds to 0.0.0.0 on every interface, and — this surprises people — Docker’s forwarding rules are consulted before most host firewall rules, so a published port can be reachable from the network even when the firewall appears to deny it. Bind to a specific address, always.

What `--cap-drop ALL` actually removes, and why the default set is dangerous

A container does not get “root or not root”. It gets a capability set — the kernel’s decomposition of root’s powers into around forty separate privileges. Docker’s default grants a subset of roughly fourteen, which is far fewer than full root and considerably more than any application needs.

Three from that default set are worth knowing by name, because each one has a documented path to the host:

CAP_DAC_OVERRIDE bypasses all file permission checks. Inside the container that means every file is readable and writable regardless of mode, so a bind-mounted host directory mounted for one file exposes all of it.
CAP_SETUID and CAP_SETGID allow arbitrary UID changes, which is what most in-container escalation actually needs.
CAP_NET_RAW allows raw sockets: ARP spoofing and traffic interception against every other container on the same bridge. This one is in the default set and is essentially never needed, and it is the reason --cap-drop ALL improves lateral-movement resistance even for a container running as an unprivileged user.

The capabilities not in Docker’s default set are the reason a default container is not trivially a host compromise: CAP_SYS_ADMIN (mount, and by extension the host filesystem), CAP_SYS_MODULE (load a kernel module — game over immediately), CAP_SYS_PTRACE, CAP_SYS_BOOT.

--privileged grants all of them and additionally disables seccomp, disables the AppArmor profile, and exposes host devices. It is not “more permissions”; it is the removal of the container security model. There is no configuration that makes a privileged container safe, which is why the correct response to “it does not work without --privileged” is to find the one capability or device it needs and grant that.

Read the effective set from inside a running container:

CONTAINER=api
docker exec "$CONTAINER" grep CapEff /proc/self/status

0000000000000000 is the goal, and it is achievable for most applications.

Proving it, rather than configuring it

Read-only / Safeaudit every running container against the baseline

$ docker ps -q | while read -r C; do
docker inspect --format '{{.Name}} priv={{.HostConfig.Privileged}} user={{.Config.User}} ro={{.HostConfig.ReadonlyRootfs}} pids={{.HostConfig.PidsLimit}} caps={{.HostConfig.CapDrop}}' "$C"
done

/api priv=false user=65532:65532 ro=true pids=256 caps=[ALL]
/legacy-batch priv=true user= ro=false pids=<nil> caps=[]
/adminer priv=false user= ro=false pids=<nil> caps=[]

Illustrative output

Two findings in three containers, and both are the shape you will actually meet: a --privileged container that someone added years ago to work around a device permission, and a container running as root with no limits because it was started with a docker run from a wiki page.

The audit is worth more than the configuration, because a baseline applies to containers created after it. Nothing retrofits.

Docker Bench for Security

Read-only / Safedocker-bench-security— The upstream invocation. Read the callout below before running it, because this is a very privileged container.

docker run --rm --net host --pid host --userns host \
--cap-add audit_control \
-v /etc:/etc:ro \
-v /usr/bin/containerd:/usr/bin/containerd:ro \
-v /usr/bin/runc:/usr/bin/runc:ro \
-v /usr/lib/systemd:/usr/lib/systemd:ro \
-v /var/lib:/var/lib:ro \
-v /var/run/docker.sock:/var/run/docker.sock:ro \
--label docker_bench_security \
docker/docker-bench-security

The `:ro` on the Docker socket provides no protection whatsoever

This is worth dwelling on, because the :ro reads as a safety measure and teaches a false intuition that people then apply elsewhere.

:ro makes the bind mount read-only. The Docker socket is not a file whose contents you read; it is a bidirectional Unix domain socket. Any process that can open it can write API requests into it — including POST /containers/create with Privileged: true and the host root filesystem bind-mounted.

So a container with the socket mounted :ro has exactly the same power as one with it mounted read-write, and both are equivalent to root on the host. The upstream project mounts it :ro by convention, not because it constrains anything.

Which means running Docker Bench is a real decision: you are granting a container host root, the host PID and network namespaces, and CAP_AUDIT_CONTROL, in exchange for an audit report. Reasonable ways to handle that:

Pin the image by digest and review it, rather than pulling a floating tag with these privileges.
Run it from the host directly — the project is a shell script, and cloning the repository and running sh docker-bench-security.sh avoids the container entirely while doing the same work.
Run it in a maintenance window, not continuously from a scheduler.

What happens if you apply this whole baseline at once to a running fleet?

Something breaks, and the value of knowing which thing in advance is high.

Ordered by how likely they are to bite, and each with its honest resolution — never “remove the control”:

--read-only breaks applications that write to unexpected paths. Web servers write PID files, Java writes to /tmp, nginx writes cache to /var/cache/nginx. Resolution: run it, read the permission errors, add a --tmpfs for each path. Fifteen minutes per service, once.
--cap-drop ALL breaks anything binding a low port or changing user at startup. Resolution: change the application to listen on a high port and publish it as 80 externally. This is better than the capability anyway.
no-new-privileges breaks images with a setuid entrypoint helper, notably older images using gosu or su-exec incorrectly. Resolution: use --user to start as the right UID directly, so nothing needs to drop privileges at runtime.
icc: false breaks containers that were relying on the default bridge. Resolution: put them on a user-defined network, which is what they should have used from the start and which also gives them DNS.
A --pids-limit that is too low kills a JVM or a Node process pool under load, and the error is confusing (fork: Resource temporarily unavailable). Resolution: measure with docker stats --format '{{.Name}} {{.PIDs}}' at peak and set roughly double.

Apply in this order: image and runtime controls per service, on a staging host, one service at a time. Then daemon controls, which need a restart. Then host sysctls, which need a reboot to be certain they survive one.

The failure mode to avoid is applying everything at once on a Friday, having three services break simultaneously, and rolling back the entire baseline because attribution is impossible under time pressure. That is how a hardening project is abandoned, and it is nearly always a sequencing failure rather than a technical one.

Write the baseline down with the attack each control interrupts. A control without a stated reason gets removed by the next person who finds it inconvenient.
Apply image and runtime controls first, per service, in staging. They are the cheapest to test and cover the two most valuable transitions.
Apply daemon controls next, and restart. Nothing in daemon.json is in effect until the daemon reads it.
Apply host sysctls last, and reboot once to confirm they persist. Check ip_forward is still 1 afterwards.
Audit running containers, not the configuration. Controls apply at creation; existing containers keep what they were born with.
Re-audit quarterly and keep the output. The diff between the baseline and the audit is the artefact that matters, not the baseline.
Never resolve a breakage by removing a control. Find the specific capability, path or device required and grant exactly that.

Why this matters

The organisations that get breached through a container almost always have the checklist. What they do not have is the mapping — so when a service would not start under --cap-drop ALL, somebody removed the flag for that one service, wrote a ticket, and the ticket aged out. Nobody could say what was lost, because the checklist said “drop capabilities” and not “this is what stops root in the container from becoming root on the host”.

Attaching each control to the attack step it interrupts does two things. It makes the cost of an exception explicit at the moment somebody wants one. And it means the audit output is readable as a threat statement rather than a compliance percentage: three containers running as root is not “94% compliant”, it is three places where step A-to-B of the sequence is already complete before the attacker arrives.