tmpfs mounts — ephemeral, fast storage · Docker & Containers

tmpfs is a RAM-backed filesystem. Files written to tmpfs live in memory, and when the container stops the tmpfs is gone — not unmounted, gone. There is nothing on disk to clean up because there was never anything on disk.

That is the whole appeal, and it is also where both of its failure modes come from.

When tmpfs is right

Performance-critical scratch space. Build intermediates, caches, compilation output, transcoding buffers. Anything that benefits from fast I/O and does not need to survive.
A writable path in a read-only container. The main modern use. --read-only plus a tmpfs at /tmp and /run is the standard hardening pattern.
Short-lived secret material. A fetched token that should not land in a layer or a volume — with the swap caveat below.

When tmpfs is wrong

Durable state. A database on tmpfs loses everything when the container stops. This is not a warning about crashes; it is the designed behaviour.
Anything larger than you have modelled. tmpfs pages are container memory, as the next section explains.
Sharing. tmpfs mounts cannot be shared between containers, and they are only available on Linux.

The failure mode: tmpfs is charged to your memory limit

A full tmpfs does not return ENOSPC. It gets the container OOM-killed.

The reasonable expectation is that a filesystem which runs out of room returns ENOSPC, the application handles it or crashes with a clear error, and you read the message and resize.

That happens only if you sized the tmpfs. If you did not, the size defaults to 50% of the host’s RAM — the host’s, not the container’s — and the container hits its cgroup memory limit long before the filesystem hits its own.

tmpfs pages are page-cache pages that cannot be reclaimed, and cgroup v2 charges them to the cgroup that faulted them in. So a container with --memory 512m and --tmpfs /tmp on a 64 GB host has a /tmp that thinks it can hold 32 GB and a memory limit that kills it at 512 MB.

The symptom is the container disappearing with exit code 137 and OOMKilled: true. There is nothing in the application log, because SIGKILL from the OOM killer leaves no time to write one.

The first diagnosis is almost always wrong. Exit 137 says “memory leak”, so people raise --memory, which raises the ceiling on how much the process can write to /tmp before dying and delays the next occurrence by exactly as much. The application’s heap was never the problem — the log rotation, the upload buffer, or the temp file the library forgot to delete was.

The tell is shmem in the container’s memory.stat. A leaking heap grows anon. A filling tmpfs grows shmem. They are different numbers and they mean different things.

Reading the accounting yourself

Every container gets its own cgroup v2 directory. On a systemd host it is /sys/fs/cgroup/system.slice/docker-$CID.scope/, where $CID is the full 64-character container ID, and four files there answer the question completely.

memory.stat breaks the charge down. anon is the process heap and stack. file is reclaimable page cache — the kernel drops it under pressure and it never causes an OOM kill on its own. shmem is tmpfs and shared memory, and it is the important one: those pages are page cache that cannot be dropped, only swapped, so under a memory limit with no swap they are as unreclaimable as the heap.
memory.current is the total currently charged.
memory.max is the limit --memory set.
memory.events counts the times the limit was hit (max) and the times something was killed for it (oom_kill). That last counter is the difference between “this container is under pressure” and “this container has been dying and restarting”, and neither docker stats nor docker ps tells you it.

docker stats shows the total, which is the number that makes a tmpfs problem look like a heap problem. The breakdown is one directory away.

Read-only / Safetmpfs accounting— Split a container's memory use into heap, page cache and tmpfs, and show its limit.

CONTAINER=web
CID=$(docker inspect --format '{{.Id}}' "$CONTAINER")
CG=/sys/fs/cgroup/system.slice/docker-$CID.scope

grep -E '^(anon|file|shmem) ' "$CG/memory.stat"
echo "current: $(cat "$CG/memory.current")"
echo "max:     $(cat "$CG/memory.max")"
echo '--- has it been killed before? ---'
cat "$CG/memory.events"

Read-only / Safea healthy container— Real output from a running container with a 512 MiB limit. shmem 0 means nothing is in tmpfs; oom_kill 0 means it has never been killed.

$ grep -E '^(anon|file|shmem) ' $CG/memory.stat; cat $CG/memory.current $CG/memory.max; cat $CG/memory.events

anon 31817728
file 230162432
shmem 0
current: 263929856
max: 536870912
low 0
high 0
max 0
oom 0
oom_kill 0
oom_group_kill 0

Read that as: 30 MB of heap, 230 MB of page cache the kernel can reclaim under pressure, nothing in tmpfs, 251 MB charged of a 512 MiB limit, never killed. If shmem were 400 MB and oom_kill were 3, you would have your answer in one command.

Sizing

Always size a tmpfs. There is no situation where inheriting half the host’s RAM is what you meant.

# Sized, and no smaller than the container can tolerate losing
docker run -d --name build \
  --memory 2g \
  --tmpfs /tmp:rw,noexec,nosuid,nodev,size=512m,mode=1777 \
  myorg/build-tools:1

Budget it explicitly: size plus the application’s expected heap must fit inside --memory, with headroom. A 2 GB limit with a 512 MB tmpfs leaves the process 1.5 GB, and a process that needs 1.4 GB has 100 MB of margin for everything else in the container.

In Compose, note that tmpfs: and the tmpfs: key under a long-form volume entry are different things:

services:
  build:
    image: myorg/build-tools:1
    mem_limit: 2g
    tmpfs:
      - /tmp:rw,noexec,nosuid,nodev,size=512m

Beyond the limit, writes fail with ENOSPC — the correct, diagnosable failure. That is what sizing buys you: a full filesystem instead of a dead container.

`--tmpfs` versus `--mount type=tmpfs`

They are not interchangeable, and the gap runs the opposite way from bind mounts.

	`--tmpfs`	`--mount type=tmpfs`
Size	`size=512m`	`tmpfs-size=536870912` (bytes)
Mode	`mode=1777`	`tmpfs-mode=1777` (default `1777`)
`noexec`, `nosuid`, `nodev`	Supported	Not available
`uid`, `gid`, `nr_inodes`	Supported	Not available
Swarm services	Not supported	Supported

So the hardened form and the Swarm-compatible form are mutually exclusive on the CLI. If you are hardening a docker run or a Compose deployment, use --tmpfs and set noexec,nosuid,nodev explicitly — they are the options that stop a writable world-mode directory from being a place to drop and run a binary.

The read-only root pattern

This is where tmpfs earns its place in production rather than in builds.

Read-only / Saferead-only root with scratch— The container filesystem is immutable; the only writable paths are the two tmpfs mounts you chose.

docker run -d --name api --read-only --memory 1g --tmpfs /tmp:rw,noexec,nosuid,nodev,size=64m --tmpfs /run:rw,noexec,nosuid,nodev,size=16m,mode=0755 --security-opt no-new-privileges myorg/api:1.4.0

--read-only alone breaks most images, because something writes a PID file, a socket, or a temp file during startup. The tmpfs mounts give those writes somewhere to go without giving anything a durable foothold in the container filesystem. noexec on both means a downloaded payload cannot be executed from the one writable place in the container.

Find out what actually needs to be writable rather than guessing:

Read-only / Safewhat does it write— Run the container normally, let it start, then ask what changed in the writable layer.

CONTAINER=api
docker diff "$CONTAINER"

docker diff lists every path added (A), changed (C) or deleted (D) in the writable layer since start. Each A or C outside a volume is a path that needs a tmpfs before --read-only will work.

tmpfs is not a guarantee that data never touches disk

“tmpfs is never written to disk” is repeated everywhere and it is not true. tmpfs pages are swappable. On a host with swap enabled and under memory pressure, the kernel can and will write tmpfs pages to the swap device — including the token you put there specifically to keep it off disk.

If tmpfs is part of a compliance story, one of these has to be true:

The host has no swap. Simplest, and common on container hosts for other reasons.
Swap is encrypted. A LUKS device with a random key generated at boot means the swapped pages are unrecoverable after a reboot, which satisfies most “not written to durable storage” requirements.
The container cannot swap. Setting --memory-swap equal to --memory gives the container no swap allowance at all. Note that this makes hitting the limit an immediate OOM kill instead of a slowdown — which, for a secret-handling container, is the trade you want.

Verify rather than assume: swapon --show on the host is one line, and an empty result is the answer you are looking for.

What happens when a sized tmpfs fills up?

The write fails with ENOSPC — No space left on device — and the container keeps running. This is the good outcome, and it is the one you bought by setting size=.

What happens next depends entirely on the application. A build tool exits with a clear error. A logging library often catches the exception and continues silently, so the first symptom is missing log lines rather than an error. A database on tmpfs, which should not exist, can corrupt exactly as it would on a full disk.

Diagnosing it from outside is two commands:

docker exec "$CONTAINER" df -h /tmp shows 100% on a tmpfs line, which distinguishes it immediately from a full host disk — df on the host will look fine.

Freeing space is just deleting files: docker exec "$CONTAINER" find /tmp -type f -delete reclaims it instantly, and the memory returns to the cgroup as it goes. Unlike a full overlay upper layer, there is nothing to compact and no restart required.

You cannot resize a tmpfs mount on a running container. size= is fixed at container creation, so a permanent fix means recreating the container with a larger value — and re-checking that the new value still fits inside --memory.

Verification that can fail

Read-only / Safeverify the mount— Confirms it is really tmpfs, really sized, and really has the security options you asked for.

CONTAINER=api

# Type and size as the kernel recorded them, not as you intended
docker exec "$CONTAINER" df -hT /tmp

# The mount options actually in force. Expect noexec, nosuid, nodev, size.
docker exec "$CONTAINER" grep ' /tmp ' /proc/self/mounts

# noexec really applies: this must FAIL
docker exec "$CONTAINER" sh -c 'cp /bin/true /tmp/x && chmod +x /tmp/x && /tmp/x' && echo 'EXECUTABLE - noexec is missing' || echo 'noexec enforced'

The last check is the one worth automating. noexec silently disappears the moment someone converts the flag to --mount type=tmpfs, which does not support it, and nothing about the container looks different afterwards.

How tmpfs differs from volumes

Feature	tmpfs	Volume
Backing	RAM (and swap, unless disabled)	Host disk
Counts against `--memory`	Yes	No
Durability	None; gone with the container	Survives container removal
Survives reboot	No	Yes
Shared between containers	No	Yes
Resize while running	No	N/A
Failure when full	`ENOSPC`, or OOM kill if unsized	`ENOSPC`