Docker build secrets and runtime secrets

There are two distinct kinds of secrets in a Docker deployment:

• Build-time secrets: credentials needed during docker build to fetch dependencies, sign artefacts, or authenticate to a private registry.
• Runtime secrets: credentials the running container needs to talk to databases, brokers, or external services.

They fail differently. A build secret that leaks is baked into an artefact you distribute and cannot recall. A runtime secret that leaks is readable while the container runs. Both need fixing; only one of them follows the image to every mirror in the world.

Build-time secrets

BuildKit mounts the secret into a single RUN step as a file that exists only for the duration of that step. It is never a layer.

# syntax=docker/dockerfile:1
FROM node:22-slim

RUN --mount=type=secret,id=npm_token,required=true \
    npm config set //registry.example.com/:_authToken="$(cat /run/secrets/npm_token)" \
 && npm ci --omit=dev \
 && npm config delete //registry.example.com/:_authToken

docker buildx build --secret id=npm_token,src=./npm_token.txt -t myorg/api:1.4.0 .

The options, and the two defaults that bite

Two of those defaults cause most of the confusion.

required defaults to false. Forget --secret on the build command and the mount is simply empty. cat /run/secrets/npm_token produces nothing, npm falls back to the public registry, and the build either succeeds with the wrong dependency or fails with an authentication error that sends you hunting for a token problem that does not exist. Set required=true on every secret mount you actually depend on; it turns a confusing build into a clear one.

mode is 0400 and uid is 0. The secret is readable by root only. If your Dockerfile has a USER app before that RUN, the step runs as a non-root user and gets Permission denied on a file it can see. Pass uid=1001 to match.

On the CLI side, --secret takes three forms:

# From a file
docker buildx build --secret id=npm_token,src=./npm_token.txt .

# From an environment variable in the shell running the build
docker buildx build --secret id=npm_token,env=NPM_TOKEN .

# Shorthand: id only, taken from the environment variable of that name
NPM_TOKEN=REPLACE_ME docker buildx build --secret id=NPM_TOKEN .

The env= form is the one to use in CI, where the credential arrives as a masked pipeline variable and writing it to a file just creates something to forget to delete.

SSH, for private git dependencies

When the credential is an SSH key rather than a token, forwarding the agent is better than mounting the key, because the private key never enters the build at all:

# syntax=docker/dockerfile:1
RUN --mount=type=ssh \
    mkdir -p -m 0700 ~/.ssh \
 && ssh-keyscan github.com >> ~/.ssh/known_hosts \
 && git clone git@github.com:org/private-repo.git /src

docker buildx build --ssh default -t myorg/api:1.4.0 .

BuildKit forwards a socket to the agent. Signing requests cross it; key material does not.

Runtime secrets

The delivery mechanism is a file, mounted read-only, that the application reads at startup.

services:
  api:
    image: myorg/api:1.4.0
    user: "1001:1001"
    secrets:
      - source: db_password
        target: db_password
        mode: 0400
        uid: "1001"
    environment:
      DB_PASSWORD_FILE: /run/secrets/db_password

secrets:
  db_password:
    file: ./secrets/db_password.txt

import os

with open(os.environ["DB_PASSWORD_FILE"], encoding="utf-8") as fh:
    password = fh.read().strip()

Two details in that Compose file are doing real work.

DB_PASSWORD_FILE is a path, not a secret. Passing the path in an environment variable is fine and is the convention the official postgres, mysql and mariadb images use with their *_FILE variables. The path leaks in docker inspect; the value does not.

.strip() is not optional. A secret file written with a text editor ends in a newline, and the newline is part of what read() returns. This produces an authentication failure against a credential that is visibly correct in every log and every copy-paste comparison, and it is responsible for more wasted hours than any other single thing in this lesson. Generate secret files with printf '%s' "$VALUE" > file, and strip on read anyway.

There is also an environment: source, which reads the value from an environment variable of the process running docker compose and delivers it to the container as a file:

secrets:
  db_password:
    environment: DB_PASSWORD

This is the right shape for CI and for orchestrators that inject credentials as pipeline variables: the value is in the environment of the deploying process, which is short-lived, and reaches the container as a file.

Verification that can fail

Verify the artefact, not the intention. These commands are the point of the lesson.

IMAGE=myorg/api:1.4.0
NEEDLE=$(cat ./npm_token.txt)

echo '--- config and env ---'
docker image inspect "$IMAGE" --format '{{json .Config}}' | grep -cF "$NEEDLE"

echo '--- build history (ARG values land here) ---'
docker image history --no-trunc "$IMAGE" | grep -cF "$NEEDLE"

echo '--- every layer, unpacked ---'
docker image save "$IMAGE" | tar -xO 2>/dev/null | grep -acF "$NEEDLE"

$ ./scan-image-for-secret.sh

--- config and env ---
0
--- build history (ARG values land here) ---
0
--- every layer, unpacked ---
2

Three zeros is a pass. The output above is the exact shape of the common failure: the Dockerfile used a secret mount, so inspect and history are clean and every quick check passes — and the token is in two files in a layer, because npm config set wrote it to ~/.npmrc.

Anything non-zero means a rebuild plus a rotation, in that order of discovery and the reverse order of urgency: rotate first, because the image may already have been pushed.

The last command is deliberately blunt — it streams the whole image tarball through grep. It is slow on a large image and it catches the config-file case that the first two miss, which is the one that actually happens.

CONTAINER=api
SECRET=/run/secrets/db_password

# Present, and with the mode and owner you expect
docker exec "$CONTAINER" stat -c '%n %U:%G %a %s' "$SECRET"

# No trailing newline: the byte count should match the credential length exactly
docker exec "$CONTAINER" sh -c "wc -c < $SECRET"

# And it is NOT in the environment
docker inspect "$CONTAINER" --format '{{range .Config.Env}}{{println .}}{{end}}' | grep -iE 'password|secret|token' || echo 'environment clean'

A mode of 444 where you asked for 400 is the Compose limitation above, visible before it matters rather than after.

Knowledge check