CI/CD for Docker images — build, scan, sign, push

A pipeline that builds, scans, signs and pushes is not a security control by virtue of containing those words. It is a control only if a failing stage can prevent the outcome that stage exists to prevent.

That reduces almost entirely to ordering, and ordering is where most real pipelines are wrong.

The ordering problem

flowchart LR
  subgraph Wrong[Common and wrong]
    W1[Build] --> W2[Push] --> W3[Scan] --> W4[Report]
  end
  subgraph Right[What a gate looks like]
    R1[Build] --> R2[Test] --> R3[Scan] --> R4[Publish]
    R3 -. fail .-> R5[Nothing published]
  end

The top pipeline is extremely common, usually because pushing first is the easiest way to give the scanner something to scan. It produces a red build, a Slack message, and a vulnerable image sitting in your registry with a production tag on it.

From that moment:

• Anything with a latest or environment tag can pull it. A node that restarts a container, an autoscaler, a docker compose pull — none of them consult your CI status.
• Deleting it does not undo it. Registries retain manifests by digest, and anything that already resolved the tag holds the digest.
• The pipeline reported a failure and the artefact shipped anyway. That is worse than no scan, because the dashboard now says the control exists.

The rule is a single sentence: a gate must run before the irreversible step, and publication is the irreversible step.

The conflict that makes this awkward

Here is the part the five-stage diagram hides, and it is a genuine engineering conflict rather than a matter of taste.

Cosign signs images that live in a registry. A signature is stored as an OCI artefact alongside the image, addressed by the image’s digest, in the same repository. There is no local image to sign — the thing being signed is a manifest at a registry address.

So “sign before push” is not achievable in the literal sense. The stage order everyone writes down is not implementable as written.

A pipeline that is actually a gate

The workflow below uses option 3 for clarity, with the build-once property made explicit, and notes where option 2 slots in.

# .github/workflows/docker.yml
name: Docker CI

on:
  push:
    branches: [main]

permissions:
  contents: read
  id-token: write        # required for cosign keyless signing
  packages: write

env:
  IMAGE: registry.example.com/myorg/myapp

jobs:
  build:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v4

      - name: Set up Buildx
        uses: docker/setup-buildx-action@v3

      # Build ONCE. --load puts the result in the local image store so the
      # test and scan stages operate on the exact bytes that will be pushed.
      - name: Build
        run: |
          docker buildx build \
            --load \
            --provenance=mode=max \
            --sbom=true \
            --tag "$IMAGE:ci-${{ github.sha }}" \
            .

      - name: Test inside the image
        run: |
          docker run --rm --network none \
            "$IMAGE:ci-${{ github.sha }}" \
            python -m pytest tests/

      # No docker socket mount. Trivy reads the local image store directly,
      # and mounting the socket into a scanner would give it host root.
      - name: Scan — this is the gate
        uses: aquasecurity/trivy-action@master
        with:
          image-ref: ${{ env.IMAGE }}:ci-${{ github.sha }}
          severity: HIGH,CRITICAL
          ignore-unfixed: true
          exit-code: '1'

      # Nothing below this line runs if the scan failed.
      - name: Log in to the registry
        run: |
          echo "${{ secrets.REGISTRY_PASSWORD }}" \
            | docker login -u "${{ secrets.REGISTRY_USER }}" \
                --password-stdin registry.example.com

      - name: Push and capture the digest
        id: push
        run: |
          docker push "$IMAGE:ci-${{ github.sha }}"
          DIGEST=$(docker buildx imagetools inspect \
            "$IMAGE:ci-${{ github.sha }}" --format '{{.Manifest.Digest}}')
          echo "digest=$DIGEST" >> "$GITHUB_OUTPUT"

      - uses: sigstore/cosign-installer@v3

      # Sign the DIGEST, never the tag. See the section below.
      - name: Sign
        run: |
          cosign sign --yes "$IMAGE@${{ steps.push.outputs.digest }}"

      # Only now does a tag anything deploys from start pointing at it.
      - name: Promote
        run: |
          docker buildx imagetools create \
            --tag "$IMAGE:1.0.0" \
            "$IMAGE@${{ steps.push.outputs.digest }}"

This is what the gate looks like when it does its job. The non-zero exit is the entire mechanism — without --exit-code 1, Trivy prints the same table and returns success, and every step below it runs anyway:

$ trivy image --severity HIGH,CRITICAL --ignore-unfixed --exit-code 1 \
registry.example.com/myorg/myapp:ci-4f2a91c
echo "exit=$?"

myapp (debian 12.7)
Total: 2 (HIGH: 1, CRITICAL: 1)

┌──────────────┬────────────────┬──────────┬────────┬───────────────┬───────────────┐
│   Library    │ Vulnerability  │ Severity │ Status │ Installed Ver │  Fixed Ver    │
├──────────────┼────────────────┼──────────┼────────┼───────────────┼───────────────┤
│ libexpat1    │ CVE-0000-00000 │ CRITICAL │ fixed  │ 2.5.0-1       │ 2.5.0-1+deb12 │
│ libssl3      │ CVE-0000-11111 │ HIGH     │ fixed  │ 3.0.11-1      │ 3.0.15-1      │
└──────────────┴────────────────┴──────────┴────────┴───────────────┴───────────────┘
exit=1

Four things in the workflow differ from the usual example and each is deliberate:

• --load then scan then push, so one build artefact goes through every stage. Two buildx build invocations produce two images and break the chain of custody.
• No -v /var/run/docker.sock:/var/run/docker.sock on the scanner. A container with the Docker socket has root on the host running it, which on a self-hosted runner is your build infrastructure. Trivy does not need it — it reads the local image store, or the registry directly over the distribution API with trivy registry login.
• The digest is captured and used from then on.
• The deployable tag is created last, from the digest, after the signature exists. Until that step, nothing that deploys myapp:1.0.0 can find anything.

Digests, and the tag you must not trust

What a passing scan proves

Verify at deploy, not only at build

A signature nobody checks is a checksum on a box nobody opens.

#!/bin/bash
set -euo pipefail

IMAGE=registry.example.com/myorg/myapp
TAG=1.0.0
IDENTITY='https://github.com/myorg/myapp/.*'
ISSUER='https://token.actions.githubusercontent.com'

# 1. Resolve the mutable tag exactly once.
DIGEST=$(docker buildx imagetools inspect "$IMAGE:$TAG" \
--format '{{.Manifest.Digest}}')
REF="$IMAGE@$DIGEST"
echo "deploying $REF"

# 2. Verify the signature against the digest, with an identity constraint.
#    Without --certificate-identity-regexp, ANY valid Sigstore signature
#    passes, including one made by an attacker with their own account.
cosign verify "$REF" \
--certificate-identity-regexp "$IDENTITY" \
--certificate-oidc-issuer "$ISSUER" >/dev/null

# 3. Verify the build provenance, not just that someone signed it.
cosign verify-attestation "$REF" --type slsaprovenance \
--certificate-identity-regexp "$IDENTITY" \
--certificate-oidc-issuer "$ISSUER" >/dev/null

# 4. Only now pull, and pull the digest so the tag cannot change under us.
docker pull "$REF"
echo "verified and pulled $REF"

Failure modes worth rehearsing

The middle row is the one to internalise: a red pipeline on a commit that changed a README is usually correct and usually urgent, and the reflex to add an ignore rule so the build goes green is how a vulnerability reaches production through a control that was working.

1. Build once. Every later stage operates on that artefact, never on a rebuild.
2. Test and scan before anything is publishable, with the scan set to a non-zero exit code so the pipeline actually stops.
3. Push to a location nothing deploys from, or push last — but never leave an unscanned image under a deployable tag.
4. Capture the digest at push time and use it for every subsequent step.
5. Sign the digest, with keyless OIDC if your CI supports it.
6. Create the deployable tag last, from the digest, once signing has succeeded.
7. Verify at deploy with an identity constraint, then pull by digest.
8. Re-scan what is deployed on a schedule. The build-time scan expires the moment it finishes.

Knowledge check