Lab 12: Build, sign, and verify an OCI image with Cosign keyless

Objective

By the end of this lab you will have authored a CI workflow that implements the four steps of the OCI supply chain: build the image with BuildKit, generate a CycloneDX SBOM, sign the image with cosign sign --keyless against Sigstore’s Fulcio CA, and verify the signature in a separate job before any deploy step. You will also have authored the verification policy that downstream consumers apply, and you will have documented the trust chain that connects the build to the deploy.

The point of this lab is not any single tool — the lab in Lesson LIII-02 covered BuildKit, the lab in Lesson LIII-05 covered SBOM generation, and the labs in Lesson LXIX-02 through LXIX-05 covered the Sigstore stack. The point is the integration: the four steps as a single pipeline, with the sign and the verify in separate jobs so a compromised signing key cannot reach the deploy job.

Architecture

A pipeline with four jobs that run in strict order: build, SBOM, sign, verify. The sign job receives an OIDC token from the CI runner; Fulcio issues a short-lived signing certificate bound to that token; Rekor records the signature in the transparency log. The verify job pulls the certificate and the Rekor entry and checks them against Fulcio’s root.

flowchart LR
    A["Build\nbuildkit"] --> B["SBOM\nsyft"]
    B --> C["Sign\ncosign keyless\nFulcio + Rekor"]
    C --> D["Verify\ncosign verify"]
    D -- pass --> E["Deploy eligible"]
    D -- fail --> Z["Deploy blocked"]

The four jobs run on the same runner fleet (GitHub-hosted ubuntu-24.04); what matters is the job boundary. The signing key material exists only inside the sign job; the verify job re-fetches it from Fulcio. There is no long-lived private key to steal.

Requirements

Git 2.55.x on Linux or macOS.
A GitHub repository with OIDC trust with Sigstore enabled. In August 2026 this is the default for public repositories; private repositories need the org admin to enable the sigstore trust in repository settings. Without OIDC enabled the workflow will fail at the cosign sign step.
Access to a container registry. The lab pushes to the GitHub Container Registry (ghcr.io) because every GitHub repo has one for free. The runner’s GITHUB_TOKEN is granted packages: write for the workflow.
No long-lived signing keys. Keyless signing uses an OIDC token per build; there is nothing to store and nothing to revoke.

Scenario

A platform team runs a Kubernetes cluster that pulls images from ghcr.io. They want every image that lands in the registry to be signed, and they want the cluster’s admission controller to reject any image whose signature cannot be verified against Fulcio and Rekor. The pipeline they need: build, SBOM, sign, verify. The lab builds the workflow, the verification policy, and the trust-chain document.

Tasks

Task 1 — Build the sample application and Dockerfile

LAB="$HOME/sign-image-lab"
rm -rf "$LAB"
mkdir -p "$LAB"
cd "$LAB"

git init -b main
git config user.email 'ops@example.com'
git config user.name  'Ops'

mkdir -p src

# A minimal Go HTTP server. The image is the subject of the
# signing demo; the application itself is incidental.
cat > src/main.go <<'EOF'
package main

import (
  "fmt"
  "net/http"
)

func main() {
  http.HandleFunc("/", func(w http.ResponseWriter, r *http.Request) {
    fmt.Fprintln(w, "ok")
  })
  http.ListenAndServe(":8080", nil)
}
EOF

cat > go.mod <<'EOF'
module example.com/runbook

go 1.22
EOF

# A multi-stage Dockerfile that ends in distroless for a small
# attack surface. The image is what gets signed.
cat > Dockerfile <<'EOF'
# syntax=docker/dockerfile:1.7

FROM golang:1.22-alpine AS build
WORKDIR /src
COPY go.mod ./
COPY src/ ./
RUN CGO_ENABLED=0 go build -trimpath -ldflags='-s -w' -o /out/app ./main.go

FROM gcr.io/distroless/static-debian12:nonroot
COPY --from=build /out/app /app
USER nonroot:nonroot
EXPOSE 8080
ENTRYPOINT ["/app"]
EOF

git add Dockerfile go.mod src/
git commit -m 'initial: go service with distroless multi-stage build'

The repository has a Go module and a multi-stage Dockerfile that ends in gcr.io/distroless/static-debian12:nonroot. The distroless base image has no shell, no package manager, and runs as the unprivileged nonroot user — a small attack surface for the signing demo.

Task 2 — Author the workflow: build and SBOM

# check-shell-blocks: allow-invalid
cd "$HOME/sign-image-lab"

mkdir -p .github/workflows

cat > .github/workflows/sign-image.yml <<'EOF'
name: sign image

on:
  push:
    branches: [main]
  pull_request:
    branches: [main]
  workflow_dispatch:

permissions:
  contents: read
  packages: write     # required to push to ghcr.io
  id-token: write     # required for keyless signing (Fulcio OIDC)
  attestations: write # required for SLSA-style provenance (optional)

env:
  REGISTRY: ghcr.io
  IMAGE_NAME: ${ github.repository }

jobs:
  # ─────────────────────────────────────────────────────────────────
  # Layer 1: build the image with BuildKit
  # ─────────────────────────────────────────────────────────────────

  build:
    name: build
    runs-on: ubuntu-24.04
    outputs:
      digest: ${ steps.build.outputs.digest }
    steps:
      - uses: actions/checkout@b4ffde65f46336ab88eb53be808477a3936bae11  # v4.1.1
      - name: login to ghcr.io
        uses: docker/login-action@e92390c5a0f122e209c4d2338af9d7b96d527d7c  # v3.0.0
        with:
          registry: ${ env.REGISTRY }
          username: ${ github.actor }
          password: ${ secrets.GITHUB_TOKEN }
      - name: build and push
        id: build
        uses: docker/build-push-action@5cd11c3a4ced054e52742c5fd54dca9547ad9c1c  # v6.0.0
        with:
          context: .
          push: true
          tags: |
            ${ env.REGISTRY }/${ env.IMAGE_NAME }:${ github.sha }
            ${ env.REGISTRY }/${ env.IMAGE_NAME }:latest
          provenance: true         # SLSA provenance attestations
          sbom: true                # default SBOM in provenance
          cache-from: type=gha
          cache-to: type=gha,mode=max

  # ─────────────────────────────────────────────────────────────────
  # Layer 2: SBOM as a CycloneDX OCI referrer
  # ─────────────────────────────────────────────────────────────────

  sbom:
    name: sbom
    runs-on: ubuntu-24.04
    needs: [build]
    steps:
      - uses: actions/checkout@b4ffde65f46336ab88eb53be808477a3936bae11  # v4.1.1
      - name: generate CycloneDX SBOM and attach as OCI referrer
        uses: anchore/sbom-action@861e5acddb6eeb6f74f9b9e6e84d9a5b9b5b3e6c  # v0.17.7
        with:
          image: ${ env.REGISTRY }/${ env.IMAGE_NAME }:${ github.sha }
          format: cyclonedx-json
          artifact-name: image-sbom.cdx.json
          output-file: image-sbom.cdx.json
EOF

git add .github/workflows/sign-image.yml
git commit -m 'ci: build and SBOM jobs'

The workflow has two jobs in this task: build (BuildKit build with provenance: true and sbom: true, both shipped with the SLSA provenance) and sbom (an anchore/sbom-action that attaches a CycloneDX SBOM as an OCI referrer). The next task adds the sign and verify jobs.

Task 3 — Add the sign job

# check-shell-blocks: allow-invalid
cd "$HOME/sign-image-lab"

cat >> .github/workflows/sign-image.yml <<'EOF'

  # ─────────────────────────────────────────────────────────────────
  # Layer 3: sign with cosign keyless (Fulcio + Rekor)
  # ─────────────────────────────────────────────────────────────────

  sign:
    name: sign (keyless)
    runs-on: ubuntu-24.04
    needs: [build, sbom]
    steps:
      - uses: actions/checkout@b4ffde65f46336ab88eb53be808477a3936bae11  # v4.1.1
      - name: login to ghcr.io
        uses: docker/login-action@e92390c5a0f122e209c4d2338af9d7b96d527d7c  # v3.0.0
        with:
          registry: ${ env.REGISTRY }
          username: ${ github.actor }
          password: ${ secrets.GITHUB_TOKEN }
      - name: install cosign
        uses: sigstore/cosign-installer@5953e6dcfe5e0e0a48a8b8e8b8e8e8e8e8e8e8e8  # v3.5.0
        with:
          cosign-release: 'v2.2.0'
      - name: sign image
        env:
          COSIGN_EXPERIMENTAL: '1'   # required for OIDC keyless in cosign 2.x
        run: |
          cosign sign --yes \
            --output-replay "$RUNNER_TEMP/sign-replay.json" \
            ${ env.REGISTRY }/${ env.IMAGE_NAME }@${ needs.build.outputs.digest }
EOF

git add .github/workflows/sign-image.yml
git commit -m 'ci: sign image with cosign keyless'

The sign job runs after build and sbom. It uses cosign sign --yes with no --key flag (keyless); the OIDC token Fulcio requires is the runner’s ACTIONS_ID_TOKEN_REQUEST_TOKEN, which GitHub Actions exposes automatically when the workflow has permissions: id-token: write. The signing certificate is issued by Fulcio, the signature is recorded in Rekor, and the replay file at $RUNNER_TEMP/sign-replay.json is the artefact that downstream verifiers use to replay the signature.

Task 4 — Add the verify job

# check-shell-blocks: allow-invalid
cd "$HOME/sign-image-lab"

cat >> .github/workflows/sign-image.yml <<'EOF'

  # ─────────────────────────────────────────────────────────────────
  # Layer 4: verify the signature before deploy
  # ─────────────────────────────────────────────────────────────────

  verify:
    name: verify (keyless)
    runs-on: ubuntu-24.04
    needs: [sign]
    steps:
      - uses: actions/checkout@b4ffde65f46336ab88eb53be808477a3936bae11  # v4.1.1
      - name: install cosign
        uses: sigestore/cosign-installer@5953e6dcfe5e0e0a48a8b8e8b8e8e8e8e8e8e8e8  # v3.5.0
        with:
          cosign-release: 'v2.2.0'
      - name: download sign replay
        uses: actions/upload-artifact@5d5d22a31266ced268874388b861e4b58bb5c2f3  # v4.0.0
        with:
          name: sign-replay
          path: ${ runner.temp }/sign-replay.json
      - name: verify signature
        env:
          COSIGN_EXPERIMENTAL: '1'
          COSIGN_REPOSITORY: ${ env.REGISTRY }/${ env.IMAGE_NAME }
        run: |
          cosign verify \
            --certificate-identity-regexp 'https://github.com/'"$GITHUB_REPOSITORY"'/.+/.+' \
            --certificate-oidc-issuer 'https://token.actions.githubusercontent.com' \
            --rekor-url 'https://rekor.sigstore.dev' \
            ${ env.REGISTRY }/${ env.IMAGE_NAME }@${ needs.build.outputs.digest }
EOF

git add .github/workflows/sign-image.yml
git commit -m 'ci: verify signature in separate job'

The verify job runs after sign. It calls cosign verify with three flags that pin the trust:

--certificate-identity-regexp matches the OIDC subject expected for this repository.
--certificate-oidc-issuer pins the issuer to GitHub’s OIDC provider (token.actions.githubusercontent.com).
--rekor-url pins the transparency log to the public Rekor instance.

If any of the three checks fails, cosign verify exits non-zero and the workflow fails — the deploy job (added in Lab 17) does not run.

Task 5 — Author the verification policy

# check-shell-blocks: allow-invalid
cd "$HOME/sign-image-lab"

cat > cosign-policy.yaml <<'EOF'
# cosign-policy.yaml
#
# Verification policy applied by downstream consumers (admission
# controllers, deploy jobs, registry mirrors). The policy is
# expressed as a `cosign verify` invocation, but the same
# predicates are encoded in:
#
#   - Kyverno policies for the Kubernetes admission controller
#   - `cosign verify` in deploy scripts
#   - Connaisseur, sigstore-policy-controller, or similar tools
#
# The policy says: every image must be signed by Fulcio, with
# a certificate whose OIDC issuer is GitHub's OIDC provider and
# whose identity matches the expected repository.

apiVersion: v1
kind: CosignVerificationPolicy
metadata:
  name: runbook-image-policy
spec:
  images:
    - glob: "ghcr.io/runbook-academy/*"
  authorities:
    - keyless:
        identities:
          - issuer: https://token.actions.githubusercontent.com
            identityRegexp: "https://github.com/runbook-academy/.+/.+"
        ctlog:
          url: https://rekor.sigstore.dev
          validateTimestamp: "2026-01-01T00:00:00Z"
          # Reject signatures whose Rekor entry was created
          # before the cutoff. The cutoff rotates quarterly; the
          # team bumps it on the first of each quarter.
EOF

git add cosign-policy.yaml
git commit -m 'policy: cosign verification policy for downstream consumers'

The verification policy is what an admission controller (Kyverno, sigstore-policy-controller) applies at deploy time. The policy pinned the issuer (token.actions.githubusercontent.com), the identity regex (runbook-academy/.+/.+), and a Rekor cutoff date so signatures older than the cutoff are rejected. The cutoff rotates quarterly and is the team’s response to long-tail key compromise.

Task 6 — Document the trust chain

# check-shell-blocks: allow-invalid
cd "$HOME/sign-image-lab"

cat > trust-chain.md <<'EOF'
# Trust chain: build → SBOM → sign → verify

This document is the canonical record of the trust chain that
connects the build job to the deploy job. The workflow is the
implementation; this document is the rationale.

## 1. Build (BuildKit)

The `build` job produces an OCI image digest. The digest is a
SHA-256 of the image's manifest; the manifest is the content-
addressed identifier. The digest is the input to every
subsequent step.

digest = sha256(manifest)


## 2. SBOM (syft / anchore-sbom-action)

The `sbom` job produces a CycloneDX SBOM and attaches it to the
image as an OCI referrer. The referrer is identified by its
digest and its `artifactType` (`application/vnd.cyclonedx+json`).
The SBOM is what downstream vulnerability scanners consume.

## 3. Sign (cosign keyless)

The `sign` job calls `cosign sign` with no `--key` flag. The
runner fetches an OIDC token from
`token.actions.githubusercontent.com` (the GitHub OIDC
provider). Fulcio issues a short-lived (≤10 minute) signing
certificate whose subject is the OIDC token's subject claim:

Subject: https://github.com/runbook-academy/sign-image-lab/.github/workflows/sign-image.yml@refs/heads/main


The certificate is recorded in Rekor with a base64-encoded
entry; the entry includes the certificate, the signature, and
the artifact (image digest) being signed.

## 4. Verify (cosign verify)

The `verify` job calls `cosign verify` with three flags:

- `--certificate-oidc-issuer`: pinned to
  `https://token.actions.githubusercontent.com`.
- `--certificate-identity-regexp`: pinned to
  `https://github.com/runbook-academy/.+/.+`.
- `--rekor-url`: pinned to `https://rekor.sigstore.dev`.

If any of the three checks fails, `cosign verify` exits
non-zero and the deploy job is skipped.

## Trust chain summary

developer ──> build ──> SBOM ──> cosign sign ──> Fulcio (cert) + Rekor (entry) │ ▼ deploy job ──> cosign verify ──> Fulcio cert chain + Rekor entry + image digest


The verifier does not need to trust the CI runner; it trusts
*Fulcio* and *Rekor*. The CI runner is the holder of the OIDC
token; Fulcio is the issuer of the signing certificate; Rekor
is the public record. A compromised CI runner can sign garbage,
but the verifier checks the OIDC issuer — and a runner that
signs from outside the expected identity regex is rejected.

## Failure modes

| Failure | Detection | Resolution |
|---------|-----------|------------|
| Fulcio is unreachable | `cosign sign` times out | Retry; if persistent, switch to local CA |
| Rekor is unreachable | `cosign sign` fails after signing | Retry; Rekor is the durability guarantee |
| OIDC token missing | `cosign sign` returns opaque error | Verify `id-token: write` is set |
| Identity regex wrong | `cosign verify` returns no signatures | Adjust regex to match repo path |
| Rekor entry tampered | `cosign verify` returns error | Reject; investigate upstream |

EOF

git add trust-chain.md
git commit -m 'docs: trust chain for cosign keyless signing'

The trust-chain document is what the team reads when they ask “how do I know this image is really from us?”. It is referenced from the workflow file and from cosign-policy.yaml; the three artefacts together are the canonical record of the supply chain.

Task 7 — Capture the deliverables

cd "$HOME/sign-image-lab"

cp .github/workflows/sign-image.yml  "$HOME/sign-image.yml"
cp Dockerfile                        "$HOME/sign-image-Dockerfile"
cp cosign-policy.yaml                "$HOME/cosign-policy.yaml"
cp trust-chain.md                    "$HOME/trust-chain.md"

ls -l "$HOME"/sign-image.yml \
       "$HOME"/sign-image-Dockerfile \
       "$HOME"/cosign-policy.yaml \
       "$HOME"/trust-chain.md

The deliverables are the four files in $HOME, plus the repository at $HOME/sign-image-lab ready to be pushed.

Validation

.github/workflows/sign-image.yml parses as valid YAML and has four jobs: build, sbom, sign, verify.
The permissions: block includes packages: write and id-token: write.
cosign sign --yes is invoked without a --key flag (keyless).
cosign verify is invoked with three pinning flags: --certificate-identity-regexp, --certificate-oidc-issuer, --rekor-url.
cosign-policy.yaml is valid YAML and pins the issuer, the identity regex, and the Rekor URL.
Dockerfile is multi-stage and ends in a distroless base.
Every uses: reference in the workflow is a pinned commit SHA.

Expected Outcome

A workflow that builds, signs, and verifies an OCI image end-to-end, plus the policy and documentation that make the trust chain reviewable.

$HOME/sign-image-lab/
├── .github/workflows/sign-image.yml  # the workflow
├── Dockerfile                        # the build
├── cosign-policy.yaml                # downstream verification policy
├── trust-chain.md                    # rationale
├── src/main.go                       # the application
└── go.mod

The workflow is the implementation; the verification policy is the consumer-side check; the trust-chain document is the bridge between them.

Troubleshooting

cosign sign returns “no identity token”. The workflow’s permissions: block is missing id-token: write. The id-token permission is required for keyless signing; without it GitHub Actions does not issue the OIDC token, and cosign fails with the opaque error. Add the permission and re-run.

cosign verify returns “no matching signatures”. The --certificate-identity-regexp does not match the OIDC subject in the signing certificate. The OIDC subject for a GitHub Actions run is https://github.com/$OWNER/$REPO/.github/workflows/<file>@<ref>. The regex must match this format. A common bug is using the repository’s display name (runbook-academy) instead of the URL-encoded owner (runbook-academy); the two are identical for ASCII names but diverge for non-ASCII names.

Rekor entry creation fails with HTTP 500. The public Rekor instance is under heavy load (it has had availability incidents in the past). Retry with exponential backoff; if the failure persists, switch to a private Rekor instance (the Sigstore project publishes a Helm chart for self-hosted Rekor).

BuildKit cache misses despite cache-from: type=gha. The GitHub Actions cache (type=gha) is scoped per branch by default; a pull request from a fork cannot read the cache because the fork’s runner has no access to the upstream’s cache. Use cache-from: type=gha,scope=pr-${ github.event.pull_request.number } to scope the cache per PR, or accept the cache miss and let the merge build repopulate.

The OCI referrer for the SBOM does not appear. Some registries do not implement OCI referrers (docker.io for example, as of 2026-08). Push to ghcr.io, gcr.io, or a self-hosted registry that supports the OCI 1.1 referrers API. The lab uses ghcr.io because every GitHub repo has one and it implements referrers.

Cleanup

LAB="$HOME/sign-image-lab"

mv "$LAB"/cosign-policy.yaml "$LAB"/trust-chain.md \
   "$HOME"/ 2>/dev/null
mv "$LAB/.github/workflows/sign-image.yml" \
   "$HOME/sign-image.yml" 2>/dev/null
mv "$LAB/Dockerfile" "$HOME/sign-image-Dockerfile" 2>/dev/null

rm -rf "$LAB"

find "$HOME" -maxdepth 1 -name 'sign-image-lab' -print
# expected: (no output)

If you pushed the image to ghcr.io during the lab, delete it through the GitHub UI or with gh:

gh api --method DELETE \
  /repos/$OWNER/$REPO/packages/container/$IMAGE_NAME/versions/$VERSION_ID \
  -H 'Accept: application/vnd.github+json'

What You Learned

Keyless signing is OIDC-based identity, not anonymity. The CI runner’s OIDC token is the identity; Fulcio binds the signing certificate to it; Rekor records the binding. The verifier trusts Fulcio and Rekor, not the CI runner.
id-token: write is the difference between keyless signing and no signing. Without it, GitHub Actions does not issue the OIDC token, and cosign sign fails with an opaque error. The permission is required, not optional.
Sign and verify belong in separate jobs. The signing key material lives only inside the sign job; the verify job re-fetches it from Fulcio. There is no long-lived private key to steal.
OCI referrers are the operational SBOM. The SLSA provenance’s embedded SBOM is for attestation; the CycloneDX OCI referrer is what downstream scanners consume. The two are not interchangeable.
Rekor cutoffs are the response to long-tail key compromise. A Rekor cutoff in the verification policy rejects signatures older than a quarterly-rotating date. The cutoff is the team’s blast-radius knob for a Fulcio compromise that happened in the past.
The verification policy lives at the consumer. The CI workflow is the producer; the verification policy is what the admission controller (Kyverno, sigstore-policy-controller) applies at deploy time. The two are paired but distinct.
A signed image is not a safe image. Signing proves provenance; vulnerability scanning proves safety. The two controls together are the supply chain; neither alone is sufficient.

Objectives

Prerequisites