Skip to main content
RunBook Academy

Git, CI/CD & GitOpsCXII · Container Delivery PipelineSignAndVerify

Sign and verify — cosign and the policy

Advanced⏱ ~26 mingitdockercosign

What you'll learn

  • Sign a built image digest keylessly with cosign from CI using the OIDC identity
  • Attach the SBOM and the vulnerability report as attestations to the same digest
  • Author a verification policy that pins the OIDC subject and the certificate chain
  • Recognise why a signature without a verifier is metadata, not enforcement

Prerequisites

Verified against Git 2.55.x teaching target; 2.40+ minimum · GitHub Actions continuous service; Aug 2026 documentation baseline · Argo CD v3.5.x teaching target; v3.0+ minimum · Flux v2.9.x · Sigstore Cosign v3.1.x · SLSA v1.2 · OCI Distribution Specification v1.1 · Git LFS v3.7.1 · Kubernetes (cross-course target) 1.36.x

Not yet marked complete on this device.

The signing stage closes the supply-chain chain. It signs the digest with the CI runner’s identity, attaches the SBOM and the vulnerability report as attestations to the same digest, and produces a verification policy that the admission controller enforces before any pod can pull the image. A signature without a verifier is metadata; a verifier without a policy is theatre. The two together are what make the chain auditable.

What cosign does in the pipeline

flowchart LR
    A["OIDC token (CI runner)"] --> B["Fulcio"]
    B --> C["Short-lived signing cert"]
    A --> D["cosign sign"]
    C --> D
    D --> E["Signature over digest"]
    E --> F["Rekor transparency log"]
    E --> G["Stored at registry"]
    A --> H["cosign attest (SBOM)"]
    A --> I["cosign attest (vuln report)"]
    H --> G
    I --> G
    G --> J["Admission controller verifies"]

cosign is the CLI; Sigstore is the broader project. Three Sigstore components participate:

  • Fulcio issues a short-lived X.509 certificate bound to the OIDC identity of the CI runner. The certificate says “this signing key was controlled by this OIDC subject during this short window”.
  • Rekor is an append-only transparency log. Every signature is recorded; the log is the tamper-evident record.
  • cosign performs the sign, attaches attestations, and performs the verify.

The keyless flow uses the OIDC token that GitHub Actions, GitLab CI, or CircleCI issues for a specific workflow run. The signing private key never persists beyond the run; Fulcio issues a fresh certificate; cosign signs with the per-run key.

Signing keylessly from CI

The CI invocation:

cosign sign --keyless image:$COMMIT_SHA

Behind this single line, cosign reads the runner’s OIDC token (the ACTIONS_ID_TOKEN_REQUEST_TOKEN and related environment variables from GitHub Actions, the equivalent in GitLab), exchanges it with Fulcio, receives a short-lived signing certificate, signs the digest, and submits the signature plus the Fulcio certificate to Rekor. The signature and the Rekor inclusion proof are stored as OCI artifacts at the registry, references to the signed digest.

The signing identity is the workflow path:

https://github.com/org/repo/.github/workflows/build.yml@refs/heads/main

That is the OIDC subject the Fulcio certificate names. The verifier pins to that exact subject. A signature from a different workflow, a fork, or a developer’s laptop has a different subject and fails verification.

Attaching the SBOM and vuln report as attestations

The signature covers the digest. The attestations cover the SBOM and the vulnerability report:

cosign attest --yes --predicate sbom.spdx.json --type spdxjson \
  image:$COMMIT_SHA
cosign attest --yes --predicate trivy-report.json --type vuln \
  image:$COMMIT_SHA

Each attestation is a signed claim: “the SBOM at this URL is the SBOM of the digest at this URL” and “the vuln report at this URL is the vuln report of the digest at this URL”. A verifier iterating the attestations on a digest can fetch the SBOM, verify its signature against the Fulcio certificate, and trust the inventory without re-running syft.

The same workflow identity signs the digest and signs the attestations. A signature and two attestations, all keyed off the same digest, all bound to the same OIDC subject.

The verification policy

The verifier enforces a policy that pins the OIDC subject, requires the SBOM attestation, and requires the vuln report attestation:

cosign verify --certificate-identity-regexp \
  'https://github.com/org/repo/\.github/workflows/build\.yml@refs/heads/main' \
  --certificate-oidc-issuer 'https://token.actions.githubusercontent.com' \
  image:$COMMIT_SHA

The --certificate-identity-regexp argument pins the workflow path. A wildcard like https://github.com/org/.* is weaker than the exact workflow path and accepts signatures from any workflow in the org, including forks and developer-keyed flows.

The same cosign verify invocation runs in two places:

  • In the admission controller (kyverno, connaisseur, the Sigstore policy controller). Every pod that requests the image is gated by the policy.
  • In the CI job itself, as a sanity check before the push is declared green. If the signature fails to verify in CI, the pipeline fails before the registry is touched.

What the signing stage does not do

The signing stage produces a signature and attestations. It does not:

  • Decide whether the image is correct. The test stage and the vulnerability scan stage do that.
  • Apply the image to the cluster. The deploy stage does that.
  • Replace the admission controller. Signing without a verifier is metadata.
  • Rotate keys (for keypair flows). Key rotation is a separate operation that updates verifiers across the cluster.

Production discipline

  1. Sign with the runner’s identity, not a developer’s. A signature from a developer’s laptop has no chain to anything. A signature from the CI runner with a pinned OIDC subject has the chain.
  2. Pin the OIDC subject in the verifier. A wildcard regex is weaker than pinning the exact workflow path.
  3. Verify in the admission controller, not only in CI. A signature that is only checked at build time is a signature that can be bypassed by an attacker who pushes directly to the registry.
  4. Fail the CI job if Rekor inclusion fails. A signature that does not land in Rekor has lost the tamper-evidence property.
  5. Document the key or keyless decision; review it quarterly. A signing policy that has not been reviewed in a year is a policy that no one remembers.

Cross-course references

  • Container Security for Production Sysadmins - Parts VII-VIII cover admission control and policy, the verifier side of this lesson.
  • Sigstore project - the broader Sigstore documentation covers Fulcio, Rekor, and cosign in detail.
  • This course, Part LIII (ContainerSupplyChain) - LIII-06 covers cosign signing in detail; this lesson wires it into the pipeline alongside the verification policy.

Quiz

Knowledge check · 4 questions

  1. Q1. A team signs every image with `cosign sign --keyless` from CI but does not configure a Kubernetes admission controller to verify. What is the security posture?

  2. Q2. In a keyless cosign flow, the signing private key is stored in the CI runner's secrets store and persists across runs.

  3. Q3. Name the two Sigstore components that participate in a keyless cosign flow, and explain what each contributes to the verification chain.

  4. Q4. Diagnose why unsigned images are still reaching the cluster despite a signing policy in place.

    A team configures keyless cosign signing in CI with the OIDC subject pinned to https://github.com/org/repo/.github/workflows/build.yml@refs/heads/main. The CI signs every image. The team deploys kyverno with a ClusterImagePolicy that runs `cosign verify` with `--certificate-identity-regexp 'https://github.com/org/.*'` - a wildcard. Production pods are running images signed by an engineer's local `cosign sign --keyless` invocation, satisfied because the OIDC subject regex matches any workflow in the org.

Passing score: 75%. Answers are checked in this browser.