Rebuild trusted artifacts — clean source, pinned versions

Invalidation is the warning. The rebuild is the fix. A team that invalidates the compromised artifacts but does not rebuild and re-deploy leaves production running on the bytes the malicious hook may have tampered with. The rebuild runs from a clean commit, with a regenerated lockfile pinned to a known-good version, on a clean runner, signed with a new key.

The rebuild sequence

The sequence is forced by the supply-chain constraint that every step must be reproducible from a trusted input:

flowchart LR
    A["invalidated artifacts"] --> B["re-trigger from clean commit"]
    B --> C["regenerate pinned lockfile"]
    C --> D["build on clean runner"]
    D --> E["re-sign with new key"]
    E --> F["deploy with admission control"]
    F --> G["three gates passed"]

• Re-trigger from clean commit. The workflow is re-run, but not on the compromised build environment.
• Regenerate pinned lockfile. The lockfile is regenerated with the known-good version of the malicious transitive.
• Build on a clean runner. A fresh runner that has never run a job for this workflow.
• Re-sign with a new key. The previous key is in the credential inventory and must be rotated.
• Deploy with admission control. The cluster’s image admission policy rejects the invalidated artifacts.

The three verification gates

Each gate answers a question about a different link:

Regenerating the lockfile

The lockfile regeneration is the supply-chain change the audit team will examine most closely:

npm install "popular-logging-library@4.3.9" \
  --package-lock-only --save-exact
npm ci

The npm ci step is the verification: a CI command that fails on drift will fail if the lockfile and the package.json range are inconsistent.

Re-build versus re-release

The team chooses between two operations for each artifact:

• Re-build. The source is the same commit. The bytes may differ because the dependency tree differs. The default.
• Re-release. The source is a newer-known-good commit with a security fix. The right choice when the original source contained a vulnerability.

A re-release requires a code change, PR, and review. A re-build requires a clean runner and re-trigger.

The signing key rotation

The signing key the previous build used is in the credential inventory from lesson 04:

cosign generate-key-pair
cosign sign --key new-key.pem "$IMAGE_REFERENCE"

The new key is stored in the secret manager with a key rotation timestamp that survives the audit.

Deploying with admission control

The rebuilt artifact is deployed through the cluster’s image admission policy:

apiVersion: security.k8s.io/v1
kind: ImagePolicyWebhook
metadata:
  name: production-image-policy
webhook:
  imagePolicy:
    allowWhenInsecure: false
    allowedRegistries:
      - production-registry.example.com

The policy rejects the invalidated digests and accepts the new signed ones.

Production discipline

1. Three gates, not one. A single gate is not a rebuild.
2. Re-build by default; re-release when source changed.
3. Signing key rotation before rebuild.
4. Admission policy is the deploy gate.
5. Lockfile diff accompanies the rebuild. The diff is the audit artefact.

Cross-course references

• Git, CI/CD & GitOps — Part LXVII-06 covers the lockfile shape.
• Git, CI/CD & GitOps — Part XCV-05 covers the rebuild sequence.
• Container Security for Production Sysadmins — Part V covers the registry-side validation patterns.

Quiz