Rebuilding per environment — why promotion, not re-build

A per-environment rebuild is the supply-chain equivalent of a chef who tastes the sauce in the kitchen and ships a different sauce to the table. The build script is the same. The commit is the same. The bytes are different, because the rebuild ran against a different environment’s data, a different timestamp, a different cache, a different dependency resolution. The artifact the team reviewed in staging is not the artifact the team deploys to production.

Four shapes the violation takes

• A separate build job per environment. A CI pipeline with build-staging and build-production jobs, each running its own terraform plan, docker build, or npm run bundle. The artifacts differ.
• Environment variables baked into the image. A Dockerfile that takes ENV arguments at build time and produces different images for different environments.
• A per-environment cache or layer. A pipeline whose cache key is keyed on environment name; the staging cache and the production cache diverge.
• Per-environment dependency resolution. A pipeline that resolves package versions at deploy time per environment rather than at build time once.

    flowchart LR
        A["build-staging job"] --> E["artifact A"]
        B["build-production job"] --> F["artifact B"]
        E --> G["Staging ran A"]
        F --> H["Production ran B"]
        G --> I["A and B differ"]
        H --> I
        I --> J["Audit trail breaks"]
        I --> K["The team reviewed A; B was not reviewed"]

All four converge: production runs bytes that were not approved.

Why staging-is-not-production is exactly the problem

The whole point of a staging environment is to exercise the exact bytes the team intends to deploy to production. If the bytes are different, staging has answered a different question than the one the team needed answered. Staging passed answers “do the staging bytes work in the staging environment?” - not what the team needed. Production failed answers “do the production bytes work in the production environment?” - which the team cannot predict because the production artifact is different.

The discipline that closes the gap is to promote, not rebuild. The build runs once; the artifact identity (the digest) is recorded; the same digest is deployed to staging and to production. Staging exercises the exact bytes; production deploys the exact bytes; the audit trail is the digest.

What promotion looks like

• One build job, many environments. The CI pipeline produces a single artifact per commit; the staging deploy and the production deploy both consume the same artifact.
• Environment variables are deploy-time data, not build-time data. Helm values, Kubernetes ConfigMaps, Terraform variables - passed at deploy time, not baked into the image.
• Caches are not environment-specific. Keyed on commit hash and dependency lockfile, not on environment name.
• Terraform plans are saved and uploaded as artifacts. terraform plan -out=tfplan; upload tfplan; production apply reads the artifact.

Production discipline

1. Build once. The CI pipeline produces a single artifact per commit; the artifact identity is the digest.
2. Promote the artifact. Staging deploys the artifact; production deploys the same artifact.
3. Environment configuration is deploy-time data.
4. terraform plan -out=tfplan is uploaded as an artifact.

Cross-course references

• This course, Part XLV (ArtifactImmutability) - digest pinning and the immutable-identity principle.
• This course, Part XLIX (InfraPipeline) - the format / lint / plan / apply pipeline.
• Terraform for Production Sysadmins, Part IX (State) - state backends and the state that the saved plan references.

Quiz