Multi-repo per environment — dev, staging, prod each their own repo

The multi-repo per environment model is the strongest GitOps isolation available. Each environment - dev, staging, production - lives in its own Git repository, with its own CODEOWNERS file, its own branch protection, its own audit log, and its own controller watching it. The repositories share no trust boundary; a compromise in one repo does not give the attacker write access to the other.

What the model isolates

The model’s price is operational; its product is isolation. Three properties are produced by the boundary:

flowchart LR
    subgraph D["dev repo"]
        D1["CODEOWNERS: dev team"]
        D2["branch protection: dev"]
    end
    subgraph S["staging repo"]
        S1["CODEOWNERS: platform team"]
        S2["branch protection: staging"]
    end
    subgraph P["prod repo"]
        P1["CODEOWNERS: SRE + security"]
        P2["branch protection: prod"]
        P3["required reviewers: 2"]
    end
    D --> DC["dev cluster"]
    S --> SC["staging cluster"]
    P --> PC["prod cluster"]
    P1 -. "no overlap" .- D1
    P1 -. "no overlap" .- S1

• Authorization isolation. A developer with merge rights on the dev repo does not have merge rights on the prod repo. The bit (the Git permission) is enforced by the Git host, not by a CODEOWNERS file that can be edited by anyone with write access.
• Audit isolation. The prod repo’s audit log contains only commits that targeted production. A git log against the prod repo is the entire history of production changes, with no application code commits mixed in.
• Failure isolation. A misconfigured branch protection on the staging repo does not weaken the prod repo’s branch protection. A compromise of the dev repo’s CI bot does not compromise the prod repo’s CI bot.

The operational cost

The model is expensive because everything is duplicated. Every environment repo has its own:

• CODEOWNERS file
• branch protection ruleset
• CI bot identity and credentials
• controller configuration (the Application or Kustomization that points the controller at the repo)
• secret references (the dev repo’s secrets live in the dev cluster, the prod repo’s secrets live in the prod cluster, and the repos do not share secret material)

A change to a shared base must be replicated to every environment repo. A promotion to staging and a promotion to production are two separate PRs, against two separate repos, with two separate reviewer chains.

flux create source git staging-repo \
  --url=https://github.com/example/staging-env \
  --branch=main \
  --interval=5m

flux create source git prod-repo \
  --url=https://github.com/example/prod-env \
  --branch=main \
  --interval=5m

Each environment runs its own controller instance (or the same controller instance with two GitRepository resources pointing at two different repos). The repository boundary is the cluster’s trust boundary.

When the model is required

The multi-repo per environment model is not the right default for most teams. It is the right default when one of the following is true:

• Regulatory isolation. PCI-DSS, HIPAA, FedRAMP, and similar regimes often require that production credentials and production configurations are not collocated with non-production data. A single monorepo with overlays collocates them by construction; per-environment repos separate them.
• Team boundary. The team that owns production is not the team that owns development. The dev repo’s owner is the application team; the prod repo’s owner is the SRE or platform team. The Git host enforces the boundary.
• Air-gapped production. Production clusters in a regulated environment sometimes cannot read from the same Git host as development. The prod repo is on a different Git host, in a different network, with different credentials.

Under the hood

The model is what the GitOps literature sometimes calls the “environment-as-a-repo” pattern. The repository is the unit of trust, and the trust unit is the environment. The trade-off is that the repository is also the unit of merge friction, and the merge friction compounds per environment.

The model is also the only one in which promotion is a declarative action. In the monorepo with overlays model, promotion is an edit to a manifest in a specific directory. In the multi-repo model, promotion is a pull request against a different repo, which is a different Git identity, a different review chain, and a different audit record. The friction is the feature: each environment’s promotion is a deliberate act, not an accidental merge.

Production discipline

The production rules for the multi-repo per environment model:

1. Each repo has its own CI bot identity. The dev CI bot cannot push to the prod repo. The bot credentials are scoped per repo, and the rotation is per-repo.
2. The promotion script is the only path between repos. A human does not git push between dev and prod. A script opens the prod PR with the verified digest from the staging PR, and the script’s audit log is the promotion record.
3. The repo per environment is the blast-radius boundary. A compromise of the dev repo’s CI credentials cannot stage anything in production because the prod repo’s credentials are different.

Cross-course references

• Git, CI/CD & GitOps for Infrastructure Engineers - Part LXXVI-01 (Application vs Environment repos) and LXXVI-02 (Monorepo with overlays) are the lighter-weight models this lesson contrasts against.
• Linux for Production Sysadmins - Part XXXV (Multi-Tenant Isolation) covers the OS-level analogue: separate users, separate namespaces, separate trust.
• Terraform for Production Sysadmins - Part XX (Workspace Isolation) covers the Terraform analogue of per-environment state, which is the same boundary at a different layer.

Quiz