Skip to main content
RunBook Academy

Git, CI/CD & GitOpsLVIII · Deployment StrategiesBlueGreen

Blue-green deployment — two environments, atomic switch, instant rollback

Intermediate⏱ ~24 mingit

What you'll learn

  • Describe how blue-green runs two identical environments and switches traffic atomically
  • Identify the load balancer / DNS / Kubernetes Service mechanics that implement the switch
  • Recognise the data compatibility constraint that determines whether blue-green is safe for a given release
  • Use the atomic switch as a rollback mechanism and identify the cases where it does not help

Prerequisites

Verified against Git 2.55.x teaching target; 2.40+ minimum · GitHub Actions continuous service; Aug 2026 documentation baseline · Argo CD v3.5.x teaching target; v3.0+ minimum · Flux v2.9.x · Sigstore Cosign v3.1.x · SLSA v1.2 · OCI Distribution Specification v1.1 · Git LFS v3.7.1 · Kubernetes (cross-course target) 1.36.x

Not yet marked complete on this device.

Blue-green is the simplest pattern to reason about and the most expensive to operate. Two environments, identical except for the version of the workload; traffic is routed to one (blue) at steady state, and the switch to the other (green) is a single atomic operation at the load balancer, DNS, or Kubernetes Service. The switch is the rollback: one flip and the prior version is back. The pattern’s limit is data: two environments running concurrently means a shared database is being written by both, and the schema the new version writes must be one the old version can read on rollback.

How blue-green works

flowchart LR
    A["Blue: v1"] --> C{"Switch"}
    B["Green: v2"] --> C
    C -- "flip" --> A2["Blue: v1 (idle)"]
    C -- "flip" --> B2["Green: v2 (live)"]

At steady state, blue serves 100% of traffic; green is deployed but idle. The release deploys green, runs smoke tests against it, then flips the traffic. Blue becomes the idle environment, kept warm for the rollback window in case the green release must be reverted.

The atomic switch is the rollback. In Kubernetes with Argo Rollouts, the switch is a Service selector change. With a classic load balancer, it is a target group reassignment. With DNS, it is a record update with a low TTL set in advance.

sequenceDiagram
    participant R as Router
    participant B as Blue (v1)
    participant G as Green (v2)
    R->>B: 100% traffic
    Note over G: v2 deployed, smoke tests pass
    R->>G: 100% traffic (atomic)
    Note over B: idle, ready for rollback

The duration of the switch is the duration of the infrastructure-level operation: a Service selector change is sub-second; a DNS change is TTL-bound; an AWS ALB target group reassignment is a few seconds.

The data question

The pattern’s safety depends on the database. Two environments that share a single database are not two environments in the data sense; they are one logical service with two runtime versions reading and writing the same state.

flowchart TD
    A["Blue v1"] --> D["Shared database"]
    B["Green v2"] --> D
    D --> A
    D --> B

The pre-flip state is safe because only blue is writing. The flip moment is unsafe if green writes a shape blue cannot read; rolling back would corrupt reads in the same way the schema mismatch would. The discipline is the same as for rolling update: the schema must be readable and writable by both versions during the coexistence window.

The pattern is cleanest when the database is owned by the service and is recreated as part of the green environment - a schema migration that runs in green, blue remains on the old schema, and the rollback works because blue’s schema is still intact. This is the classical blue-green: two of everything, including the database.

Rollback simplicity

The atomic switch is what makes blue-green attractive for high-risk releases. A bad release is reverted by re-pointing traffic at blue; the rollback is sub-second at the load balancer. Compare with rolling update, where rollback is a fresh rolling update through the Deployment controller, or with canary, where rollback is an abort plus a separate deploy.

The simplicity has a cost: blue is idle but warm, doubling the steady-state cost of the workload for as long as blue is kept ready for rollback. Teams that care about cost discard blue after the green release is declared stable, losing the instant-rollback property. Teams that care about rollback keep blue warm for a defined retention window (hours to days).

When blue-green is the right pattern

Blue-green fits workloads where:

  • The release is high-risk and rollback must be instant. A payments release that must be revertable in under a minute benefits from blue-green.
  • The two environments can each own their database. A service that provisions its schema per environment avoids the shared-database problem.
  • Smoke testing against production-like traffic is desired. Green can be exercised by synthetic traffic or a portion of shadow traffic before the flip.

Blue-green is wrong for workloads where the shared database is the norm and the schema is non-additive, or where the infrastructure cost of running two environments is prohibitive.

Production discipline

  1. Use blue-green for high-risk releases that need instant rollback. Use rolling update or canary otherwise.
  2. Design the schema for the coexistence window. The shared-database case requires expand-and-contract.
  3. Set the rollback retention window deliberately. A team that discards blue immediately has rolling update with extra steps.
  4. Smoke-test green before the flip. A service that deploys but does not boot is a service that fails the flip.

Cross-course references

  • Kubernetes for Production Sysadmins - Part XIV (Workloads) covers the Service selector mechanics the switch relies on.
  • This course, Part LVIII-01 (The deployment pattern taxonomy) establishes where blue-green sits in the trade-off space.
  • This course, Part LVIII-03 (Canary and progressive delivery) covers the pattern that addresses small blast radius without doubling the fleet.

Quiz

Knowledge check · 4 questions

  1. Q1. A team runs blue-green against a service with a shared database. The release adds a new column to a table. Green writes the new column on the first request after the flip. The team rolls back to blue within thirty seconds. What does blue see?

  2. Q2. The atomic switch that makes blue-green attractive for rollback also implies that the rollback reverts any database state written by the green environment during the release window.

  3. Q3. Name two infrastructure mechanisms that implement the atomic switch in a blue-green deploy, and the latency trade-off between them.

  4. Q4. Diagnose why a blue-green rollback worked at the load balancer but the service still served corrupt responses, and identify the data-layer gap.

    A team runs blue-green on a checkout service. Blue (v1) is live and serving traffic. Green (v2) is deployed and smoke tests pass. The team flips the load balancer. Within minutes, customer support tickets report that checkout is showing incorrect totals. The team rolls back to blue. The rollback at the load balancer completes in two seconds. The incorrect-totals issue persists. The investigation finds that v2 wrote a new field to a 'cart_items' table that v1's serializer does not recognise. v1 reads the table, fails to parse the new field, and substitutes a default value of zero for the line item total. The blue rollback at the load balancer works; the data corruption persists because the shared database was never reverted.

Passing score: 75%. Answers are checked in this browser.