Pulumi and Kubernetes GitOps — the Kubernetes operator pattern

Pulumi takes a different shape from Terraform in the GitOps model. Terraform’s GitOps path runs in an external service (Atlantis) or in an in-cluster operator (terraform-controller); both wrap the same terraform plan and terraform apply binary. Pulumi’s GitOps path runs Pulumi programs as Pods in the cluster, with the Pulumi Kubernetes Operator driving a Stack custom resource as the desired state. The state lives in Pulumi Cloud or a self-hosted backend; the program lives in Git; the outputs land as Kubernetes secrets.

The four OpenGitOps principles are satisfied by the same mechanism as the terraform-controller: declarative (a Pulumi program in TypeScript, Python, or Go), versioned (a Git ref on the program), pulled (the operator reads the ref), and continuously reconciled (the operator loops). The differences from Terraform are in the loop’s surface area, not its shape.

flowchart LR
    A["Git: Pulumi program"] --> B["Stack CR"]
    B --> C["Operator"]
    C -->|"pulumi up --refresh"| D["Pulumi backend"]
    D -->|"state"| C
    C -->|"run"| E["Job Pod"]
    E -->|"cloud API"| F["Cloud"]
    F -->|"observed"| C
    C -->|"outputs"| G["Kubernetes secret"]

The operator watches Stack resources. When a new ref is detected on the referenced program, the operator schedules a Job that runs pulumi up --refresh, writes the outputs to a Kubernetes secret named after the stack, and updates the Stack’s status. The next reconciliation tick repeats the cycle; drift in the cloud is detected by the next --refresh and surfaced as a non-empty diff in the Stack status.

The Stack custom resource

The Stack custom resource is the desired-state declaration. It names the program, the ref, the backend, the workspace (the stack name in Pulumi parlance), and the cloud credentials to inject. The operator reconciles the Stack’s declared configuration against the actual cloud state.

apiVersion: pulumi.com/v1alpha1
kind: Stack
metadata:
  name: production-network
spec:
  programRef:
    repo: git@github.com:acme/pulumi-network.git
    ref: refs/tags/v2.1.0
  cloud: secretsmanager
  config:
    region: us-east-1
  envs:
    AWS_ACCESS_KEY_ID:
      valueFrom:
        secretKeyRef:
          name: aws-creds
          key: access-key-id
    AWS_SECRET_ACCESS_KEY:
      valueFrom:
        secretKeyRef:
          name: aws-creds
          key: secret-access-key
  destroyOnDelete: true

The programRef block is the GitOps declaration: a repository, a ref (a tag, branch, or commit), and the path inside the repo. The cloud: secretsmanager field selects the Pulumi backend - here, AWS Secrets Manager, one of the self-hosted options - and config carries the configuration values that the program reads.

The envs block is the secrets boundary. The operator mounts the referenced Kubernetes secrets as environment variables in the Job Pod, the Pod passes them to the Pulumi process, and Pulumi uses them to talk to the cloud provider. The secret values never appear in the Stack’s status or in the program’s logs; they are visible only inside the Job’s container.

Preview versus up

Pulumi’s loop has a preview step (pulumi preview) that produces a diff without applying, and an up step (pulumi up) that applies the diff after an optional confirmation. The operator runs pulumi up --refresh in the Job; the --refresh flag makes the up step query the live cloud state before planning, so the diff includes drift.

The preview step is available to the operator through the Stack’s status. The operator runs pulumi preview separately and writes the output to the Stack’s status.preview field. A reviewer can inspect the preview, set spec.continue to acknowledge, and the next reconciliation runs pulumi up. This is the human boundary in the operator model; the Atlantis-equivalent approval comment in Kubernetes is a status field.

sequenceDiagram
    participant O as Operator
    participant S as Stack CR
    participant J as Job Pod
    participant B as Pulumi backend
    participant C as Cloud
    O->>S: detect ref change
    S->>J: run pulumi preview
    J->>C: refresh
    J->>B: read state
    J->>S: status.preview = diff
    O->>S: spec.continue = true
    S->>J: run pulumi up
    J->>C: apply
    J->>B: write state
    J->>S: status.outputs = outputs

The boundary is the spec.continue flag. A Stack without that flag set will not run pulumi up; the operator treats it as a pending review. The flag is the audit anchor: the Kubernetes API server log records who set it and when.

Pulumi Cloud versus self-hosted

The choice between Pulumi Cloud and a self-hosted backend is a choice between managed and unmanaged durability. Pulumi Cloud provides a managed state store with concurrency locks, access logs, and audit trails out of the box. Self-hosted backends require the operator to configure concurrency locks (S3 with DynamoDB, Azure Blob with leases, Secrets Manager with optimistic locking) and to manage access control on the storage resource itself.

The production discipline is to choose Pulumi Cloud for team-managed state and self-hosted for environments where data residency or regulatory constraints require it. The operator treats the backend as an opaque endpoint; the configuration is in the Stack CR, not in the operator.

Production discipline

1. Pin the program ref. A programRef.ref: refs/heads/main means the operator reconciles against whatever the program repository’s default branch points at. Pin to a tag or a commit SHA.
2. Backend per environment. A shared backend across dev and production is a coordination point. Use a backend per environment, with access control that mirrors the environment boundary.
3. Preview before continue. Treat spec.continue=true as a database commit. The Kubernetes RBAC on the Stack CR must require the same reviewers as the Atlantis PR approval comment.

Cross-course references

• This course, Part LXXXVIII-01 (What infra GitOps is) - the model that the Pulumi operator implements for Pulumi programs.
• Terraform for Production Sysadmins - Parts IX-XII (State) cover the state backend discipline that self-hosted Pulumi backends require.
• Kubernetes for Production Sysadmins - Parts on Jobs and operators cover the runtime model the operator uses.

Quiz