Helm production discipline — safety, review, and the discipline of pinning

Helm production discipline is the operational rigour that turns a useful tool into production deployment infrastructure. This lesson walks the safety nets, the multi-release tooling, the GitOps integration, and the discipline.

The safety nets

flowchart LR
    A[Helm safety nets] --> B[Pin versions]
    A --> C[Sign charts]
    A --> D[--dry-run before apply]
    A --> E[--atomic for rollback]
    A --> F[--wait for readiness]
    A --> G[helm diff in PR review]
    A --> H["Retention: --history-max"]

The safety nets:

• Pin versions. helm install --version 1.0.0 ensures the same chart version is installed.
• Sign charts. cosign or Notary v2 signatures verify authenticity.
• —dry-run before apply. helm install --dry-run shows what would happen without applying.
• —atomic for rollback. If the upgrade fails, Helm rolls back automatically.
• —wait for readiness. helm upgrade --wait waits for resources to be Ready.
• helm diff in PR review. helm diff upgrade shows the changes; reviewed in the PR.
• Retention: —history-max. Keep enough revisions to roll back; default is 10.

helmfile for multi-release management

# helmfile.yaml
repositories:
  - name: bitnami
    url: https://charts.bitnami.com/bitnami
  - name: oci
    oci: true
    url: registry.example.com/charts

releases:
  - name: postgresql
    namespace: prod-data
    chart: bitnami/postgresql
    version: 12.1.0
    values:
      - values/postgresql.yaml

  - name: myapp
    namespace: prod-app
    chart: oci/myapp
    version: 1.2.3
    values:
      - values/myapp.yaml

environments:
  prod:
    values:
      - values-prod/overrides.yaml

helmfile is a declarative spec for multiple Helm releases. It supports:

• Multiple releases in one file.
• Environment-specific values overrides.
• Diff before apply (helmfile diff).
• Apply with safety flags (helmfile apply).

helmfile diff
helmfile apply

helmfile is the multi-release equivalent of Helm’s single-release commands.

Argo CD Helm integration

flowchart LR
    A[Git repository] --> B[Helm chart]
    B --> C[Argo CD Application]
    C --> D["Argo CD: helm template"]
    D --> E["Cluster: apply manifests"]

Argo CD can manage Helm charts natively:

apiVersion: argoproj.io/v1alpha1
kind: Application
metadata:
  name: myapp
  namespace: argocd
spec:
  source:
    repoURL: https://github.com/example/manifests
    targetRevision: main
    path: apps/myapp
    helm:
      valueFiles:
        - values-prod.yaml
      parameters:
        - name: image.tag
          value: "2.0.0"
  destination:
    server: https://kubernetes.default.svc
    namespace: prod-app
  syncPolicy:
    automated:
      prune: true
      selfHeal: true

Argo CD:

1. Reads the Helm chart from Git.
2. Renders the chart with the values.
3. Applies the rendered manifests to the cluster.
4. Continuously reconciles.

The GitOps benefits (audit trail, drift detection, PR review) apply to Helm charts.

The Helm production discipline

flowchart LR
    A[Helm production discipline] --> B[Pin version]
    B --> C[Sign chart]
    C --> D["CI: lint, template, diff"]
    D --> E[PR review]
    E --> F[Deploy with --atomic --wait]
    F --> G[Monitor]
    G --> H{Rollback needed?}
    H -->|Yes| I[helm rollback]
    H -->|No| J[Continue]

The discipline:

1. Pin version. Never use latest in production.
2. Sign chart. Sign with cosign before publishing.
3. CI: lint, template, diff. Every change goes through CI.
4. PR review. The diff is reviewed in the PR.
5. Deploy with —atomic —wait. Safety nets enabled.
6. Monitor. Watch the release’s health.
7. Rollback if needed. Use helm rollback if the release fails.

The operational failure modes

Helm production discipline fails for predictable reasons:

• No version pinning. A chart was upgraded unexpectedly because the version was not pinned.
• No signing. A tampered chart was installed.
• No dry-run. An unexpected change reached production because no one reviewed the diff.
• No —atomic. A failed upgrade left the cluster in an inconsistent state.
• No —wait. A release was marked successful before resources were ready; workloads failed.
• No history. A rollback was impossible because the previous revisions were garbage collected.

Quiz

Production discipline

Helm in production rests on five non-negotiable elements:

• Pin versions. Always specify —version.
• Sign charts. Sign with cosign; verify before install.
• —atomic and —wait. Safety nets on every upgrade.
• helm diff in PR review. The diff is reviewed before merge.
• helmfile or Argo CD for multi-release. No ad-hoc individual commands.

Helm is production deployment infrastructure. Treat it with the same rigour as any other production tool: pin, sign, review, monitor.