Skip to main content
RunBook Academy

Git, CI/CD & GitOpsLXXXII · GitOps SecretsEncryptedAtRest

SOPS and Mozilla SOPS — file-level encryption for the GitOps repository

Advanced⏱ ~26 mingitsopsage

What you'll learn

  • Encrypt a YAML file with SOPS against an age recipient
  • Decrypt a SOPS file at the command line and at apply time via Flux
  • Distinguish SOPS per-value encryption from per-file encryption and the operational consequence of each
  • Configure Flux with a SOPS-age private key to decrypt manifests on reconcile

Prerequisites

Verified against Git 2.55.x teaching target; 2.40+ minimum · GitHub Actions continuous service; Aug 2026 documentation baseline · Argo CD v3.5.x teaching target; v3.0+ minimum · Flux v2.9.x · Sigstore Cosign v3.1.x · SLSA v1.2 · OCI Distribution Specification v1.1 · Git LFS v3.7.1 · Kubernetes (cross-course target) 1.36.x

Not yet marked complete on this device.

SOPS is the file-level encryption tool that the GitOps world settled on for the encrypted-at-rest class. Unlike a whole-file encryption tool, SOPS encrypts only the values marked sensitive, leaving keys, structure, and metadata in plaintext. The result is a file that diffs cleanly, reviews cleanly, and merges cleanly — and that exposes nothing confidential to a reader of the repository.

The encryption model

A SOPS-encrypted YAML file looks like an ordinary YAML file in which the sensitive values are base64-wrapped ciphertext blobs with an ENC[ marker. The structure, the keys, and the non-sensitive values are unmodified.

flowchart LR
    A["plaintext YAML"] --> B["sops --encrypt"]
    B --> C["ciphertext YAML committed"]
    C --> D["GitOps controller applies"]
    D --> E["sops --decrypt"]
    E --> F["Kubernetes Secret"]
    G["private key"] --> E

Three properties follow from the model:

  • Diff-friendly. A reviewer reading a pull request sees which fields changed, which are encrypted, and which are not. The encryption is at value granularity, not file granularity.
  • Multi-recipient. SOPS supports multiple key holders simultaneously — age, PGP, KMS — so the same file can be decrypted by a controller holding one key and audited by an engineer holding another.
  • Per-value, not per-file. A reviewer cannot accidentally commit an unencrypted sensitive field; the SOPS configuration rule names which keys are encrypted.

The encryption command

The basic encryption flow against an age recipient:

sops --encrypt --age "$SOPS_AGE_RECIPIENT" \
     secret.yaml > secret.enc.yaml

The variable $SOPS_AGE_RECIPIENT is the age public key the recipient generated with age-keygen. SOPS encrypts every value under the keys listed in the .sops.yaml configuration file. The output is a SOPS-encrypted YAML file ready to commit.

The decryption flow:

sops --decrypt secret.enc.yaml

Decryption uses the age private key SOPS finds in the standard locations or via the $SOPS_AGE_KEY_FILE variable.

The Flux integration

Flux supports SOPS decryption natively. The configuration points Flux at a private key, and Flux decrypts SOPS-encrypted manifests on apply:

apiVersion: v1
kind: Secret
metadata:
  name: sops-age
  namespace: flux-system
type: Opaque
stringData:
  identity.agekey: |
    AGE-SECRET-KEY-1Q...

The above Secret, when referenced in the Flux Kustomization resource’s decryption field, makes Flux decrypt every SOPS-encrypted file in that Kustomization before applying it.

The reconcile flow:

  1. Flux pulls the commit from Git.
  2. Flux encounters a SOPS-encrypted manifest.
  3. Flux calls the SOPS decryption library with the controller’s private key.
  4. The plaintext manifest is applied to the cluster.

The plaintext never leaves the Flux controller process. The cluster’s Secret resource is created with the decrypted value; the value is not visible in any log line or in any intermediate file Flux writes to disk.

Per-value encryption rules

The .sops.yaml file controls which keys are encrypted:

creation_rules:
  - path_regex: secrets/.*\.yaml$
    age: >-
      age1ql3z7hjy54pw3hyww5ayyfg7zqgvc7w3j2elw8zmrj2kg5sfn9aqmcac8p

The rule above matches every YAML file under secrets/ and encrypts the values under those keys. The keys themselves (apiVersion, kind, metadata, data, stringData) are left readable so that the file is reviewable in a pull request.

The consequence: a reader of the repository sees a Kubernetes Secret manifest, with a name, a namespace, and ciphertext under data or stringData. They know which secret; they do not know the value.

When to prefer SOPS over Sealed Secrets

The two tools solve the same problem with different defaults. SOPS is the right choice when:

  • Multi-cluster decryption is needed. The same file can be encrypted for multiple recipients; each cluster decrypts with its own key.
  • The repository is broadly readable. Anyone can read the structure; only the right recipient can read the values.
  • Audit needs non-cluster decryption. An auditor with their own key can decrypt without cluster access.

Sealed Secrets is the right choice when:

  • The cluster is the trust boundary. A SealedSecret minted for one cluster should never decrypt on another.
  • The team is small. Cluster-bound keys are easier to reason about than file-bound keys.

Both can coexist in the same repository for different Secret types. A team might use Sealed Secrets for cluster-internal credentials and SOPS for cross-cluster shared credentials.

Production discipline

  1. The .sops.yaml is committed. It contains public recipients and rules; it is safe.
  2. The private key is never in the repository. Same rule as for any other secret, plus the rule that an error message quoting the key path is also a leak.
  3. Key rotation re-encrypts every file. A rotation of the age keypair without re-encrypting the existing files leaves them readable to the old key holder.
  4. The encryption rule covers every sensitive field. A field not covered by .sops.yaml is committed in plaintext. Adding a new Secret type requires checking the rule.

Cross-course references

  • Linux for Production Sysadmins - Part XXXIV (ConfigMgmt) covers file-level encryption at rest; the threat-model framing is the same.
  • Ansible for Production Sysadmins - Part XXXVII (RepoArch) covers Ansible Vault; the model is conceptually identical to SOPS.
  • Terraform for Production Sysadmins - Part XV (SensitiveVars) covers state-side encryption.

Quiz

Knowledge check · 4 questions

  1. Q1. Why does SOPS encrypt values rather than entire files?

  2. Q2. A SOPS-encrypted file can be decrypted by any engineer who has read access to the Git repository.

  3. Q3. Write the SOPS command that encrypts a YAML file against an age recipient whose public key is stored in the environment variable SOPS_AGE_RECIPIENT.

  4. Q4. Decide the response when a SOPS-age private key is rotated but the SOPS files in the repository are not re-encrypted.

    The platform team rotates their SOPS age keypair for hygiene. They update the .sops.yaml file with the new recipient and update Flux's decryption Secret with the new private key. They do not re-encrypt the existing SOPS files in the repository. Flux continues to reconcile; cluster Secrets are produced; everything appears to work.

Passing score: 75%. Answers are checked in this browser.