Velero backup lifecycle — schedules, hooks, and resource selection

The Velero backup lifecycle is the operational centre of a Kubernetes backup program. This lesson walks the Backup CRD, the Schedule CRD, resource selection, hooks, retention, the phases, and the operational discipline of treating the schedule as production infrastructure.

The Backup CRD

stateDiagram-v2
    [*] --> New: Backup CRD created
    New --> InProgress: velero picks up
    InProgress --> Completed: success
    InProgress --> PartiallyFailed: some errors
    InProgress --> Failed: unrecoverable error
    Completed --> [*]
    PartiallyFailed --> [*]
    Failed --> [*]

A Backup is a CRD that captures the scope, hooks, retention, and current state of a single backup run:

apiVersion: velero.io/v1
kind: Backup
metadata:
  name: daily-full
  namespace: velero
spec:
  includedNamespaces:
    - prod-app
    - prod-data
  excludedResources:
    - events
    - events.k8s.io
  labelSelector:
    matchLabels:
      backup: include
  includeClusterResources: true
  defaultVolumesToKopia: true
  ttl: 720h
  hooks:
    resources:
      - name: postgres
        namespace: prod-data
        labelSelector:
          matchLabels:
            app: postgres
        pre:
          - exec:
              container: db
              command: ["/scripts/quiesce.sh", "freeze"]
              timeout: 5m
        post:
          - exec:
              container: db
              command: ["/scripts/quiesce.sh", "thaw"]
              timeout: 5m

The fields:

• includedNamespaces / excludedNamespaces — scope by namespace. Default is all namespaces.
• includedResources / excludedResources — scope by resource kind. Common excludes are events (high cardinality, low value) and events.k8s.io.
• labelSelector — only include objects with matching labels. Useful for opting in specific workloads.
• includeClusterResources — include cluster-scoped objects (CRDs, ClusterRoles, StorageClasses).
• defaultVolumesToKopia — default file-level backup for PVCs without explicit annotation.
• ttl — how long Velero keeps the backup before deleting it. 720h = 30 days.
• hooks — PRE/POST hooks applied to matching workloads before and after the backup.

The Schedule CRD

A Schedule creates Backups on a cron:

apiVersion: velero.io/v1
kind: Schedule
metadata:
  name: daily-full
  namespace: velero
spec:
  schedule: '0 3 * * *'
  template:
    spec:
      includedNamespaces:
        - prod-app
        - prod-data
      ttl: 720h
      includeClusterResources: true
      defaultVolumesToKopia: true

The Schedule CRD’s template.spec is a Backup spec without the name. Velero fills in the name with a timestamp (daily-full-20260816030000). The cron syntax is standard; 0 3 * * * is daily at 3am.

velero schedule get

NAME           STATUS    SCHEDULE      LAST BACKUP                  SELECTOR
daily-full     Enabled   0 3 * * *    2026-08-16 03:00:00 +0000    <none>

The phases

The phases and what they mean:

PartiallyFailed is the most operationally subtle. A backup can be PartiallyFailed because:

• Some objects were skipped due to API server errors.
• Some PVCs failed to snapshot or copy.
• Some PRE/POST hooks failed.

The errors are listed in velero backup describe. The operator must read the errors and decide whether the backup is acceptable for restore or whether the PartiallyFailed state hides a real loss.

velero backup describe daily-full --details

...
Errors:
  Velero:     0
  Cluster:    0
  Namespaces:
    prod-data:
      persistentvolumeclaims/pvc-xxx: Failed to snapshot: ...

Retention and TTL

The TTL is enforced by Velero; expired backups are deleted from the BSL. The TTL is independent of the object store’s lifecycle policy — Velero deletes its own bookkeeping, the bucket policy deletes the bytes.

velero backup delete daily-full-20260816030000

Manual deletion removes the backup immediately. The object store’s versioning is the safety net: an accidental delete can be undone by removing the S3 delete marker.

The operational failure modes

The backup lifecycle fails in production for predictable reasons:

• Schedule paused. A Schedule’s paused: true stops new backups. Operators pause during planned maintenance; forgetting to unpause leaves the cluster without backups.
• Hook failure. A PRE hook that times out fails the backup with a hook error. The PVC may already be snapshotted; the backup is PartiallyFailed.
• Label selector mismatch. A backup with a labelSelector that matches no objects creates an empty backup. The phase is Completed but the restore is empty.
• TTL too short. A TTL of 24h deletes daily backups before the operator can use them. Match the TTL to the retention requirement.
• Schedule overlaps. A backup that runs longer than the cron interval starts a new backup before the old one finishes. The old backup is moved to Failed by the controller to prevent pile-up.

Quiz

Production discipline

The Velero backup lifecycle in production rests on five non-negotiable elements:

• Scope backups deliberately. Use includedNamespaces and labelSelector to scope each backup. A backup that captures everything is rarely the right answer.
• Exclude events. Events are high-cardinality and low-value. Add events and events.k8s.io to excludedResources in every production backup.
• Alert on PartiallyFailed. PartiallyFailed hides real losses. Alert on every PartiallyFailed event and triage the errors.
• Test schedules by stopping them. A Schedule that has never been paused and unpaused may fail silently when paused (forgotten). Quarterly pause drills prevent surprises.
• Document the runbook. The Schedule’s spec, the retention, the hook chain, the alert thresholds — all belong in the runbook.

The backup lifecycle is the operational centre of a backup program. A Schedule that runs every night but is never monitored is not a backup — it is a hope.