Skip to main content
RunBook Academy

KubernetesXCVII · Kubernetes Backup ToolsKubernetes backup tools

Velero restore — selector logic, namespace mapping, and the ordering traps

Advanced⏱ ~17 minvelerokubectl

What you'll learn

  • Configure a Restore CRD with selectors and namespace mapping
  • Map namespaces during restore for cross-cluster recovery
  • Identify the ordering traps in restore (CRDs, StorageClass, RBAC)
  • Apply the operational discipline of treating restore as a runbook

Prerequisites

Verified against Kubernetes 1.34.x · kubeadm 1.34.x · kubectl 1.34.x · etcd 3.6.x · CoreDNS 1.11.x · containerd 1.7.x / 2.x · 2026-08-16

Not yet marked complete on this device.

A Velero restore re-creates objects from a backup. The Restore CRD supports selectors, namespace mapping, resource filtering, and storage location mapping. This lesson walks the Restore mechanics, the namespace mapping for cross-cluster recovery, the ordering traps, and the operational discipline of treating restore as a runbook.

The Restore CRD

sequenceDiagram
    participant User
    participant API as API Server
    participant V as velero server
    participant S3 as Object Storage

    User->>API: apply Restore CRD
    API->>V: watch Restore
    V->>S3: GET tarball for backup
    V->>API: apply objects in order
    V->>API: create VolumeSnapshot from snapshots
    V->>API: bind restore PVCs
    V->>API: update Restore.status.phase=Completed

A Restore:

apiVersion: velero.io/v1
kind: Restore
metadata:
  name: daily-full-restore
  namespace: velero
spec:
  backupName: daily-full-20260816030000
  includedNamespaces:
    - prod-app
    - prod-data
  namespaceMapping:
    prod-app: prod-app-restored
    prod-data: prod-data-restored
  includeClusterResources: true
  itemOperationTimeout: 1h

The fields:

  • backupName — the source backup. Required.
  • includedNamespaces / excludedNamespaces — scope by namespace. Default is all namespaces in the backup.
  • namespaceMapping — rename namespaces during restore. prod-app becomes prod-app-restored in the target cluster.
  • includeClusterResources — restore cluster-scoped objects (CRDs, ClusterRoles).
  • itemOperationTimeout — max time per object. Default is 1h.

Namespace mapping for safe testing

flowchart LR
    A[Source backup] --> B[Namespace prod-app]
    A --> C[Namespace prod-data]
    D[Namespace mapping] --> E[prod-app -> prod-app-restored]
    D --> F[prod-data -> prod-data-restored]
    B --> E
    C --> F

Namespace mapping is invaluable for safe restore testing:

velero restore create test-restore \
  --from-backup daily-full-20260816030000 \
  --namespace-mappings prod-app:prod-app-restored,prod-data:prod-data-restored

The restore creates the objects in prod-app-restored and prod-data-restored. The original prod-app is untouched. The operator can then kubectl get all -n prod-app-restored to verify the restore before deleting the test namespaces.

Resource selection

Restores can filter what they apply:

spec:
  backupName: daily-full-20260816030000
  includedResources:
    - persistentvolumeclaims
    - services
    - configmaps
  excludedResources:
    - secrets
  includeClusterResources: false

Common patterns:

  • Restore PVCs without workloads. Useful for data-only recovery — restore the data into a PVC, mount it for forensic analysis.
  • Restore workloads without RBAC. Useful when the target cluster has different RBAC.
  • Restore everything except secrets. Useful when secrets are managed by an external system (Vault, external-secrets).

Storage location mapping

A restore can specify a different BSL/VSL than the backup used:

spec:
  backupName: daily-full-20260816030000
  storageLocation: dr-bsl
  volumeSnapshotLocations:
    - dr-vsl

This is the cross-cluster restore path: the backup was created in cluster A’s BSL; the restore reads from cluster B’s BSL that points to the same bucket.

The ordering traps

Velero enforces an internal restore order, but the operator must still get the cluster-level ordering right:

flowchart TD
    A[1. CRDs] --> B["2. StorageClass, CSI driver"]
    B --> C[3. VolumeSnapshots]
    C --> D[4. PVCs referencing snapshots]
    D --> E["5. Services, ConfigMaps, Secrets"]
    E --> F[6. Workloads]
    F --> G["7. NetworkPolicy, Ingress, RBAC"]
    G --> H[8. Validate]

The traps:

  • CRDs missing. Velero restores CRs whose CRDs do not exist. The API server rejects them. The error appears in velero restore describe.
  • StorageClass missing. Restore PVCs reference a StorageClass that does not exist on the target cluster. The PVCs stay Pending.
  • RBAC different. The target cluster has a different RBAC model. The ServiceAccount restored may lack the bindings it had in the source cluster.
  • NetworkPolicy blocks traffic. Restoring NetworkPolicies before workloads can block the workload from reaching its dependencies. The workload starts but cannot connect.

Validating the restore

Three checks after every restore:

# 1. The restore is Completed
velero restore describe daily-full-restore --details

# 2. The objects are present
kubectl get all -n prod-app-restored

# 3. The PVCs are bound
kubectl get pvc -n prod-data-restored

If any check fails, the restore is incomplete. The fix is to identify the missing prerequisite and re-run the restore.

The operational failure modes

Restore fails in production for predictable reasons:

  • Backup not in BSL. The restore references a backup that has been deleted or is in a different bucket. Velero returns “backup not found”.
  • CSI driver mismatch. The target cluster’s CSI driver does not support the source cluster’s snapshots. The VolumeSnapshot stays in Provisioning.
  • RBAC insufficient. The Velero ServiceAccount lacks permission to create the target objects. The restore is PartiallyFailed with permission errors.
  • Namespace conflict. The target namespace already exists. Velero’s namespace mapping renames it, but if the source namespace exists, the restore creates duplicates.
  • Quota exceeded. The target namespace has a ResourceQuota that the restore exceeds. The restore fails with quota errors.

Quiz

Knowledge check · 4 questions

  1. Q1. Why use namespace mapping in a Velero restore?

  2. Q2. Velero enforces the internal restore order: CRDs first, then CRs, then workload objects.

  3. Q3. A restore into a sandbox cluster completes for manifests but PVCs stay Pending with FailedBinding. The source cluster used a different CSI driver. Diagnosis and fix?

    The restore was created with `--from-backup daily-full`. The PVCs are Pending. The events show `FailedBinding: VolumeSnapshot snapcontent-xxx is not supported by driver`. The source cluster uses the EBS CSI driver; the sandbox cluster uses a different CSI driver that cannot clone the snapshot.

  4. Q4. Name three ordering prerequisites that must be present on the target cluster before a Velero restore starts.

Passing score: 75%. Answers are checked in this browser.

Production discipline

Velero restore in production rests on five non-negotiable elements:

  • Test-restore quarterly. A backup that has never been restored is a hope. Quarterly test-restores into sandbox namespaces prove the restore chain.
  • Map namespaces for safety. Use --namespace-mappings to keep the originals untouched during a test.
  • Validate the restore end-to-end. After every restore, check velero restore describe, kubectl get all, and kubectl get pvc. The restore is complete only when the workloads are functional.
  • Document the runbook. The runbook lists the restore command, the namespace mappings, the validation checks, and the rollback. A restore during an incident is not the time to compose the command from memory.
  • Match the storage infrastructure. The target cluster’s CSI driver and StorageClass must match the source cluster’s. Cross-driver restore is not transparent; document the dependency.

Restore is the moment of truth. A backup program without restore validation is hope dressed up as discipline.