Skip to main content
RunBook Academy

KubernetesLV · Storage SnapshotsStorage snapshots

Application-consistent snapshots — quiesce, freeze, and the application-level hooks

Advanced⏱ ~17 minkubectlpsql

What you'll learn

  • Explain why CSI snapshots are crash-consistent by default
  • Apply the techniques for application-consistency (quiesce, freeze, fsync)
  • Use the application-level hooks (pg_start_backup, FLUSH TABLES WITH READ LOCK)
  • Validate the snapshot via restore test

Prerequisites

Verified against Kubernetes 1.34.x · kubeadm 1.34.x · kubectl 1.34.x · etcd 3.6.x · CoreDNS 1.11.x · containerd 1.7.x / 2.x · 2026-08-16

Not yet marked complete on this device.

The single most important fact about CSI snapshots: they are crash-consistent by default. The application must cooperate to achieve application-consistency. This lesson walks the techniques, the application hooks, and the production pattern for application-consistent snapshots.

The default: crash-consistent

When a VolumeSnapshot is created, the CSI driver calls the backend’s snapshot API. The backend captures the volume at one moment. The application is not notified; writes may be in progress; the snapshot is crash-consistent.

sequenceDiagram
    participant U as User
    participant API as API server
    participant SC as Snapshot controller
    participant CSI as CSI driver
    participant BE as Backend
    participant DB as DB
    Note over U: Default behavior (no coordination)
    U->>API: create VolumeSnapshot
    API->>SC: snapshot controller sees it
    SC->>CSI: CreateSnapshot
    CSI->>BE: create snapshot
    BE-->>CSI: snapshot created

The default is crash-consistent: the snapshot captures the disk at one moment, but the application may be in the middle of a transaction. On restore, the application must recover (e.g., PostgreSQL WAL replay).

The techniques for application-consistency

The three techniques:

Quiesce (application-level)

The application is told to enter backup mode:

# PostgreSQL
psql -c "SELECT pg_start_backup('snapshot-2026-08-16', true);"
# ... snapshot ...
psql -c "SELECT pg_stop_backup();"

# MySQL
mysql -e "FLUSH TABLES WITH READ LOCK;"
# ... snapshot ...
mysql -e "UNLOCK TABLES;"

The application pauses writes; the snapshot is captured; the application resumes.

Freeze (filesystem-level)

The filesystem is frozen via fsfreeze:

fsfreeze -f /var/lib/postgresql/data
# ... snapshot ...
fsfreeze -u /var/lib/postgresql/data

Freeze blocks writes at the kernel level; the snapshot captures the frozen state.

fsync (application-level flush)

The application flushes dirty pages:

# PostgreSQL
psql -c "CHECKPOINT;"

# MySQL
mysql -e "FLUSH LOGS;"

fsync ensures all in-memory data is on disk before the snapshot. Combined with quiesce, this is the most reliable approach.

The coordination procedure

sequenceDiagram
    participant O as Operator
    participant DB as Database
    participant API as API server
    participant CSI as CSI driver
    Note over O,DB: Quiesce
    O->>DB: pg_start_backup
    DB-->>O: backup mode
    O->>DB: CHECKPOINT
    DB-->>O: dirty pages flushed
    Note over O,CSI: Snapshot
    O->>API: create VolumeSnapshot
    API->>CSI: CreateSnapshot
    CSI-->>API: snapshot created
    Note over O,DB: Resume
    O->>DB: pg_stop_backup
    DB-->>O: backup mode ended

The operator:

  1. Tells the database to enter backup mode.
  2. The database flushes dirty pages.
  3. The operator creates the VolumeSnapshot.
  4. The snapshot is captured while the database is in backup mode.
  5. The operator tells the database to exit backup mode.

The application-specific hooks

PostgreSQL

#!/bin/bash
# PostgreSQL application-consistent snapshot

# Quiesce
psql -h $DB_HOST -U $DB_USER -c "SELECT pg_start_backup('snapshot-$(date +%Y%m%d)', true);"

# Flush
psql -h $DB_HOST -U $DB_USER -c "CHECKPOINT;"

# Snapshot
kubectl apply -f - <<EOF
apiVersion: snapshot.storage.k8s.io/v1
kind: VolumeSnapshot
metadata:
  name: postgres-snap-$(date +%Y%m%d)
spec:
  source:
    persistentVolumeClaimName: data-postgres-0
  volumeSnapshotClassName: postgres-snap
EOF

# Wait for snapshot
kubectl wait --for=jsonpath='{.status.readyToUse}'=true \
  volumesnapshot/postgres-snap-$(date +%Y%m%d) --timeout=600s

# Resume
psql -h $DB_HOST -U $DB_USER -c "SELECT pg_stop_backup();"

MySQL

#!/bin/bash
# MySQL application-consistent snapshot

# Quiesce
mysql -h $DB_HOST -u $DB_USER -p$DB_PASS -e "FLUSH TABLES WITH READ LOCK;"

# Snapshot
kubectl apply -f mysql-snap.yaml

# Resume
mysql -h $DB_HOST -u $DB_USER -p$DB_PASS -e "UNLOCK TABLES;"

MongoDB

MongoDB’s WiredTiger storage engine has built-in crash recovery; a crash-consistent snapshot is recoverable through WAL replay. The procedure:

# MongoDB snapshot (with the database running)
# The snapshot is crash-consistent; WiredTiger replays the journal on restore
kubectl apply -f mongo-snap.yaml

# Or use the filesystem-level freeze
fsfreeze -f /var/lib/mongodb
kubectl apply -f mongo-snap.yaml
fsfreeze -u /var/lib/mongodb

The Operator pattern for snapshots

Production Operators (Cloud Native PG, Zalando, etc.) encode the application-consistent snapshot procedure:

# Cloud Native PG: scheduled backup
apiVersion: postgresql.cnpg.io/v1
kind: Cluster
metadata:
  name: my-app-db
spec:
  instances: 3
  storage:
    size: 100Gi
  backup:
    barmanObjectStore:
      destinationPath: s3://my-bucket/backups
      s3Credentials:
        accessKeyId:
          name: barman-creds
          key: ACCESS_KEY_ID
        secretAccessKey:
          name: barman-creds
          key: SECRET_ACCESS_KEY
    retentionPolicy: "30d"

The Operator:

  • Calls pg_start_backup / pg_stop_backup around the snapshot.
  • Coordinates with the database for application- consistency.
  • Schedules the snapshot.
  • Manages the retention.

The validation

After creating the snapshot, validate via restore:

# 1. Create a recovery namespace
kubectl create namespace postgres-recovery-test

# 2. Create a PVC from the snapshot
kubectl apply -n postgres-recovery-test -f - <<EOF
apiVersion: v1
kind: PersistentVolumeClaim
metadata:
  name: data-recovery
spec:
  accessModes: ["ReadWriteOnce"]
  storageClassName: db-ssd
  resources:
    requests:
      storage: 100Gi
  dataSource:
    name: postgres-snap-20260816
    kind: VolumeSnapshot
    apiGroup: snapshot.storage.k8s.io
EOF

# 3. Create a recovery Pod
kubectl apply -n postgres-recovery-test -f postgres-recovery-pod.yaml

# 4. Validate the data
kubectl exec -n postgres-recovery-test postgres-recovery -- \
  psql -c "SELECT count(*) FROM users;"
kubectl exec -n postgres-recovery-test postgres-recovery -- \
  psql -c "SELECT pg_is_in_recovery();"

# 5. Clean up
kubectl delete namespace postgres-recovery-test

The validation is the final test. A snapshot that produces a corrupted restore is not application- consistent.

Quiz

Knowledge check · 4 questions

  1. Q1. What is the default consistency level of a CSI snapshot?

  2. Q2. A CSI snapshot is application-consistent by default.

  3. Q3. Your team's PostgreSQL snapshots are crash-consistent. The restore is corrupted. Design the application-consistent snapshot procedure.

    PostgreSQL on Kubernetes. CSI snapshots are taken hourly via a CronJob. The snapshots are crash-consistent. On restore, the database reports corruption. The team needs to switch to application-consistent snapshots.

  4. Q4. Explain why a CSI snapshot is crash-consistent by default and what is required for application-consistency.

Passing score: 75%. Answers are checked in this browser.

Production discipline

  • A CSI snapshot is crash-consistent by default. The application must cooperate for application- consistency.
  • Use application-level hooks. pg_start_backup, FLUSH TABLES WITH READ LOCK.
  • Use an Operator. Cloud Native PG, Zalando, Strimzi encode the application knowledge.
  • Validate the snapshot via restore. A snapshot that produces a corrupted restore is not application- consistent.
  • Document the procedure. Every snapshot policy has a documented procedure; the consistency level is explicit.