Skip to main content
RunBook Academy

← All break/fix scenarios in Ceph

intermediateceph-rbd-clone~22 min

RBD clone snapshot mismatch - clone flatten fails

Reported symptoms

  • ●A clone reports parent mismatch
  • ●rbd flatten fails with image has snapshots
  • ●Clone I/O errors at the parent snapshot

Evidence

  • · rbd info <image> shows the parent snapshot
  • · rbd snap ls <parent> shows the snapshot exists
  • · rbd children <parent>@<snap> shows the clone
Diagnosis and resolutionclick to reveal

Root cause

A clones parent snapshot was deleted (against best practice); the clone cannot read the dependent data.

Remediation

Do not delete the parent snapshot. Restore the parent from backup. If the parent is gone, flatten the clone.

Verification

The clone is in a consistent state; the application can read and write the clone; data is consistent.

Prevention

Always protect the parent snapshot (rbd snap protect) before cloning. Never delete a parent snapshot while a clone depends on it.

Architecture

The student examines the broken architecture diagram and identifies the responsible component. The cluster is running Ceph Tentacle (20.2.x); the incident is one a production operator must diagnose from the evidence presented.

Symptoms

  • A clone reports parent mismatch
  • rbd flatten fails with image has snapshots
  • Clone I/O errors at the parent snapshot

Evidence

  • rbd info <image> shows the parent snapshot
  • rbd snap ls <parent> shows the snapshot exists
  • rbd children <parent>@<snap> shows the clone

Student investigation

The student follows the methodology: define the symptom, determine the impact, gather evidence, identify the component, form a hypothesis, test safely, restore, validate.

Progressive hints

  • Hint 1: check the cluster state with ceph -s first.
  • Hint 2: read ceph health detail and identify the affected PGs / OSDs / daemons.
  • Hint 3: use ceph osd tree, ceph pg dump, or ceph mds stat as the next-level diagnostic.
  • Hint 4: the root cause is documented in the frontmatter root_cause field.

Validation

The student runs the verification steps and confirms the symptom cleared.

Root cause

See the frontmatter root_cause field.

Remediation

Apply the fix from the frontmatter remediation field.

Prevention

Apply the prevention measures from the frontmatter prevention field.