The CSI architecture for Ceph

Why this matters in production

Ceph-CSI is several components with different responsibilities, and a storage problem in Kubernetes is usually located in exactly one of them.

The components

flowchart TD
  A[PVC created] --> B[csi-provisioner sidecar]
  B --> C[CSI controller plugin]
  C --> D[Ceph: create RBD image]
  E[Pod scheduled] --> F[csi-node plugin on that node]
  F --> G[Map RBD image, mount filesystem]
  G --> H[Pod starts]

kubectl -n ceph-csi get pods
kubectl -n ceph-csi get daemonset,deployment

What each failure affects

The split matters: a provisioner outage is invisible to running workloads and blocks all new ones, which is a very different urgency from a node plugin outage.

Locating a problem

# a PVC stuck Pending → provisioner
NAME=acme
NODE=stor-04
kubectl describe pvc ${NAME}
kubectl -n ceph-csi logs deploy/csi-rbdplugin-provisioner -c csi-rbdplugin

# a pod stuck ContainerCreating → node plugin on that node
kubectl describe pod ${NAME}
kubectl -n ceph-csi logs -l app=csi-rbdplugin --field-selector spec.nodeName=${NODE}

# everything failing → Ceph itself
ceph -s

The Kubernetes event on the object names the phase, and the phase names the component.

kubectl get events --sort-by=.lastTimestamp | grep -i -E 'volume|mount|attach'

Configuration

apiVersion: v1
kind: ConfigMap
metadata:
  name: ceph-csi-config
  namespace: ceph-csi
data:
  config.json: |
    [{
      "clusterID": "b3d5f2a1-...",
      "monitors": ["10.0.2.11:3300","10.0.2.12:3300","10.0.2.13:3300"]
    }]

The clusterID is the Ceph cluster’s fsid and links the StorageClass to this configuration entry.

ceph fsid
kubectl -n ceph-csi get cm ceph-csi-config -o jsonpath='{.data.config\.json}'

Quiz

Production discipline

Read the failing phase from the Kubernetes event before opening any logs — Pending means the provisioner Deployment and ContainerCreating with a mount error means the node plugin on that specific node. The two have separate logs and separate failure domains.

Cross-course references

• Kubernetes: the controller/node split appears in every CSI driver
• Linux: cluster-level provisioning versus node-local attachment is a general pattern