The CSI gRPC protocol — the wire format, errors, and idempotency

CSI uses gRPC for communication between Kubernetes and storage drivers. The protocol is well-defined: services, methods, messages, status codes, and idempotency. This lesson walks the wire format and the production discipline.

The three services

CSI defines three gRPC services:

service Identity {
  rpc GetPluginInfo(GetPluginInfoRequest) returns (GetPluginInfoResponse);
  rpc GetPluginCapabilities(GetPluginCapabilitiesRequest) returns (GetPluginCapabilitiesResponse);
  rpc Probe(ProbeRequest) returns (ProbeResponse);
}

service Controller {
  rpc CreateVolume(CreateVolumeRequest) returns (CreateVolumeResponse);
  rpc DeleteVolume(...) ...
  rpc ControllerPublishVolume(...) ...
  rpc ControllerUnpublishVolume(...) ...
  rpc ValidateVolumeCapabilities(...) ...
  rpc ListVolumes(...) ...
  rpc GetCapacity(...) ...
  rpc CreateSnapshot(...) ...
  rpc DeleteSnapshot(...) ...
  rpc ListSnapshots(...) ...
  rpc ControllerExpandVolume(...) ...
  rpc ControllerGetVolume(...) ...
}

service Node {
  rpc NodeStageVolume(...) ...
  rpc NodeUnstageVolume(...) ...
  rpc NodePublishVolume(...) ...
  rpc NodeUnpublishVolume(...) ...
  rpc NodeGetVolumeStats(...) ...
  rpc NodeExpandVolume(...) ...
  rpc NodeGetCapabilities(...) ...
  rpc NodeGetInfo(...) ...
}

The Identity service provides metadata about the driver. The Controller service runs cluster-wide operations. The Node service runs per-node operations.

The request/response messages

Each RPC has a request and response message. The messages are defined in protobuf:

message CreateVolumeRequest {
  string name = 1;
  CapacityRange capacity_range = 2;
  VolumeCapabilities volume_capabilities = 3;
  map<string, string> parameters = 4;
  map<string, string> secrets = 5;
  Topology topology_requirement = 6;
  AccessibilityRequirements accessibility_requirements = 7;
}

message CreateVolumeResponse {
  Volume volume = 1;
}

The name field is the CSI-internal name (typically the PVC’s UID). The capacity_range is min and max bytes. The parameters are StorageClass parameters. The secrets are credentials (passed via Secret references, not directly).

gRPC status codes

CSI uses standard gRPC status codes for errors:

The Kubernetes sidecar (e.g., csi-provisioner) interprets the codes and translates them to Kubernetes events and PVC status.

Idempotency

CSI operations are required to be idempotent:

“The CSI spec requires that all CSI operations be idempotent. This means that an operation can be called multiple times without changing the result beyond the initial application.”

The implementation:

• CreateVolume: the name field identifies the volume. Calling CreateVolume twice with the same name returns the same volumeHandle.
• DeleteVolume: deleting an already-deleted volume returns success.
• NodeStageVolume: staging an already-staged volume is a no-op.
• NodePublishVolume: publishing an already-published volume is a no-op.

Idempotency is critical because Kubernetes reconciles state. The kubelet may call NodeStageVolume multiple times for the same volume; the CSI driver must converge to the desired state without failing.

flowchart LR
    A[kubelet calls NodeStageVolume] --> B{CSI driver checks state}
    B -->|already staged| C[Return OK without action]
    B -->|not staged| D[Format and mount]
    D --> E[Return OK]

Volume capabilities

The VolumeCapability message describes what the volume can do:

message VolumeCapability {
  oneof access_type {
    BlockVolume block = 1;
    FilesystemVolume mount = 2;
  }
  AccessMode access_mode = 3;
}

message FilesystemVolume {
  string fs_type = 1;
  repeated string mount_flags = 2;
}

message AccessMode {
  enum Mode {
    UNKNOWN = 0;
    SINGLE_NODE_WRITER = 1;        // RWO
    SINGLE_NODE_READER_ONLY = 2;   // ROX
    MULTI_NODE_READER_ONLY = 3;    // ROX
    MULTI_NODE_SINGLE_WRITER = 4;  // RWX
    MULTI_NODE_MULTI_WRITER = 5;   // RWX
    SINGLE_NODE_SINGLE_WRITER = 6; // RWOP
    SINGLE_NODE_MULTI_WRITER = 7;  // RWO (multiple writers on same node)
  }
}

The CSI spec maps Kubernetes access modes to CSI access modes:

• RWO -> SINGLE_NODE_WRITER
• ROX -> SINGLE_NODE_READER_ONLY or MULTI_NODE_READER_ONLY
• RWX -> MULTI_NODE_SINGLE_WRITER or MULTI_NODE_MULTI_WRITER
• RWOP -> SINGLE_NODE_SINGLE_WRITER

The production error handling

A CSI driver in production must handle:

• Network failures: retry with backoff.
• Backend throttling: back off; respect the rate limit.
• Parameter validation: return INVALID_ARGUMENT with a specific message.
• Volume not found: return NOT_FOUND; the kubelet will trigger reconciliation.
• Precondition failures: return FAILED_PRECONDITION (e.g., the volume is in use).

The Kubernetes sidecar (csi-provisioner, csi-snapshotter) translates the gRPC codes to Kubernetes events. The operator reads the events to diagnose.

Quiz

Production discipline

• Verify the driver is idempotent. Test every operation by calling it twice; the result must be the same.
• Use the gRPC status codes correctly. INVALID_ARGUMENT for parameter errors; PERMISSION_DENIED for IAM/RBAC; NOT_FOUND for missing volumes.
• Test the error paths. A driver that handles success but fails on edge cases is a production liability.
• Run the CSI conformance suite. The Kubernetes-CSI organization provides conformance tests that validate drivers against the spec.
• Document the driver’s gRPC errors. Operators who see gRPC codes must be able to translate them to Kubernetes events.