The Container Runtime Interface — kubelet-to-runtime contract

The Container Runtime Interface (CRI) is the gRPC API the kubelet calls to create and manage Pods. The kubelet knows the cluster’s desired state; the runtime knows the local containers. The CRI is the bridge between the two. This lesson walks the CRI’s gRPC services, the relationship between PodSandbox and Container, and the operational implications of the design.

The CRI’s gRPC services

The CRI defines two gRPC services:

service RuntimeService {
  rpc RunPodSandbox(RunPodSandboxRequest) returns (RunPodSandboxResponse);
  rpc StopPodSandbox(StopPodSandboxRequest) returns (StopPodSandboxResponse);
  rpc RemovePodSandbox(RemovePodSandboxRequest) returns (RemovePodSandboxResponse);
  rpc PodSandboxStatus(PodSandboxStatusRequest) returns (PodSandboxStatusResponse);
  rpc ListPodSandbox(ListPodSandboxRequest) returns (ListPodSandboxResponse);
  rpc CreateContainer(CreateContainerRequest) returns (CreateContainerResponse);
  rpc StartContainer(StartContainerRequest) returns (StartContainerResponse);
  rpc StopContainer(StopContainerRequest) returns (StopContainerResponse);
  rpc RemoveContainer(RemoveContainerRequest) returns (RemoveContainerResponse);
  rpc ContainerStatus(ContainerStatusRequest) returns (ContainerStatusResponse);
  rpc ListContainers(ListContainersRequest) returns (ListContainersResponse);
  rpc Exec(ExecRequest) returns (ExecResponse);
  rpc ExecSync(ExecSyncRequest) returns (ExecSyncResponse);
  // ...
}

service ImageService {
  rpc ListImages(ListImagesRequest) returns (ListImagesResponse);
  rpc ImageStatus(ImageStatusRequest) returns (ImageStatusResponse);
  rpc PullImage(PullImageRequest) returns (PullImageResponse);
  rpc RemoveImage(RemoveImageRequest) returns (RemoveImageResponse);
  // ...
}

The RuntimeService manages the Pods and containers. The ImageService manages the image cache. The two are independent; the kubelet can call either without blocking the other.

The kubelet connects to the runtime over a Unix socket:

kubelet --container-runtime=remote \
        --container-runtime-endpoint=unix:///run/containerd/containerd.sock

The default for containerd is /run/containerd/containerd.sock. The default for cri-dockerd (the Docker shim) is /var/run/dockershim.sock.

The PodSandbox and Container

The CRI distinguishes two concepts:

• PodSandbox: the container that holds the Pod’s network namespace, cgroup, and filesystem. The PodSandbox is the container that the kubelet runs the CNI plugin against. The PodSandbox is sometimes called the “pause container” because it pauses (does nothing) for the Pod’s lifetime.
• Container: the application container. The Container is added to the PodSandbox; the Container shares the PodSandbox’s network namespace and filesystem.

The two are independent at the API level. The kubelet creates the PodSandbox first, then creates the Containers and joins them to the PodSandbox.

sequenceDiagram
    autonumber
    participant K as kubelet
    participant R as Runtime

    K->>R: RunPodSandbox(network=ns,cgroup=parent)
    R-->>K: PodSandboxID
    K->>R: CreateContainer(PodSandboxID, image, mounts)
    R-->>K: ContainerID
    K->>R: StartContainer(ContainerID)
    R-->>K: Container started

The PodSandbox holds the namespace; the Containers run inside it. The kubelet is the orchestrator.

The kubelet-to-runtime flow

The kubelet’s startPod flow:

sequenceDiagram
    autonumber
    participant K as kubelet
    participant CNI as CNI plugin
    participant R as Runtime

    K->>R: RunPodSandbox(log_directory, ...)
    Note over R: create network namespace
    Note over R: create cgroup
    Note over R: start pause container
    R-->>K: PodSandboxID
    K->>CNI: ADD (Pod's namespace)
    CNI->>CNI: attach veth, assign IP
    CNI-->>K: network ready
    K->>R: CreateContainer(image, mounts, env)
    R-->>K: ContainerID
    K->>R: StartContainer(ContainerID)
    R-->>K: container started
    K->>R: ContainerStatus(ContainerID)
    R-->>K: Running

The kubelet is the orchestrator. The runtime is the worker. The CNI is the network agent.

The CRI’s proto definitions

The CRI’s proto definitions are in the k8s.io/cri-api package. The kubelet imports the package and uses the generated gRPC stubs. The runtime implements the gRPC server.

The CRI’s versioning is stable. The CRI v1 is the current version; the CRI v1alpha is deprecated. The kubelet supports v1, v1alpha2, and (legacy) v1alpha1.

The runtime’s CRI implementation is tested by the cri-tools project’s critest program. The program runs the runtime through the CRI’s conformance tests.

The PodSandbox’s metadata

The PodSandbox’s metadata is the kubelet’s interpretation of the Pod:

message PodSandboxMetadata {
  string name = 1;
  string namespace = 2;
  string uid = 3;
  uint32 attempt = 4;
}

The kubelet fills the metadata from the Pod’s spec. The runtime stores the metadata in the local state; the ListPodSandbox API returns the metadata for each PodSandbox.

The Container’s spec

The Container’s spec is the kubelet’s interpretation of the container:

message ContainerSpec {
  string image = 1;
  repeated Mount mounts = 2;
  repeated string env = 3;
  map<string, string> annotations = 4;
  // ...
}

The kubelet fills the spec from the Pod’s spec.containers[]. The runtime creates the container based on the spec.

The CRI’s spec is intentionally minimal. The runtime implements the spec; the runtime may add additional features (e.g., seccomp, apparmor) that are not in the CRI.

The runtime’s image cache

The runtime’s image cache is the same as the runtime’s local image store. The kubelet’s PullImage request causes the runtime to pull the image; the runtime stores the image in the cache.

The runtime’s cache is the largest directory on the node. The kubelet’s image GC uses the ListImages API to enumerate the cache and the RemoveImage API to remove images.

The runtime’s cache is shared across Pods. The same image layer is not stored twice; the runtime uses content-addressable storage.

The CRI’s exec and port-forward

The CRI supports the Exec and PortForward requests. The kubelet uses these to support kubectl exec and kubectl port-forward.

The Exec request returns a URL that the kubelet proxies to the operator. The ExecSync request runs the command and returns the output; the PortForward request returns a URL that the kubelet proxies.

The runtime’s exec implementation is the primary mechanism for kubectl exec. The kubelet proxies the operator’s request to the runtime’s exec endpoint.

The CRI’s resource metrics

The CRI exposes the container’s resource metrics via the ContainerStats API. The kubelet’s metrics-spy and the Metrics Server use the API to scrape the container’s metrics.

The ContainerStats request returns the container’s CPU, memory, and network usage. The metrics are sampled at the request time; the runtime may cache the metrics.

The CRI’s operational signals

The CRI’s signals:

• Runtime logs: the runtime’s logs include the CRI request and response. The operator can see the kubelet’s calls and the runtime’s replies.
• Runtime metrics: the runtime exposes the CRI metrics on its metrics endpoint. The metrics include the request count, the latency, and the error rate.
• Runtime status: the runtime’s Status API returns the runtime’s state. The kubelet uses the status to detect a failing runtime.

The operator’s primary signals are the runtime’s logs and the node’s status. The kubelet’s logs include the CRI errors; the node’s status reflects the kubelet’s state.

Quiz

Production discipline

• The CRI is the kubelet’s only window into the runtime. The kubelet calls the runtime via the CRI; the runtime is the only process that creates and manages containers on the node.
• The PodSandbox is the network namespace. The PodSandbox is created by the kubelet; the kubelet runs the CNI plugin against the PodSandbox. The PodSandbox is the isolation unit.
• The Container is the process. The Container is added to the PodSandbox; the Container shares the PodSandbox’s namespace. The Container is the execution unit.
• The runtime’s image cache is the largest directory on the node. The kubelet’s image GC uses the ListImages and RemoveImage APIs to manage the cache.
• The CRI’s logs are the primary diagnostic. The kubelet’s logs include the CRI errors; the runtime’s logs include the CRI requests. The operator should read both.
• Audit the CRI’s conformance at every release. A runtime that does not pass the CRI’s conformance tests is a runtime that does not behave correctly. The conformance test is the primary signal.