Runtime migration history — Docker, dockershim, and the path to containerd

The container runtime in Kubernetes has evolved from Docker to dockershim to cri-dockerd to containerd. The migration was a major cluster change that touched every node. This lesson walks the history, the deprecation timeline, the migration path, and the operational patterns.

The Docker era

Kubernetes’ original container runtime was Docker. The kubelet called Docker’s API to create and manage containers. The Docker API was not the CRI; the kubelet had a shim layer (dockershim) that translated the kubelet’s CRI calls to Docker’s API calls.

The architecture:

flowchart LR
    A[kubelet] -->|CRI| B[dockershim]
    B -->|Docker API| C[dockerd]
    C -->|containerd| D[Container]

The dockershim was a piece of the kubelet that translated the CRI to the Docker API. The dockershim was deprecated in Kubernetes 1.20 and removed in 1.24.

Why Docker was deprecated

Docker was deprecated as a Kubernetes runtime for several reasons:

• The CRI is the standard. Docker did not implement the CRI; the dockershim was a custom shim. The CRI is the standard API for container runtimes; Docker’s decision to not implement the CRI made Docker a special case.
• Docker is a developer tool, not a runtime. Docker is designed for developers; the production runtime should be purpose-built for the kubelet’s use case. containerd was extracted from Docker in 2015 and is designed as a runtime.
• The kubelet’s dependency on Docker was a coupling. The kubelet should depend on the CRI, not on a specific runtime. The dockershim was a workaround; the CRI is the right answer.

The deprecation was announced in 1.20 (December 2020). The removal was in 1.24 (May 2022). The migration gave cluster operators two years to move to a CRI-compliant runtime.

The cri-dockerd era

After the dockershim was deprecated, the Docker community released cri-dockerd, a CRI-compliant shim that uses Docker as the runtime. The cri-dockerd allows Docker to be used as a Kubernetes runtime without the dockershim.

The architecture:

flowchart LR
    A[kubelet] -->|CRI| B[cri-dockerd]
    B -->|Docker API| C[dockerd]
    C -->|containerd| D[Container]

The cri-dockerd is a separate process that runs alongside the kubelet. The kubelet calls the cri-dockerd’s CRI endpoint; the cri-dockerd translates to Docker’s API.

The cri-dockerd is maintained by the Docker community. It is a bridge for clusters that want to keep Docker as the runtime. The cri-dockerd is not the production-default runtime; containerd is.

The containerd era

containerd is the production-default runtime. The kubelet calls containerd directly via the CRI. The containerd uses runc as the runtime; the snapshotter is overlayfs.

flowchart LR
    A[kubelet] -->|CRI| B[containerd]
    B -->|runc| C[Container]

The containerd is a CNCF graduated project. It is the default runtime for Kubernetes distributions (kubeadm, EKS, GKE, AKS, OpenShift). The containerd’s CRI plugin is built into the containerd binary; no separate shim is required.

The deprecation timeline

The migration was a hard deadline. Clusters that did not migrate by 1.24 were unable to upgrade beyond 1.24.

The migration path

The migration from Docker to containerd is a node-level operation. The process:

1. Drain the node. kubectl drain <name> evicts all Pods and cordons the node.
2. Stop the kubelet. The kubelet must be stopped before the runtime is changed.
3. Stop Docker. systemctl stop docker.
4. Install containerd. The containerd binary is installed; the configuration is created.
5. Update the kubelet’s flags. The kubelet’s --container-runtime-endpoint flag is changed to /run/containerd/containerd.sock.
6. Start containerd. systemctl start containerd.
7. Start the kubelet. The kubelet connects to containerd.
8. Verify the runtime is healthy. The kubelet’s metrics expose the runtime status.
9. Uncordon the node. kubectl uncordon <name>.

The migration is disruptive to the node’s Pods. The drain ensures the Pods are evicted before the migration.

The containerd’s CRI plugin

The containerd’s CRI plugin is built into the containerd binary. The plugin is enabled by default in the containerd configuration:

[plugins."io.containerd.grpc.v1.cri"]
  sandbox_image = "registry.k8s.io/pause:3.9"
  containerd_snapshotter = "overlayfs"

The plugin implements the CRI’s gRPC server. The kubelet calls the CRI’s gRPC server over the Unix socket.

The plugin’s configuration is the runtime’s knobs:

• sandbox_image: the pause container’s image.
• containerd_snapshotter: the snapshotter for the container’s filesystem.
• disable_apparmor: whether to disable the AppArmor profile.
• disable_cgroup: whether to disable the cgroup configuration.
• disable_hugetlb_controller: whether to disable the hugetlb controller.

The migration’s challenges

The migration’s challenges:

• Image cache. The image cache is not shared between Docker and containerd. The migration requires re- pulling the images. The cluster’s image cache is reset.
• Runtime configuration. The runtime configuration is different. The containerd’s configuration file is different from Docker’s daemon.json.
• Logs. The container logs are in different locations. The containerd uses /var/log/pods/ for the structured logs; Docker uses /var/lib/docker/containers/.
• CNI plugin. The CNI plugin’s configuration is the same, but the runtime’s CNI integration is different. The CNI plugin’s binary path may differ.

The migration is a node-level operation. The cluster must be migrated one node at a time.

The current state

The current state of the container runtime in Kubernetes 1.34:

• containerd is the default. Most Kubernetes distributions use containerd.
• CRI-O is the alternative. Red Hat’s OpenShift uses CRI-O.
• cri-dockerd is the bridge. Docker is still supported via cri-dockerd, but the production- default is containerd.

The future:

• Kata and gVisor are gaining adoption for security-isolated workloads.
• The CRI is stable. The CRI v1 is the current version.

Quiz

Production discipline

• containerd is the production-default runtime. The kubelet calls containerd directly via the CRI. The production rule is to use containerd.
• The migration is a node-level operation. The cluster’s control plane is unaffected; the kubelet’s configuration on each node is changed.
• Audit the runtime at every node repave. A new node that joins the cluster with the wrong runtime is a node that is failing silently. Validate the runtime at bootstrap.
• Document the runtime’s configuration. The containerd’s configuration file is the runtime’s source of truth. The configuration should be version-controlled and applied to every node.
• Monitor the runtime’s metrics. The runtime’s metrics expose the runtime’s health. The operator should alert on the error rate and the resource usage.
• Test the runtime in non-production. A staging cluster that mirrors production is the right place to test runtime changes.