Skip to main content
RunBook Academy

Docker & ContainersXXIX Β· Docker UpgradesExecution

The in-place daemon upgrade β€” what happens to running containers

Advanced⏱ ~26 mindockersystemd

What you'll learn

  • Predict whether a given upgrade interrupts running containers
  • Explain why KillMode=process is what makes live restore possible
  • Execute a version-pinned upgrade rather than an unbounded one
  • Verify after the upgrade that containers actually survived

Prerequisites

Verified against Docker Engine 29.x Β· Docker Engine 28.x Β· Docker Compose 2.x Β· containerd 2.x Β· runc 1.2.x Β· BuildKit 0.20+ Β· Linux kernel 5.15+ Β· Ubuntu 24.04 LTS Β· Debian 12 (Bookworm) Β· 2026-08-11

Not yet marked complete on this device.

There is no such thing as installing a new Docker package without restarting the daemon. The postinstall script does it, unconditionally, before it returns control to you.

Whether that restart is a brief gap in the API or an outage of every container on the host depends on two things that were decided long before the upgrade started. This lesson is about both of them, and about the command sequence that makes the outcome predictable.

The postinstall restarts the unit

The Debian and Ubuntu packages carry the standard dh_installsystemd scaffolding, which on an upgrade runs a restart rather than a start:

if [ "$1" = "configure" ] || [ "$1" = "abort-upgrade" ] ... ; then
        if [ -d /run/systemd/system ]; then
                systemctl --system daemon-reload >/dev/null || true
                if [ -n "$2" ]; then
                        _dh_action=restart
                else
                        _dh_action=start
                fi
                deb-systemd-invoke $_dh_action 'docker.service' 'docker.socket' >/dev/null || true
        fi
fi

$2 is the previously configured version. On a first install it is empty and the action is start; on an upgrade it is set and the action is restart. The containerd.io package carries an identical block for containerd.service.

Two consequences:

  1. There is no β€œinstall now, restart later” mode. Downloading the package and installing it are separable (apt-get download, --download-only), but dpkg configuring it is the restart.
  2. apt-get upgrade restarts the daemon. Any unattended patching job that can reach these packages can restart Docker without a change ticket.

Why containers can survive it

Live restore: the boolean that decides the blast radius

Read-only / Safecheck first
docker info --format 'live-restore={{.LiveRestoreEnabled}}'

It is off by default. A host where nobody enabled it takes a full container outage on every daemon upgrade.

Configuration change/etc/docker/daemon.json
{
"live-restore": true
}

Enabling it is itself a daemon restart β€” and the restart that turns it on is not covered by it. Do that once, in its own window, well before the upgrade you care about.

What live restore covers, precisely

UpgradeContainers survive?
Patch release, 28.3.1 to 28.3.2Yes β€” this is what it is designed for
Feature release, 28.2.x to 28.3.xUsually, but not guaranteed
Line upgrade, 28.x to 29.xNo β€” do not rely on it
Skipping releases in one stepNo β€” the daemon may fail to reconnect
Any upgrade that also changes daemon.jsonNo β€” see below

Docker’s own documentation states that live restore supports patch releases and that skipping releases during an upgrade may leave the daemon unable to restore its connection to the containers.

The upgrade sequence

Service impact possiblepinned upgrade
#!/usr/bin/env bash
set -euo pipefail

TARGET='5:28.3.2-1~ubuntu.24.04~noble'

# --- Before
docker info --format 'live-restore={{.LiveRestoreEnabled}}'
docker ps --format '{{.Names}} {{.Status}}' | tee /tmp/containers.before
docker inspect --format '{{.Name}} {{.State.StartedAt}}' $(docker ps -q) | tee /tmp/started.before

# --- Refresh metadata only; this does not install anything
sudo apt-get update

# --- Confirm the target version exists for this distribution
apt-cache madison docker-ce | grep -F "$TARGET" || { echo "target version not available" >&2; exit 1; }

# --- Install exactly these versions. The daemon restarts inside this step.
sudo apt-get install -y "docker-ce=$TARGET" "docker-ce-cli=$TARGET" docker-buildx-plugin docker-compose-plugin containerd.io

# --- After
systemctl is-active docker
docker version
docker ps --format '{{.Names}} {{.Status}}' | tee /tmp/containers.after
docker inspect --format '{{.Name}} {{.State.StartedAt}}' $(docker ps -q) | tee /tmp/started.after

# The proof: StartedAt unchanged means the container was never restarted
diff /tmp/started.before /tmp/started.after && echo 'PASS: no container was restarted' || echo 'WARN: at least one container restarted during the upgrade'

The StartedAt comparison is the whole verification. docker ps showing containers Up after the upgrade proves nothing β€” a container that was killed and restarted by its restart policy also shows Up. Only the start timestamp distinguishes β€œsurvived” from β€œrestarted quickly”.

Read-only / Safethe proof
$ diff /tmp/started.before /tmp/started.after && echo 'PASS'
PASS

Illustrative output

What is interrupted regardless

Even with live restore working perfectly, some things stop for the duration of the daemon restart:

  • The API. docker ps, docker logs, docker exec β€” anything that goes through the socket returns Cannot connect to the Docker daemon for a few seconds. Monitoring that scrapes the Docker API will register a gap and may alert.
  • In-flight docker exec sessions. Dropped. An interactive debugging session in another terminal disconnects.
  • docker logs -f followers. Dropped. Log shippers using the Docker API reconnect; log shippers reading the JSON files directly do not notice.
  • Builds. A build in progress fails.
  • Healthcheck evaluation. Paused for the restart window, then resumes. A container mid-way through its start-period may report differently afterwards.
  • Container restart policies. Not evaluated while the daemon is down. A container that exits during the gap stays exited until the daemon returns.

Knowledge check

Knowledge check Β· 4 questions

  1. Q1. Which systemd directive in docker.service is what allows container processes to survive a daemon restart?

  2. Q2. After an upgrade, `docker ps` shows every container as `Up`. What does that prove about whether they survived?

  3. Q3. With live restore working correctly, which of these are still interrupted during a daemon upgrade? Select all that apply.

  4. Q4. Live restore is reliable across a line upgrade such as 28.x to 29.x.

Passing score: 75%. Answers are checked in this browser.