Orchestration — when Compose is not enough

Single-host Docker Compose runs a great deal of production software, well, for years. It also has a boundary, and the boundary is sharper than the usual “when you need to scale” phrasing suggests.

This lesson does not argue for or against orchestration. It states what is on each side of the line and what crossing it costs, so that the decision is made against a requirement you can write down.

What a single Docker host structurally cannot do

Four capabilities. Each is absent for a structural reason, not because someone has not implemented it yet.

1. Reschedule a workload when the host fails

Docker’s restart policies (always, unless-stopped, on-failure) are implemented by dockerd on the host the container is on. When the container dies, the daemon restarts it. When the daemon dies, live-restore keeps containers running but nothing restarts them. When the host dies, there is no process left anywhere that knows the container should exist.

This is not a gap that better configuration closes. Rescheduling requires something outside the failed host to (a) hold the desired state, (b) detect the failure, and (c) have authority to start work elsewhere. That thing is a scheduler with a cluster-wide store, and it is the defining feature of an orchestrator.

The practical consequence: on a single Docker host, the recovery time for a host failure is however long it takes a human to notice and act. If your target is minutes, you need something else. If your target is “a few hours, during business hours”, a single host with good backups genuinely meets it, and much of the industry pretends otherwise.

2. Perform a rolling update as a primitive

docker compose up with a changed image recreates the affected containers. On a single host with one replica of a service, recreation is a gap in service — short, but real, and it happens whether or not the new container turns out to work.

What an orchestrator provides instead is a rollout: start the new instance, wait for it to pass its health check, shift traffic, stop the old one, and if the new one never becomes healthy, stop and roll back automatically. Every clause there requires more than one instance and something that controls traffic between them.

3. Distribute secrets across a cluster

Compose supports secrets:, and on a single host they are files. The Compose file points at a path, Docker mounts it into the container at /run/secrets/<name>, and the file lives on that host’s disk, placed there by you.

That is a perfectly reasonable secret mechanism for one host. It does not extend: there is no mechanism in standalone Docker for a secret to exist once and be delivered, encrypted in transit and at rest, to whichever host a workload happens to land on. Swarm has one (secrets are stored in the Raft log, encrypted, and delivered to the tasks that need them). Kubernetes has one. A directory of files copied around by Ansible is not one, though it is what most single-host deployments actually use — and, for a small number of hosts, it is defensible as long as you are honest that the secrets are on disk on every host that ever ran the workload.

4. Provide cross-host container networking and service discovery

Docker’s embedded DNS resolves service names within a user-defined network on one daemon. Two containers on two hosts cannot reach each other by name; there is no shared network namespace and no shared DNS.

You can build around this — publish ports and route by IP, put a proxy in front, use a service registry — and people do. But the moment a container can move between hosts, addressing it by host and port stops working, and you are building service discovery. Overlay networking exists precisely because that build-around does not scale.

What Compose can do that people assume it cannot

Several of the reasons cited for “outgrowing Compose” are not actually limits. Being clear about these avoids a migration that solves nothing.

The last row is the one worth dwelling on. Two replicas behind a proxy that removes unhealthy backends is a rolling update. It is manual, it is specific to your setup, and it works. Many teams migrate to Kubernetes to obtain a property they could have had from a Compose file with replicas: 2 and a proxy configured to health-check.

The cost of an orchestrator

This is the part usually left out of the comparison table, and it is not a one-time migration cost. It is a recurring operational load.

The options, described rather than ranked

flowchart TB
  A["One host, Compose"]
  B["Two hosts, Compose + LB<br/>manual failover"]
  C["Docker Swarm"]
  D["HashiCorp Nomad"]
  E["Kubernetes"]
  A --> B
  A --> C
  B --> C
  C --> E
  A --> D
  D --> E

The arrows are the transitions people actually make; note that Swarm to Nomad is not one of them, and that plenty of teams go straight from one host to Kubernetes because that is what their platform team already runs.

Docker Swarm

Built into the engine. docker swarm init and you have a cluster.

# On the first manager
docker swarm init --advertise-addr 192.0.2.10

# Read the join token for workers
docker swarm join-token -q worker

# On each worker, with the token from above
WORKER_TOKEN=REPLACE_ME
docker swarm join --token "$WORKER_TOKEN" 192.0.2.10:2377

# Back on a manager: deploy the stack
docker stack deploy --compose-file compose.yml myapp

# Watch the rollout, which is the thing Compose could not do
docker service ps myapp_web
docker service inspect myapp_web \
--format '{{json .Spec.UpdateConfig}}'

What it genuinely gives you: rescheduling on node failure, rolling updates with update_config and automatic rollback, encrypted overlay networking, secrets in the Raft store, and routing-mesh load balancing — with a Compose file you largely already have and no components to install.

The honest caveats: feature development is slow and has been for years; the community and third-party ecosystem are small, so you will find fewer answers to unusual problems; and the pool of people who know it is shrinking, which is a real staffing consideration. It is maintained, not abandoned, and for a team that wants multi-host failover and nothing else it remains the lowest-cost way to get it.

HashiCorp Nomad

A general scheduler: containers, raw binaries, JVM applications and QEMU guests through one control plane. Notably simpler to operate than Kubernetes — a single binary that is both server and client.

job "myapp" {
  datacenters = ["dc1"]
  type        = "service"

  group "app" {
    count = 3

    # Ports are declared here as LABELS, then referenced by name in the
    # task config. `ports = ["8080"]` in the task is not a port number
    # and will not work.
    network {
      port "http" {
        to = 8080
      }
    }

    service {
      name     = "myapp"
      port     = "http"
      provider = "nomad"

      check {
        type     = "http"
        path     = "/healthz"
        interval = "10s"
        timeout  = "2s"
      }
    }

    task "server" {
      driver = "docker"

      config {
        image = "registry.example.com/myorg/myapp:1.0.0"
        ports = ["http"]
      }

      resources {
        cpu    = 500
        memory = 512
      }
    }
  }
}

Nomad allocates a host port dynamically for each http label and exposes it to the task as NOMAD_PORT_http; to = 8080 maps it to a fixed port inside the container. Reach for Nomad when your workloads are genuinely heterogeneous, or when you want scheduling without the Kubernetes operating model.

Kubernetes

The largest feature set, the largest ecosystem, and the largest operational surface. The reasons to choose it that hold up:

• Your organisation already runs it, and a second platform costs more than the mismatch.
• You need something only its ecosystem provides — operators for stateful systems, a specific service mesh, cluster autoscaling against a cloud API.
• Managed control plane (EKS, GKE, AKS) removes the hardest part of the operational cost, and the remainder is acceptable.

The reason that does not hold up is “it is the standard”. It is the standard for a class of problem, and running one service on three hosts is not that class.

The decision, in the form of questions

1. What is your actual recovery time objective for a host failure? Write the number down. If it is hours, a single host with tested restores meets it and an orchestrator is optional.
2. How often do you deploy? Rolling updates matter in proportion to deploy frequency. Twice a month rarely justifies a control plane.
3. Is the workload stateless? If not, orchestration multiplies the storage problem rather than solving it. Fix state first.
4. Who operates the control plane at 03:00? Name the person. If there is no answer, a managed control plane or no control plane are the honest options.
5. Would two hosts and a load balancer meet the requirement? For availability alone, very often yes, at a fraction of the operational cost.
6. What does the organisation already run? A second platform has a cost that no technical comparison captures.

$ docker compose ps --format '{{.Service}} {{.State}} {{.Health}}'

web running healthy
worker running healthy
db running healthy

If that output is stable for months at a time and a deploy is a five-minute planned event that nobody notices, the case for an orchestrator has to come from somewhere other than reliability. That is a legitimate place for it to come from — a platform team, a compliance requirement, a hiring strategy — but it should be named rather than dressed up as a technical necessity.

Knowledge check