Private registries — Distribution, Harbor, GitLab, Quay · Docker & Containers

A private registry is the foundation of a production Docker supply chain, and it is also a stateful service sitting directly on the deploy path. Those two facts pull in different directions: the first argues for features, the second argues for something you can restore from a tarball at 03:00.

The four common options

CNCF Distribution — the reference open-source registry, and the code most of the others are built on. Minimal: no UI, no RBAC beyond what you put in front of it, no scanning, no replication. Roughly one process and a storage backend.
Harbor — the CNCF graduated registry. Distribution plus a Postgres database, a UI, RBAC, scanning, replication, retention policies and signing policy enforcement. Substantially more moving parts.
GitLab Container Registry — bundled with GitLab. Namespaces track projects, permissions track GitLab permissions, CI integration is free.
Quay — Red Hat’s registry. Clair scanning, geo-replication, mirroring from public registries, robot accounts. Available hosted (Quay.io) or self-hosted.

The honest selection criterion is not the feature matrix. It is: how many components must be restored, in what order, for a pull to work again?

	Components holding state	Restore order
Distribution	Storage backend only	One step
GitLab registry	Registry storage + GitLab database	Database, then storage
Harbor	Registry storage + Postgres + Redis + config	Database and storage together, then services
Quay	Registry storage + database + config	Database and storage together

Distribution’s minimalism is a feature on a bad day. Harbor’s policy enforcement is a feature on every other day. Pick according to which day you are more worried about, and know the answer before you need it.

A Distribution registry you can reason about

Configuration changeminimal registry— Distribution 3.x. The 3.0 release moved the default config path to /etc/distribution/config.yml; on 2.x it was /etc/docker/registry/config.yml.

docker run -d \
--name registry \
--restart always \
-p 5000:5000 \
-v /srv/registry:/var/lib/registry \
registry:3

That single bind mount is the entire state of the service. Everything a Distribution registry knows — every layer, every manifest, every tag — lives under it. The container itself is disposable.

Configuration changea config worth having— Delete must be enabled explicitly, or manifest deletion returns 405 and no retention policy can work.

version: 0.1
log:
level: info
storage:
filesystem:
  rootdirectory: /var/lib/registry
delete:
  enabled: true
maintenance:
  readonly:
    enabled: false
http:
addr: :5000
headers:
  X-Content-Type-Options: [nosniff]

Two keys there deserve attention now because they matter later:

storage.delete.enabled defaults to off. With it off, the registry accepts pushes forever and refuses every delete, so retention is impossible and the disk fills. Every operator discovers this the first time they try to remove an image.
storage.maintenance.readonly.enabled is the switch that makes garbage collection safe. It is off here; it must be on while GC runs. That is a whole lesson of its own, and it starts here in the config.

Every key has an environment-variable equivalent formed by upper-casing the path with underscores, which is how you configure the container without maintaining a file:

Configuration changethe same setting as an environment variable— REGISTRY_ + the config path, upper-cased, underscore-separated.

docker run -d --name registry \
-e REGISTRY_STORAGE_DELETE_ENABLED=true \
-p 5000:5000 \
-v /srv/registry:/var/lib/registry \
registry:3

Storage backends

The filesystem driver is the right default for a single registry with a real backup. It stops being right the moment you want two registry replicas, because two Distribution processes over one shared filesystem is a correctness question about that filesystem, not about Distribution.

For anything horizontally scaled, use an object-store driver — S3, Azure Blob, GCS. The registry becomes stateless, replicas become trivial, and the durability problem moves to a service that already solved it. The cost is latency on every blob operation and a bill that scales with pull traffic.

Probing a registry properly

“The registry is up” usually means “the TCP port answered”, which tells you almost nothing. The API has a defined entry point: a GET /v2/ must return 200 OK when the registry is serving and authentication is not required, or 401 Unauthorized when it is. Anything else is a fault.

Read-only / SafeAPI liveness— 200 or 401 both mean the registry API is alive. A 502 means your proxy is up and the registry is not.

REGISTRY=https://registry.example.com

curl -fsS -o /dev/null -w '%{http_code}\n' "$REGISTRY/v2/" || true

Illustrative output

That is liveness. It is not readiness, because a registry can answer /v2/ perfectly while its storage backend is unreachable. The check that actually corresponds to “a deploy will work” is a manifest fetch for something you know exists:

Read-only / Safereadiness — can a real pull resolve?— Fetches a known manifest by digest and compares the returned Docker-Content-Digest. This exercises auth, the tag index and the storage backend.

REGISTRY=registry.example.com
REPO=myorg/myapp
DIGEST=sha256:REPLACE_ME
ACCEPT='application/vnd.oci.image.index.v1+json,application/vnd.docker.distribution.manifest.list.v2+json'

curl -fsSI -H "Accept: $ACCEPT" \
"https://$REGISTRY/v2/$REPO/manifests/$DIGEST" \
| grep -i 'docker-content-digest'

docker-content-digest: sha256:REPLACE_ME

Illustrative output

If the returned digest does not match the one you requested, stop and investigate — you are talking to something that is not serving the content you asked for.

TLS, and the flag you should not reach for

A registry must be served over TLS. When the certificate is wrong, the daemon fails the pull with an x509 error, and there is a tempting configuration key that makes the error go away:

insecure-registries is daemon-wide and permanent

{
  "insecure-registries": ["registry.example.com"]
}

This does not “allow a self-signed certificate for this one pull”. It tells the daemon to stop verifying TLS for that host entirely, for every user of that daemon, for every image, until somebody removes the line — and it also permits a plain HTTP fallback. Anyone who can position themselves on that network path can now serve you images.

The correct fix for a private certificate authority is to trust the CA, not to stop checking. Place the CA certificate at /etc/docker/certs.d/registry.example.com/ca.crt and the daemon will verify against it normally. For a registry on a non-standard port, the directory name includes the port.

Read-only / Safewhat the daemon believes— Confirms no insecure registry has been left configured, and shows which mirrors are in effect.

docker info --format 'insecure: {{.RegistryConfig.IndexConfigs}}
mirrors: {{.RegistryConfig.Mirrors}}'

insecure: map[docker.io:0x40001a2000]
mirrors: []

Illustrative output

Harbor, GitLab and Quay in one paragraph each

Harbor is the standard self-hosted choice when you want policy enforced rather than documented: per-project RBAC, tag retention rules (configured per project in the UI or API — not in harbor.yml), Trivy-backed scanning with a “block pull if severity above X” gate, and replication to another Harbor or to a public registry. The operational cost is a Postgres database that is now on your restore path, and an upgrade procedure with more steps than “pull the new image”.

Configuration changeHarbor offline installer— Set the version explicitly; a placeholder in angle brackets is input redirection in bash, not a prompt to substitute.

HARBOR_VERSION=v2.15.0
cd /opt
curl -fsSLO "https://github.com/goharbor/harbor/releases/download/${HARBOR_VERSION}/harbor-offline-installer-${HARBOR_VERSION}.tgz"
tar xzf "harbor-offline-installer-${HARBOR_VERSION}.tgz"
# edit harbor/harbor.yml: hostname, TLS paths, database password
sudo ./harbor/install.sh

GitLab Container Registry is the lowest-marginal-cost option if you already run GitLab: a project’s registry namespace, permissions and CI credentials all follow from the project itself. Configuration lives in gitlab.rb:

registry_external_url 'https://registry.example.com'
gitlab_rails['registry_enabled'] = true

The trade is coupling. A GitLab outage or a GitLab upgrade window is now a registry outage or a registry upgrade window, and your deploys were probably not scoped into that maintenance notice.

Quay brings Clair scanning, geo-replication, repository mirroring from upstream registries, and robot accounts designed for automation. Signing in the modern Quay/OCI world is Cosign and OCI referrers; Notary v1 is being retired and its successor work moved to the Notation project, so treat any documentation referring to “Notary v2” as referring to Notation.

Replication

Replication solves two different problems and the design differs by which one you have:

Latency. Consumers far from the registry pay a network tax on every pull. A read-only replica or pull-through cache close to them fixes it, and a cache is usually enough.
Availability. If the registry disappearing must not stop deploys, you need a second registry that is authoritative, not a cache — and therefore a decision about how clients fail over to it.

Harbor offers scheduled, optionally bidirectional replication. Quay offers asynchronous geo-replication. Distribution offers neither; you put a pull-through cache in front of it or you copy images yourself with docker buildx imagetools create, which transfers the whole multi-platform index without pulling layers to the local host.

What is actually on the disk

Under /var/lib/registry/docker/registry/v2 there are exactly two trees, and understanding the split explains most registry behaviour.

blobs/ is content-addressed storage. Every layer, every image config and every manifest body is stored once, under a path derived from its own SHA-256. Two repositories that share a base layer share one file. Nothing here knows its own name.

repositories/<name>/ is the naming layer. It contains _manifests/tags/<tag>/current/link — a small file whose contents are a digest — plus _manifests/revisions/ recording every manifest digest this repository has held, and _layers/ linking the repository to the blobs it references.

A tag is therefore a file containing a digest. Re-pointing a tag rewrites that file. Nothing else changes, nothing is versioned, and the previous digest remains in revisions/ where nothing will look at it unless you go and read it.

Two consequences follow directly:

A backup of blobs/ alone is not a backup. You would have every byte of every image and no way to name any of them. This is the most common registry backup defect.
Deleting a tag does not free space. It removes a link file. The blobs stay until garbage collection walks the repository tree, marks everything still reachable, and sweeps the rest — which is why GC needs the registry read-only, and why the phrase “the registry is full” is usually a statement about GC rather than about images.

What happens when the registry is unreachable?

Be precise about this, because the blast radius is much smaller than people assume and the parts that do break are the parts that matter at the worst time.

Unaffected. Every running container. The daemon does not contact a registry to keep a container running, and a docker restart of an existing container does not pull. Local builds whose base images are cached are also fine.

Broken immediately. Any pull of an image not already on the host. That means a new deployment, a new host, a scaled-out node, and any Compose service with pull_policy: always. Also every docker build whose FROM is not in the local cache — including builds that looked cached because BuildKit had the layers but still needed to resolve the tag.

Broken silently. Anything that treats a pull failure as non-fatal. Some deployment tooling falls back to the local image and reports success, which means an outage produces a fleet running mixed versions rather than an error.

The one that hurts. Rollback. The rollback procedure says “deploy the previous tag”, the previous image was pruned off the host last Tuesday, and the registry is the thing that is down. This is the scenario that argues for a host-side image retention policy separate from anything the registry does.

Mitigations, in order of what they actually buy:

Retain the previous release image on the host. Free, and it is the only one that fixes the rollback case.
A pull-through cache with a long TTL near the consumers, which keeps serving whatever it has already seen.
A second authoritative registry with a documented failover. Expensive, and only worth it if a deploy freeze during an outage is genuinely unacceptable.
Digest pinning, which removes the tag-resolution round trip but does not let you fetch blobs you have never downloaded.

The four common options

A Distribution registry you can reason about

Storage backends

Probing a registry properly

TLS, and the flag you should not reach for

Harbor, GitLab and Quay in one paragraph each

Replication

Knowledge check