Docker Hub — limits, namespaces, official images

Docker Hub is the default public registry. Most images come from it, most Dockerfiles start with a FROM that resolves to it, and most CI pipelines depend on it without ever naming it. That last property is the problem: a dependency nobody wrote down is a dependency nobody monitors.

This lesson is about making the dependency explicit — what a pull costs you, how it is counted, and how to stop paying for it on every build.

Namespaces

docker pull nginx              # docker.io/library/nginx:latest
docker pull myorg/myapp        # docker.io/myorg/myapp:latest
docker pull myorg/myapp:dev    # docker.io/myorg/myapp, tag dev

Official images live in library/. User and organisation images live in their own namespace. The full form is <registry>/<namespace>/<repository>:<tag>, and every part of it has a default: the registry defaults to docker.io, the namespace defaults to library, the tag defaults to latest.

Those defaults are convenient and they are also how an image reference silently changes meaning when you move it between environments.

IMG=nginx:1.27
docker pull "$IMG"
docker image inspect "$IMG" --format '{{index .RepoDigests 0}}'

nginx@sha256:0000000000000000000000000000000000000000000000000000000000000000

Write the fully qualified reference in anything that is not an interactive shell. docker.io/library/nginx:1.27 means one thing on every host; nginx:1.27 means whatever that host’s daemon configuration says it means.

Rate limits — the number that becomes an outage

Docker Hub meters pulls. As documented at the time of writing:

Two details in that table do the damage.

The unauthenticated bucket is per source address, not per machine. Every CI runner behind one NAT gateway shares a single allowance of 100 pulls per six hours. A fleet of twenty runners each pulling five base images exhausts it in one round of builds. The /64 clause means the usual IPv6 mitigation — every host gets its own address — does not help either, because a /64 is what a single site is normally allocated.

Authenticating moves you into a different bucket, not an unlimited one. A Docker Personal login raises the ceiling to 200 per six hours and attributes usage to the account rather than the address, which is strictly better but still finite and still shared by every job using that credential.

Reading your remaining quota before it bites

The limit is observable. Ask the Hub token service for a pull token, then issue a HEAD for a manifest and read the headers — a HEAD against the manifest is how the client checks an image, and the response carries the rate-limit counters.

REPO=ratelimitpreview/test
TOKEN=$(curl -fsSL \
"https://auth.docker.io/token?service=registry.docker.io&scope=repository:${REPO}:pull" \
| sed -n 's/.*"token":"\([^"]*\)".*/\1/p')

curl -fsSI -H "Authorization: Bearer ${TOKEN}" \
"https://registry-1.docker.io/v2/${REPO}/manifests/latest" \
| grep -i 'ratelimit'

ratelimit-limit: 100;w=21600
ratelimit-remaining: 20;w=21600
docker-ratelimit-source: 192.0.2.1

w=21600 is the window in seconds — six hours. docker-ratelimit-source is the address the limit is being charged to, which is the field that settles the argument about which NAT gateway your runners actually leave through.

Add the same credentials you use in CI (curl -u) and the headers change to reflect that account’s bucket instead. If the headers are absent entirely, the account has an unlimited plan or the repository is covered by a publisher agreement — either way you are not being metered on that pull.

The pull-through cache

The real fix is not a bigger quota. It is not pulling from Hub forty times for forty jobs that all want the same base image.

Distribution can run in proxy mode, where it serves images from a local store and fetches from the upstream only on a miss. One upstream pull per image per TTL, for the whole fleet.

version: 0.1
storage:
filesystem:
  rootdirectory: /var/lib/registry
http:
addr: :5000
proxy:
remoteurl: https://registry-1.docker.io
ttl: 168h

docker run -d --name hub-cache \
--restart always \
-p 5000:5000 \
-v /srv/hub-cache:/var/lib/registry \
-v /srv/hub-cache-config.yml:/etc/distribution/config.yml:ro \
registry:3

Then point the daemon at it:

{
"registry-mirrors": ["https://hub-cache.example.com"]
}

sudo systemctl reload docker
docker info --format '{{json .RegistryConfig.Mirrors}}'

["https://hub-cache.example.com/"]

Three constraints you must know before you build a design around this:

• Only Docker Hub can be mirrored. The upstream documentation is explicit: “It’s currently not possible to mirror another private registry. Only the central Hub can be mirrored.” A cache in front of ghcr.io or your own Harbor is a different mechanism.
• You cannot push to a pull-through cache. It is read-only by construction. Pushes must go to the real registry.
• registry-mirrors is a daemon setting, so it applies to every pull on the host and is invisible in the image reference. A reader of a Compose file cannot tell whether nginx:1.27 came from Hub or from your cache.

Verifying the cache is actually being used

A mirror that is configured but silently bypassed is the common outcome, and nothing on the client says so. Prove it from the cache’s side:

CACHE=https://hub-cache.example.com

docker pull nginx:1.27
curl -fsSL "$CACHE/v2/_catalog" | tr ',' '\n'

{"repositories":["library/nginx"]}

The second pull of the same image is the one that matters. Time both: a cache hit is limited by your LAN, a miss by your upstream link, and the difference is usually an order of magnitude.

Official images

Docker Official Images — the library/* namespace — are built from Dockerfiles in public repositories, reviewed by Docker, and rebuilt when their base or a bundled package gets a security update.

That is a meaningful assurance and it is narrower than it sounds:

• It says the build is reviewed. It says nothing about whether the tag you are pulling today is the artefact that was reviewed, because tags move.
• “Rebuilt for security updates” means the image is rebuilt when a fix exists. It does not mean the image has no known CVEs at any given moment; a freshly rebuilt official image routinely carries open findings for which no distro fix has shipped.

Trust Docker Official Images as a starting point. Pin by digest so the thing you trusted is the thing you run, and re-derive the digest deliberately when you want the update.

Mirroring images you depend on

For anything on the deploy path, copy it into a registry you control:

SRC=nginx:1.27
DST=registry.example.com/mirror/nginx:1.27

docker pull "$SRC"
DIGEST=$(docker image inspect "$SRC" --format '{{index .RepoDigests 0}}')
echo "mirroring $DIGEST"

docker tag "$SRC" "$DST"
docker push "$DST"

docker buildx imagetools create does the same job without pulling the layers to the local host, which matters for multi-platform images: a docker pull followed by docker push on an amd64 host silently drops every other platform from the index.

docker buildx imagetools create \
--tag registry.example.com/mirror/nginx:1.27 \
docker.io/library/nginx:1.27

Knowledge check