Skip to main content
RunBook Academy

← All assessments

Final assessment · expert · ~45 min · pass ≥ 80%

Final assessment: Production Docker environment

Objectives

  • Design a complete production Docker environment from a scenario brief
  • Diagnose a live production incident using evidence-based reasoning
  • Decide between hardening remediations based on risk and cost
  • Plan a safe upgrade with rollback and observable validation
  • Justify a restore decision during a partial failure

10 questions · last verified 2026-08-09

Auto-scored questions

The questions below are scored automatically. Combine them with the scenario rubric in the body of this page to assess your readiness.

Knowledge check · 10 questions

  1. Q1. A container restarts every 90 seconds. `docker inspect` shows `OOMKilled: true`. The host has 16 GB of RAM and the container has no memory limit set. Which is the most appropriate first remediation?

  2. Q2. Which of the following statements are true about Docker restart policies? Select all that apply.

  3. Q3. A named volume is backed up by copying `/var/lib/docker/volumes/<name>/_data` while the container is running.

  4. Q4. Name the command you would run first to investigate why a container is unreachable from outside the host, after confirming the container is running.

  5. Q5. You rotate a TLS certificate on a Caddy reverse proxy in front of your Docker stack. Which sequence most safely validates the new certificate is live?

  6. Q6. Which of the following are correct mitigations for the disk-full runbook? Select all that apply.

  7. Q7. Running `docker run --rm -v /var/run/docker.sock:/var/run/docker.sock alpine sh` lets the container read the host''s /etc/shadow.

  8. Q8. What two lines from a Compose file together prove the application is running with a non-root UID and a minimal capability set?

  9. Q9. A Prometheus scrape of cAdvisor shows a container at 95% of its CPU limit for 30 minutes. The application''s p99 latency is also elevated. What is the safest immediate response?

  10. Q10. For a production Docker host, which of the following are mandatory before declaring it "production-ready"? Select all that apply.

Passing score: 75%. Answers are checked in this browser.

Scenarios (graded on evidence and rollback)

Each scenario below is what a senior engineer would hand a mid-level operator during a real incident. There is no auto-grader; the rubric shows what a strong answer must contain.

Scenario 1 — The 03:00 page

It is 03:00 on a Saturday. Your Prometheus alert fires: HighMemoryUsage(container=api, threshold=90%). The api container is in a Restarting state. Logs from the last hour show:

[02:51] INFO  api    request_count=12043
[02:54] INFO  api    request_count=13921
[02:58] WARN  api    pool_exhaustion db_connections=98/100
[02:59] ERROR api    db timeout (5s) on /v2/orders
[03:00] FATAL api    OOM: cannot allocate
[03:00] INFO  api    shutting down (SIGTERM)
[03:01] INFO  api    started (pid 17)

What evidence do you collect in the first five minutes, before changing anything?

A strong answer includes:

  • docker inspect api for OOMKilled, RestartCount, exit code.
  • docker stats --no-stream api for current memory vs. limit.
  • journalctl -u docker --since "10 minutes ago" for daemon messages.
  • dmesg | grep -i oom for kernel OOM events and which process was killed.
  • cat /sys/fs/cgroup/.../memory.events for oom_kill counter.
  • The application’s own metrics: connection-pool size, request rate, query latency.
  • The current Compose file (cat compose.yml or equivalent).

What is your mitigation, and how do you prove it works?

A strong answer:

  • Stops the restart loop first (set --restart no or remove the policy) to stabilise the system while you investigate.
  • Decides between three remediations, in order:
    1. Bump the memory limit if the working set is genuinely larger than the limit; verify with docker stats after the change.
    2. Reduce the working set (cache size, query batch size) if the growth is unbounded.
    3. Add a replica if the workload is per-process stateful but the single instance is too small for the traffic.
  • Validates with an end-to-end request (curl /v2/orders) and watches docker stats for ten minutes.

What is your rollback if your first remediation makes it worse?

A strong answer names a specific image tag or compose override to return to, and a way to verify the rollback without restarting the investigation.

Scenario 2 — The certificate that expired

The on-call channel lights up: “Site is down, browser says cert expired.” You check the reverse proxy: cert expired 14 minutes ago. The renewal cron ran last night at 03:00 and the log says “renewal succeeded”, but the new cert is not on disk.

Where do you look first?

A strong answer:

  • docker exec caddy caddy list-certificates (Caddy) or openssl s_client -connect api.example.com:443 -servername api.example.com < /dev/null 2>/dev/null | openssl x509 -noout -dates for the live cert dates.
  • docker exec caddy ls -la /data/caddy/certificates/acme-v02.api.letsencrypt.org-directory/ to see what Caddy has on disk.
  • The renewal cron log (/var/log/caddy-autorenew.log or journald) for the actual command run, not just the conclusion.
  • Whether the renewal webserver was actually reachable from Let’s Encrypt at the time of the challenge.

What is your mitigation, and how do you prove the new cert is live?

A strong answer:

  • Triggers a renewal manually (docker exec caddy caddy reload or certbot renew --force-renewal).
  • Verifies the live cert via the admin API or by curling with -v and inspecting the cert chain.
  • Resolves the root cause of the silent failure (port 80 closed during challenge, rate limit, DNS race) before declaring done.

Scenario 3 — The CVE drop

A CVE is published against libssl3 in your base image (myorg/base:1.4.2). Your image scanner (Trivy) flags all images built on that base. Your stack runs seven services; six use the vulnerable base.

Walk through your response end-to-end.

A strong answer:

  1. Identify affected images. Run trivy image --severity CRITICAL myorg/base:1.4.2 and confirm the CVE; run the same on every built image that inherits from it.
  2. Assess severity. Check the CVE’s CVSS, the affected function, and whether your service exposes that code path. A high CVSS in an unused code path may be deferred.
  3. Identify a patched base. Track upstream — does myorg/base:1.4.3 exist? If not, can you rebase to 1.5.0 without breakage?
  4. Rebuild. Trigger a rebuild with the new base, signed by your CI.
  5. Test. Deploy to staging; run smoke tests including the exact attack path described in the CVE.
  6. Deploy. Use your existing rolling-update strategy; pin by digest; watch error rates.
  7. Roll back if needed. If the rebuild breaks a service, redeploy the previous image tag and document the incident.

A strong answer explicitly distinguishes “the CVE exists” from “the CVE affects us”.

Scenario 4 — The “should we orchestrate?” question

Your team is growing. Compose currently runs the whole stack on one host. The next feature requires three replicas and zero-downtime deploys. Engineering wants to “move to Kubernetes”.

What is your reasoning?

A strong answer:

  • Articulates the actual gap: capacity, availability, or developer experience. The remediation depends on which.
  • For capacity: add a second host, share state via external Postgres/Redis, front with HAProxy. No orchestration needed yet.
  • For availability: at the multi-host boundary you have a real choice between Swarm (familiar), Nomad (simple), Kubernetes (industry standard). Names the trade-offs, not the buzzword.
  • Recognises that orchestration is a skill and operational cost, not just a deploy mechanism.

A weak answer just says “Kubernetes”.

Scenario 5 — The disaster recovery drill

It is Q3. Your compliance team asks for evidence that the disaster-recovery plan works. You have 90 days to demonstrate a recovered host.

What does your drill look like?

A strong answer:

  • Pick a backup from N days ago (typically the worst acceptable RPO).
  • Provision a clean host (same OS, kernel, Docker version).
  • Reinstall Docker; restore /etc/docker/daemon.json from version control.
  • Restore volumes from the chosen backup.
  • Pull images from your registry.
  • Deploy the Compose stack with the restored secrets.
  • Run end-to-end smoke tests against the recovered application.
  • Time the entire process; record the result; fix the gaps.

A strong answer also covers:

  • The documentation that was wrong (and how it was corrected).
  • The things that surprised you (and how the next drill will be faster).

After the assessment

  • If you passed: the capstone in Part XXXVIII is your portfolio.
  • If you missed a scenario: re-read the relevant Part, re-do the associated lab, then return to the scenario.
  • If you missed an auto-scored question: re-read the linked lesson.