Skip to main content
RunBook Academy

Git, CI/CD & GitOpsXCVI · Incident: Malicious DependencyIncidentResponse

The supply-chain incident arrives — detection and initial triage

Advanced⏱ ~26 mingit

What you'll learn

  • Distinguish a public advisory, a CI scanner hit, and a confirmed dependency compromise
  • List the four channels that surface a malicious dependency: advisory feed, scanner alert, runtime signal, partner disclosure
  • Answer the four scope questions in the first ten minutes: which package, which version, which projects, which builds
  • Open a response channel and assign an incident commander before any remediation begins

Prerequisites

Verified against Git 2.55.x teaching target; 2.40+ minimum · GitHub Actions continuous service; Aug 2026 documentation baseline · Argo CD v3.5.x teaching target; v3.0+ minimum · Flux v2.9.x · Sigstore Cosign v3.1.x · SLSA v1.2 · OCI Distribution Specification v1.1 · Git LFS v3.7.1 · Kubernetes (cross-course target) 1.36.x

Not yet marked complete on this device.

A malicious dependency is an incident to respond to, not a problem to solve. The first signal reaches the on-call engineer through one of four channels, and the first ten minutes are spent on scope, not on remediation. The questions answered in those ten minutes determine every subsequent decision.

How the alert reaches the team

The four channels are not equivalent. Each carries different evidence, different urgency, and a different starting assumption:

flowchart LR
    A["malicious package event"] --> B["advisory feed"]
    A --> C["CI scanner hit"]
    A --> D["runtime signal"]
    A --> E["partner disclosure"]
    B --> F["on-call paged"]
    C --> F
    D --> F
    E --> F
    F --> G["scope question set"]
    G --> H["incident commander assigned"]
    H --> I["response channel opened"]
  • Advisory feed. A public advisory lists the package, the malicious version range, and the recommended action. Urgency is high; the attack is documented.
  • CI scanner hit. npm audit, pip-audit, or trivy fs --security-checks vuln . flagged the package on a pull request or scheduled scan. Urgency is moderate; the package may be in the lockfile but not yet built.
  • Runtime signal. A deployed artifact contacted a known command-and-control endpoint or spawned an unexpected child process. Urgency is highest; the malicious code is executing in production.
  • Partner disclosure. A security researcher, vendor, or another team reported the malicious package to the team’s security contact.

The four scope questions

Before any remediation, the on-call answers:

  1. Which package. The exact name, the malicious version range, the registry, and the dependency tree position (direct or transitive).
  2. Which version. The lockfile range that resolved to the malicious version, the publish timestamp, and whether the malicious version is still installable.
  3. Which projects. Every repository and workflow that declares the package as a direct or transitive dependency. Determines the impact set.
  4. Which builds. Every workflow run in the compromise window — from the first build that could have pulled the malicious version to the current build.

The lockfile is the source of truth for question 4. The lockfile records the exact version and integrity hash the team installed; comparing the lockfile’s recorded version against the advisory’s malicious range is the mechanical scope check:

npm ls "$PACKAGE_NAME"
pip-audit -r requirements.lock
trivy fs --security-checks vuln .

Opening the response

Three artefacts are produced before remediation: a dedicated response channel, an incident commander who owns the sequence, and a timestamped timeline that becomes the artefact the security team audits.

Production discipline

  1. Log everything. A timestamped entry for every action before the action is taken.
  2. One commander. A single decision-maker who sequences the response.
  3. No remediation before scope. Revert, repin, and consumer updates are deferred until the four scope questions are answered.

Cross-course references

  • Git, CI/CD & GitOps — Part LXVII-06 (Package Pinning and Lockfiles) covers the lockfile shape the scope check reads.
  • Git, CI/CD & GitOps — Part LXXI-04 (Dependency Compromise Attack) covers the four shapes of dependency-compromise attack the scope discriminates between.
  • Container Security for Production Sysadmins — Part V (Image Scanning) covers the runtime signal channel.

Quiz

Knowledge check · 4 questions

  1. Q1. A `trivy fs --security-checks vuln .` scan flags a transitive dependency in a project's lockfile. The advisory describes a malicious version range that overlaps the lockfile's pinned version. What is the highest-leverage first action?

  2. Q2. A CI scanner hit on a transitive dependency is not sufficient evidence that the malicious package has executed in production.

  3. Q3. Name the four scope questions the on-call engineer must answer in the first ten minutes of a confirmed malicious-dependency alert.

  4. Q4. A public advisory names a transitive dependency of a popular logging library as malicious. The advisory lists versions 4.4.0 through 4.4.2. Triage the alert and identify the project set.

    The team runs twelve Node services. Each service declares the logging library as a direct dependency in `package.json`. The lockfile is committed to each repository. The advisory's malicious version range overlaps the lockfile's pinned version in eight of the twelve services. The scanner has flagged all eight. The remaining four services pin the logging library to an older version range that does not overlap.

Passing score: 75%. Answers are checked in this browser.