Skip to main content
RunBook Academy

← All break/fix scenarios in Git, CI/CD & GitOps

intermediateci-third-party~30 min

Third-party action update broke pipeline

Reported symptoms

  • Every CI run since a specific timestamp fails with the same error from a third-party action
  • The workflow file has not changed; the failing step uses `uses: <org>/<action>@v3`
  • The action's `action.yml` on the new commit declares a required input that the workflow does not provide
  • GitHub shows the action tag `v3` resolving to a new commit SHA — the same tag, different code
  • The action's repository shows a recent commit by the maintainer (or by a new collaborator, possibly after a repo transfer)
  • Other workflows using the same action fail at the same step with the same error
  • A diff of the new `action.yml` against the previous one shows new required inputs or new permission scopes

Evidence

  • · `curl -sL https://api.github.com/repos/<org>/<action>/git/refs/tags/v3 | jq .ref,.object.type` resolves to a new commit SHA
  • · The previous SHA (recorded in a workflow-pinning comment) is reachable via `git ls-remote https://github.com/<org>/<action>.git <old-sha>` but not via `<org>/<action>@v3`
  • · `gh api repos/<org>/<action>/commits?per_page=5` shows a new commit at the tag tip with a non-maintainer author or an unusual timestamp
  • · The new commit's diff (`gh api repos/<org>/<action>/compare/<old-sha>...<new-sha>`) shows new required inputs, new `permissions:`, or new network calls
  • · The workflow step log shows an input-validation error pointing at a missing `with:` field
  • · The action's README has not been updated; the new behaviour is not documented
  • · A `git diff` on the workflow file in this repository shows no change in the past 30 days
  • · `step-security/harden-runner` or similar security tooling has flagged the new commit as a tag-swap or unusual maintainer change
Diagnosis and resolutionclick to reveal

Root cause

Git tags in GitHub Actions are pointers, and a maintainer can move a tag at any time. `uses: <org>/<action>@v3` tells GitHub "give me the commit that `v3` points at right now" — which is whatever the maintainer decided to point at, including malicious code, an unfinished refactor, or a tag swap following a maintainer compromise. The pin-by-tag pattern is the supply-chain equivalent of resolving DNS at every request: it is convenient, and it is the mechanism by which the wrong code gets into the pipeline. The structural failure is the absence of pin-by-commit: every third-party action reference should resolve to a 40-character SHA, with a comment recording the version it corresponds to, so that upgrading is a deliberate, reviewed operation rather than an unannounced tag movement.

Remediation

Roll back to a known-good SHA immediately: change `uses: &lt;org&gt;/&lt;action&gt;@v3` to `uses: &lt;org&gt;/&lt;action&gt;@<40-char-sha>` with a comment `# v3.2.1` recording the version that SHA corresponds to. The recovery is fast because the previous SHA is reachable by direct reference even if the tag has moved. Then audit the new commit before ever adopting it: diff the new commit against the previous SHA, look for new network calls, new file writes, new permission scopes, and any change to `action.yml` that adds required inputs or removes optional ones. If the action is in a security-sensitive path, disable it entirely and switch to a fork under the team''s own organisation, or to a vendor-supported alternative. Notify the action maintainer if the change appears unintentional.

Verification

All CI jobs that use the action are green after the SHA pin. The workflow file lists the SHA with a version comment. A diff of the new commit against the pinned SHA is captured in the runbook, including any new permissions or network calls. Any future SHA bump is a reviewed PR, not an automatic update.

Prevention

Pin every third-party action to a 40-character commit SHA, not a tag. Use `actions/pin-github-action` or `dependabot` with the `package-ecosystem: github-actions` configuration to keep the SHAs current while still requiring a review to land the bump. For high-trust actions (those that touch credentials, run on production deploys, or execute arbitrary code with repo access), vendor them: fork the action into the team''s own organisation and reference the fork. Enable the GitHub organisation-level "Allowed Actions" list so only pre-approved actions can be referenced from any workflow in the org. The principle is the same as for dependencies: a tag is a moving pointer, a SHA is an address, and addresses are what supply-chain integrity is built on.

A Git tag is a moving pointer. Pin by SHA — every time, for every third-party action — and let SHA bumps arrive as reviewed PRs, not as silent movements of a tag.