Syntax check, yamllint and ansible-lint as CI stages

Part XXV covers what --syntax-check proves and Part XXV’s lint lesson covers the profile ladder and how to adopt it on an inherited repository. This lesson is about assembling them into a pipeline, which raises a question neither answers on its own:

What does each stage catch that the previous stage structurally cannot?

If a stage has no answer to that, it is duplicated work with an extra failure mode. All three below have an answer, and the answers do not overlap.

One deliberately bad playbook

Everything in this lesson is measured against the same five lines. It is short enough to hold in your head and wrong in four different ways:

- hosts: all
  tasks:
  - shell: systemctl restart nginx
  - name: install
    apt: name=nginx state=latest

No play name. Task list indented flush with its key. A shell invoking something a module handles. Free-form module arguments. Unqualified module names. A package pinned to latest. No changed_when on a command that always reports changed.

Stage 1: --syntax-check

$ ansible-playbook -i inventories/staging playbooks/bad.yml --syntax-check

playbook: playbooks/bad.yml

Exit status 0. Nothing wrong was found, because nothing wrong was present in the class this gate covers.

--syntax-check answers one question: can this playbook be parsed and assembled into plays and tasks? It catches unparseable YAML, an unknown playbook keyword, a role that cannot be found, a import_playbook that points at nothing. It does not evaluate style, does not resolve variables, does not know that shell was a poor choice, and — as Part XXV establishes — is blind to anything inside a dynamic include_tasks.

Keep it first anyway. It is the fastest gate by an order of magnitude, and a repository that cannot be assembled has nothing to lint.

Stage 2: yamllint

$ yamllint -f parsable playbooks/bad.yml

playbooks/bad.yml:1:1: [warning] missing document start "---" (document-start)
playbooks/bad.yml:3:3: [error] wrong indentation: expected at least 3 (indentation)

Exit status 1.

Ansible parsed this file happily; YAML permits the indentation and Ansible does not care about a missing document start. What yamllint adds is consistency of the text, which matters for a reason that is not aesthetic: a repository with uniform YAML produces diffs where the changed lines are the changed semantics. A repository with drifting indentation produces review noise, and review noise is how a scope change gets skimmed.

The exit codes are worth knowing exactly, because they are what the “fail on warnings” decision is made of:

Measured on yamllint 1.38.0: a file whose only finding is the document-start warning exits 0 by default and 2 under -s.

Stage 3: ansible-lint

This is where the five-line playbook finally gets the treatment it deserves — and where the trap lives.

$ ansible-lint --profile production --format pep8 playbooks/bad.yml

playbooks/bad.yml:1:3: name[play][/]: All plays should be named.
playbooks/bad.yml:3: command-instead-of-module: systemctl used in place of systemd module
playbooks/bad.yml:3: command-instead-of-shell: Use shell only when shell functionality is required.
playbooks/bad.yml:3:5: fqcn[action-core]: Use FQCN for builtin module actions (shell).
playbooks/bad.yml:3: name[missing][/]: All tasks should be named.
playbooks/bad.yml:3: no-changed-when: Commands should not change things if nothing needs doing.
playbooks/bad.yml:3: yaml[indentation][/]: Wrong indentation: expected at least 3
playbooks/bad.yml:4:11: name[casing][/]: All names should start with an uppercase letter.
playbooks/bad.yml:4: no-free-form: Avoid using free-form when calling module actions. (apt)
playbooks/bad.yml:4: package-latest: Package installs should not use latest.
playbooks/bad.yml:5:5: fqcn[action-core]: Use FQCN for builtin module actions (apt).

Failed: 11 failure(s), 0 warning(s) in 1 files processed of 1 encountered. Profile 'production' was required, but 'min' profile passed.

Exit status 2.

Note what ansible-lint catches that neither previous stage could: no-changed-when is a statement about idempotence, package-latest is a statement about reproducibility, and command-instead-of-module is a statement about check-mode support. Those are semantic findings about how the automation will behave, which is a different category from “does it parse” and “is it formatted consistently”.

Note also that it re-reports the indentation as yaml[indentation]. The overlap with yamllint is real, and it is the reason some teams drop the separate yamllint stage. Keep it if you want yamllint’s full rule set — line length, key ordering, trailing spaces — which ansible-lint’s embedded subset does not cover. Drop it if you do not, and say which you chose.

Knowledge check