A worked replacement: one shell-heavy play, rewritten

Arguments about shell versus modules tend to be abstract. This lesson is not. Here is a play that a competent engineer in a hurry would write, and the same play written the way this course argues for, with the specific thing gained at each step.

The play as written

- name: Deploy the metrics collector
hosts: webservers
become: true
vars:
  collector_version: '2.4.1'
  collector_port: 9110
tasks:
  - name: Install the package
    ansible.builtin.shell: apt-get install -y metrics-collector={{ collector_version }}

  - name: Write the config
    ansible.builtin.shell: |
      cat > /etc/metrics-collector/collector.conf <<EOF
      listen_port = {{ collector_port }}
      hostname = {{ inventory_hostname }}
      EOF

  - name: Set permissions
    ansible.builtin.shell: chmod 640 /etc/metrics-collector/collector.conf && chown root:metrics /etc/metrics-collector/collector.conf

  - name: Enable and restart
    ansible.builtin.shell: systemctl enable metrics-collector && systemctl restart metrics-collector

  - name: Verify
    ansible.builtin.shell: curl -sf http://localhost:{{ collector_port }}/healthz

Read the failure modes before reading the rewrite, because they are the reason for it:

• It restarts the collector on every run, on every host, whether or not anything changed. On a 200-host fleet that is 200 unnecessary service interruptions per run.
• --check proves nothing. Every task is skipped, so the dry run reports a clean play regardless of what would happen.
• --diff shows nothing, so nobody reviewing the change can see what the config will become.
• The recap always says changed=5, so there is no way to tell a converged fleet from a drifting one.
• {{ collector_version }} reaches apt-get through a shell, so a version string from a CI variable is an injection point.
• curl -sf failing does not distinguish “the service is broken” from “curl is not installed”.

Task 1: installing a package

- name: Install the metrics collector
ansible.builtin.package:
  name: "metrics-collector={{ collector_version }}"
  state: present

What you get back: the module queries the package database first, so a host that already has 2.4.1 reports ok and the recap starts meaning something. It works under --check, predicting the change without making it. It works on dnf hosts and apt hosts from one task, so the play stops encoding a distribution assumption. And the version string is passed as data rather than assembled into a shell line.

Task 2: writing a config file

- name: Write the collector config
ansible.builtin.template:
  src: collector.conf.j2
  dest: /etc/metrics-collector/collector.conf
  owner: root
  group: metrics
  mode: '0640'
  validate: /usr/bin/metrics-collector --check-config %s
notify: Restart metrics collector

What you get back, and this is the substitution that pays for the whole exercise:

• --diff works. A reviewer sees the exact lines that will change on that host, before the run.
• changed is true only when the rendered content differs from what is on disk. That is what makes the handler correct.
• Permissions are part of the same declaration, so task 3 disappears entirely — and the mode is quoted, for reasons the YAML part of this course devotes a lesson to.
• validate runs the new content through the daemon’s own config checker before installing it. A syntactically broken config is rejected while the old one is still in place, rather than being written and then failing at restart. On a fleet, that is the difference between one failed task and a service outage.
• A backup is one keyword away (backup: true), which is a genuine rollback path rather than a hope.

Task 3: permissions

Gone. It was an artefact of cat > file not being able to express ownership, and it was reporting changed every run on top of that.

Task 4: enable and restart

# In tasks:
- name: Enable the collector at boot
ansible.builtin.systemd_service:
  name: metrics-collector
  enabled: true
  state: started

# In handlers:
- name: Restart metrics collector
ansible.builtin.systemd_service:
  name: metrics-collector
  state: restarted

What you get back: the && chain has become two different kinds of thing, correctly separated.

enabled: true and state: started are desired state — they report ok on a host where the unit is already enabled and running, and changed on one where it was not. state: restarted is an action, and it lives in a handler so that it happens only when the config task above actually changed something.

That single move eliminates the fleet-wide unnecessary restart. It also makes the restart appear in the output only when it happened, which is what an operator needs to see during a rollout.

Task 5: verification

- name: Wait for the collector to accept connections
ansible.builtin.wait_for:
  port: "{{ collector_port }}"
  host: 127.0.0.1
  timeout: 30

- name: Confirm the health endpoint reports healthy
ansible.builtin.uri:
  url: "http://127.0.0.1:{{ collector_port }}/healthz"
  return_content: true
  status_code: 200
register: health
changed_when: false
failed_when: health.json.status != 'ok'

What you get back: curl -sf collapsed several distinct outcomes — service not listening yet, service returning a 500, curl not installed, DNS or proxy interference — into one non-zero exit code. This version separates them.

wait_for handles the race the original had — a restarted service is not listening the instant systemctl returns, and a curl immediately afterwards fails intermittently on loaded hosts. uri asserts the status code explicitly, returns the parsed body, and does not depend on curl being installed on the managed node. failed_when inspects the response rather than trusting a flag. And changed_when: false is truthful here, because a health check reads and does not write.

Where shell survives the rewrite

Not every task converts, and pretending otherwise is how people end up with a command task wrapped in six lines of set_fact gymnastics.

- name: Read the collector's registered fleet identity
ansible.builtin.command: /usr/sbin/collectorctl identity --quiet
register: identity
changed_when: false

- name: Register this host with the metrics backend
ansible.builtin.command:
  argv:
    - /usr/sbin/collectorctl
    - register
    - --site
    - "{{ site_name }}"
when: identity.stdout | trim == ''

Two things make this acceptable where the original play’s tasks were not: the probe declares itself read-only truthfully, and the work task uses argv so site_name is data rather than shell text.

The evidence

The argument for the rewrite is not aesthetic, and it is settled by running the play twice.

$ ansible-playbook -i inventories/prod collector.yml --limit web01.example.com

# before
PLAY RECAP *********************************************************************
web01.example.com          : ok=5    changed=5    unreachable=0    failed=0    skipped=0

# after
PLAY RECAP *********************************************************************
web01.example.com          : ok=7    changed=0    unreachable=0    failed=0    skipped=0

changed=0 on a converged host is the entire point. It means the run did nothing, which means the host matches its definition, which means you can run this play at 03:00 during an incident and know that it will only touch what is wrong.

Blast radius: the summary argument

The rewritten play is not merely tidier. It is safer in ways that map directly onto the questions this course keeps asking:

Knowledge check