Producing evidence an auditor can read

An audit run produces a recap and some diffs. Six months later somebody asks: on 11 August, was openssh-server at the patched version on every host in the payment segment?

The recap cannot answer that. It was in a terminal, and the terminal is gone. If it was captured to a log, the log is prose — greppable at best, and it records what would change rather than what was.

Evidence is a different artefact from a report. This lesson is about producing it as a by-product of the run.

Gather state as facts, not as parsed output

The first instinct for a compliance check is command plus a parser:

# Do not do this.
- name: Check the openssh version
  ansible.builtin.shell: dpkg-query -W -f='${Version}' openssh-server
  register: ssh_version
  changed_when: false

Four problems, and they compound at fleet scale. It is skipped in check mode unless marked (the previous lesson). It is distribution-specific, so the same role needs a second branch for rpm. Parsing is brittle. And it produces a string that some later task has to interpret.

package_facts does the same job as a fact.

$ ansible-playbook facts.yml --check

TASK [ansible.builtin.package_facts] *******************************************
ok: [localhost]

TASK [ansible.builtin.debug] ***************************************************
ok: [localhost] => {
  "msg": "count=825"
}

TASK [ansible.builtin.debug] ***************************************************
ok: [localhost] => {
  "msg": [
      {
          "arch": "amd64",
          "category": "net",
          "name": "openssh-client",
          "origin": "Ubuntu",
          "source": "apt",
          "version": "1:10.2p1-2ubuntu3.5"
      }
  ]
}

Three things about that structure matter for writing checks against it.

The value is a list, not a dict. ansible_facts.packages['openssh-client'] is a list of installed versions, because a system can have more than one version of a package installed. The documentation describes it as mapping “the package name to a non-empty list”. A check written as ansible_facts.packages['x'].version is wrong; it needs an index or a map(attribute='version').

Absence is a missing key, not a null. Testing membership comes before testing version:

- name: The package must be installed and at or above the patched version
  ansible.builtin.assert:
    that:
      - "'openssh-server' in ansible_facts.packages"
      - ansible_facts.packages['openssh-server']
        | map(attribute='version') | max is version(patched_version, '>=')
    fail_msg: "openssh-server missing or below {{ patched_version }}"
    quiet: true

source tells you which manager reported it, which is what makes the same check work across a mixed estate without a when on ansible_facts['os_family'].

service_facts is the equivalent for services, populating ansible_facts.services with per-service state and status.

Keep collection cheap

Fact gathering is per host, and a compliance run is fleet-wide. The default setup collects everything.

$ ansible-playbook subsets.yml --check

TASK [ansible.builtin.debug] ***************************************************
ok: [localhost] => {
  "msg": "all subset key count = 121"
}

TASK [ansible.builtin.debug] ***************************************************
ok: [localhost] => {
  "msg": "min subset key count = 55"
}

The expensive subsets are hardware (which reads devices and mounts), mounts, and on some platforms network with every interface. A compliance role that needs the distribution, the kernel and the hostname should say so:

- name: Collect only the facts the policy needs
ansible.builtin.setup:
  gather_subset:
    - '!all'
    - 'min'
    - 'distribution'
    - 'kernel'
check_mode: false

At 30 hosts the saving is not worth the complexity. At 3,000, on an hourly schedule, it is the difference between a compliance job and a noticeable load on the estate.

The evidence file

One file, per host, per run. Machine-readable. Written by the run that produced the findings, so it cannot disagree with them.

- name: Assemble the evidence record for this host
ansible.builtin.set_fact:
  compliance_evidence:
    schema: 1
    host:
      inventory_hostname: "{{ inventory_hostname }}"
      fqdn: "{{ ansible_facts['fqdn'] }}"
      machine_id: "{{ ansible_facts['machine_id'] | default('unknown') }}"
    run:
      policy_version: "{{ compliance_policy_version }}"
      run_id: "{{ compliance_run_id }}"
      collected_at: "{{ ansible_date_time.iso8601 }}"
      mode: "{{ 'audit' if not (compliance_enforce | bool) else 'enforce' }}"
    findings: "{{ compliance_findings }}"

- name: Write the evidence file to the controller
ansible.builtin.copy:
  content: "{{ compliance_evidence | to_nice_json }}
"
  dest: >-
    evidence/{{ compliance_run_id }}/{{ inventory_hostname }}.json
  mode: '0640'
delegate_to: localhost
become: false

Every field earns its place.

Why the file goes to the controller

delegate_to: localhost writes the evidence to the controller rather than the host it describes.

The reason is the same one from the backup lesson, in a different context. A host that is non-compliant because it was compromised is a host whose local evidence file can be edited. A host that is non-compliant because it is broken is a host you may not be able to read the file from. Evidence about a thing should not be stored only on that thing.

The controller is not the final destination either — Part L covers what happens when the controller is lost — so the pipeline is: written to the controller by the run, then shipped to wherever assessments are retained.

Knowledge check