Skip to main content
RunBook Academy

AnsibleXLIII · Observability and Auditing of AutomationObservability and auditing

The controller log nobody turned on

Advanced⏱ ~25 minansible-coreansible-configlogrotate

What you'll learn

  • State the default of log_path and what its absence means for an audit
  • Enable controller logging from configuration and read the record format
  • Size, rotate and restrict the log file rather than merely creating it
  • Predict exactly which task output reaches the log and which does not
  • Treat the log file as sensitive material with a retention decision attached

Prerequisites

Verified against ansible-core 2.21.x · ansible (community package) 14.x · Python (controller) 3.12+ · ansible-lint 26.x · Molecule 26.x · Ubuntu 24.04 LTS · Debian 12 (Bookworm) · RHEL / Rocky / AlmaLinux 9.x · 2026-08-11

Not yet marked complete on this device.

Ask most teams where their Ansible runs are recorded and the answer is “in CI”. Ask what happens when someone runs a play from their own machine, and the answer is a pause.

There is a setting that would record it. It ships disabled, it has shipped disabled for the entire life of the tool, and it takes one line to turn on. This lesson is about that line and the three things it commits you to.

The default is nothing

Read from the controller rather than from memory:

Read-only / Safethe setting, its default, and every way to set it
$ ansible-config list | grep -A 9 'DEFAULT_LOG_PATH:'
DEFAULT_LOG_PATH:
default: null
description:
- File to which Ansible will log on the controller.
- When not set the logging is disabled.
env:
- name: ANSIBLE_LOG_PATH
ini:
- key: log_path
  section: defaults

“When not set the logging is disabled.” Not “logs to syslog”, not “logs at a reduced level”. Disabled.

So on a default controller, the complete durable record of a run is whatever the operator’s terminal scrollback happens to contain, until they close the window. If the run was a scheduled job, it is whatever the scheduler captured. If it was ansible-playbook typed by a person at 02:00 during an incident, it is nothing.

That is the gap. Every question in lesson 5 — who changed three hundred servers, when, with what — starts from a record on the controller, and by default there is not one.

Turning it on

In ansible.cfg, under source control with the rest of the project configuration:

[defaults]
log_path = /var/log/ansible/ansible.log

or for a single run, without changing the repository:

Read-only / Safelogging for one invocation
ANSIBLE_LOG_PATH=./run-$(date -u +%Y%m%dT%H%M%SZ).log ansible-playbook -i inventory/ site.yml --check

What lands in the file is the same stream the stdout callback rendered, with a header per line:

Read-only / Safethe record format
$ head -9 secret.log
2026-08-11 23:06:41,791 p=367984 u=ebrandi n=ansible INFO| PLAY [Does no_log protect the log file] ****
2026-08-11 23:06:41,795 p=367984 u=ebrandi n=ansible INFO| TASK [Task without no_log] *****************
2026-08-11 23:06:41,804 p=367984 u=ebrandi n=ansible INFO| ok: [web1] => {
  "msg": "token is REPLACE_ME_TOKEN_VALUE"
}
2026-08-11 23:06:41,805 p=367984 u=ebrandi n=ansible INFO| TASK [Task with no_log] ********************
2026-08-11 23:06:41,812 p=367984 u=ebrandi n=ansible INFO| ok: [web1]
2026-08-11 23:06:41,813 p=367984 u=ebrandi n=ansible INFO| PLAY RECAP *********************************
2026-08-11 23:06:41,813 p=367984 u=ebrandi n=ansible INFO| web1  : ok=2  changed=0  unreachable=0  failed=0  skipped=0  rescued=0  ignored=0

Four fields per line, and each is doing audit work:

  • A timestamp with milliseconds — the join key against managed-node evidence in lesson 5.
  • p= the controller process id — distinguishes concurrent runs.
  • u= the operating-system user who ran it — the only place a human name appears anywhere in this chain, and the reason lesson 5 keeps coming back to it.
  • n= the Python logger name.

Note what is not there: the playbook path, the inventory used, the --limit applied, the repository commit. The log records what the run printed, not what the run was. Reconstructing that is lesson 4’s job.

Three consequences, all of them yours now

It grows without bound

Ansible writes to the file and does nothing else with it. There is no rotation, no size cap, no compression. A verbose nightly run over a four-hundred-host fleet produces tens of megabytes a night into one file, forever, until the controller’s root filesystem fills — at which point the controller stops being able to run anything, including the play that would fix it.

Rotation is a logrotate fragment, and it is not optional:

# /etc/logrotate.d/ansible
/var/log/ansible/ansible.log {
    daily
    rotate 90
    compress
    delaycompress
    missingok
    notifempty
    create 0640 ansible ansible
}

rotate 90 is a retention decision, and it should be made by whoever owns your change-audit requirements rather than defaulted. Ninety days answers “what happened last quarter”; seven does not.

The create line matters for the next consequence.

It is created with your umask

This one surprises people, and it is verifiable in one command.

Read-only / Safethe mode Ansible actually created the log with
$ stat -c '%a %U' secret.log
664 ebrandi

Group-writable and world-readable, because that is what the umask produced. On a controller with the common default umask of 022 you get 644 — still world-readable.

On a shared controller, that means every account on the machine can read every line of every run, including the contents of the next section. The create 0640 ansible ansible in the logrotate fragment fixes it for rotated files; the live file needs the directory and the file to be created correctly in the first place:

Configuration changerestrict the log before the first run writes to it
install -d -o ansible -g ansible -m 0750 /var/log/ansible
install -o ansible -g ansible -m 0640 /dev/null /var/log/ansible/ansible.log

The controller-security part of this course treats the controller as the most privileged machine in the estate. This file is one of the reasons.

It contains whatever the run printed

Here is the demonstration that matters, and the result is uncomfortable.

A play with two tasks. Both render the same variable. One carries no_log: true:

- name: Does no_log protect the log file
  hosts: web1
  gather_facts: false
  vars:
    api_token: 'REPLACE_ME_TOKEN_VALUE'
  tasks:
    - name: Task without no_log
      ansible.builtin.debug:
        msg: "token is {{ api_token }}"

    - name: Task with no_log
      ansible.builtin.debug:
        msg: "token is {{ api_token }}"
      no_log: true

Run with ANSIBLE_LOG_PATH set, then search the log:

Read-only / Safeone of the two tasks put the token on disk
$ grep -c REPLACE_ME_TOKEN_VALUE secret.log
1

no_log worked exactly as documented on the task that carried it: the log line for that task is ok: [web1] with no result body at all. And the task that did not carry it wrote the secret to a file on disk, in a directory that was world-readable, with no indication anywhere that something sensitive had just become durable.

Filtering, and what it cannot do

There is one more setting worth knowing, mostly so that you do not over-estimate it:

Read-only / Safelog filtering operates on logger names
$ ansible-config list | grep -A 8 'DEFAULT_LOG_FILTER:'
DEFAULT_LOG_FILTER:
default: []
description: List of logger names to filter out of the log file.
env:
- name: ANSIBLE_LOG_FILTER
ini:
- key: log_filter
  section: defaults
name: Name filters for python logger

“Logger names” — the n= field. It suppresses noise from a named Python logger. It is not a content filter, and there is no supported way to say “log everything except things that look like credentials”. Redaction has to happen at the task, with no_log, before the value ever becomes an event.

Knowledge check

Knowledge check · 4 questions

  1. Q1. What durable record does a default ansible-core controller keep of a playbook run?

  2. Q2. A play has two tasks that render the same secret variable, one of them carrying no_log: true. The run is made with ANSIBLE_LOG_PATH set. What is in the log file?

  3. Q3. Which of these are genuine operational consequences of enabling log_path? Select all that apply.

  4. Q4. log_filter cannot be used to keep credentials out of the log file, because it filters by Python logger name rather than by content.

Passing score: 75%. Answers are checked in this browser.