How Ansible chooses a Python · Ansible

Part VI introduced INTERPRETER_PYTHON as one configuration setting among many: what the values mean, what the warning says, and why silencing it globally is a bad trade. That lesson owns the setting.

This lesson owns the mechanism. Where discovery sits in the execution of a play, what it actually sends to the host, how the result is stored, and the three behaviours that are not documented anywhere but in the code. You need those to diagnose a mixed fleet, because the symptoms only make sense once you know the sequence.

Everything below was read from, or executed against, ansible-core 2.21.3.

Where discovery happens

Discovery is not a step in the play. It has no task, it does not appear in --list-tasks, and it produces no ok: line. It is triggered lazily, by the first module that needs an interpreter and does not have one.

The sequence for a single host, first module of the run:

The controller starts building the module payload and needs a shebang, so it asks: what is ansible_python_interpreter for this host?
The configuration system answers with the effective value — inventory variable if set, otherwise INTERPRETER_PYTHON, otherwise the default auto.
If that value is auto or auto_silent, the controller looks for a cached fact named discovered_interpreter_python for this host.
If the fact is absent, module assembly is abandoned by raising an internal InterpreterDiscoveryRequiredError, discovery runs, and the result is stored as that fact.
Assembly restarts, now with a concrete path, and the module ships.

Step 4 is the interesting one, and it explains the timing you observe: discovery costs one extra round trip, once per host per run, before the first module executes — and not before the second, because by then the fact exists.

The probe

Discovery cannot use a module, because using a module is the thing it is trying to make possible. So it sends a shell command over the connection directly.

The command is built from INTERPRETER_PYTHON_FALLBACK — one command -v per candidate, joined with semicolons and bracketed by two markers so the controller can find the output in whatever else the login shell prints:

echo FOUND; command -v 'python3.14'; command -v 'python3.13'; command -v 'python3.12'; command -v 'python3.11'; command -v 'python3.10'; command -v 'python3.9'; command -v '/usr/bin/python3'; command -v 'python3'; echo ENDFOUND

The controller matches everything between FOUND and ENDFOUND, keeps only lines beginning with /, and takes the first one. The default candidate list on 2.21.3 is exactly:

Read-only / Safethe candidate list, verified on 2.21.3— Dumps the effective configuration and selects the fallback list. No host is contacted; this reads the controller configuration only.

$ ansible-config dump | grep INTERPRETER

INTERPRETER_PYTHON(default) = auto
INTERPRETER_PYTHON_FALLBACK(default) = ['python3.14', 'python3.13', 'python3.12', 'python3.11', 'python3.10', 'python3.9', '/usr/bin/python3', 'python3']

Newest first. That ordering is the entire reason the warning exists: a host acquires python3.13 as a transitive dependency of something unrelated, and the next run silently moves module execution onto it.

If nothing matches, discovery warns that no interpreters were found and returns /usr/bin/python3 as a last resort — which on a host with no Python is not there, so the first module then fails.

Under the hood

Three behaviours follow from the implementation and are worth knowing because they each produce a confusing symptom.

Discovery resolves through PATH, not the filesystem. command -v is a shell builtin that walks the PATH of the account you connect as, under that account’s login shell. A Python installed at /opt/python3.12/bin/python3.12 and not on that PATH is invisible to discovery even though it exists. The symptom is “we installed Python and Ansible still cannot find it”.

Discovery does not run under become. The probe is executed with privilege escalation explicitly disabled. It therefore sees the connecting account’s PATH and permissions, not root’s. On a host where the interpreter is readable only by root, or is on root’s PATH alone, discovery fails while a manual sudo which python3 succeeds — which is exactly the check an engineer reaches for, and it gives the wrong answer.

A local connection skips discovery entirely. When the connection plugin reports that the remote is local, the controller uses ansible_playbook_python — the interpreter running ansible-playbook itself. Verified on 2.21.3 against implicit localhost: both ansible_playbook_python and the effective ansible_python_interpreter report the same venv Python. This is why a role that works perfectly against localhost can fail on the first real host: locally it ran under your controller venv, which has every library you installed.

What actually triggers discovery

Part VI made the point that interpreter_python accepts any string, because it doubles as a path and therefore cannot carry a list of valid choices. Here is the precise consequence, read from the module assembly code in 2.21.3.

The controller triggers discovery when the effective value is empty, or is one of exactly two strings: auto and auto_silent. Any other value is used verbatim as the interpreter path, and becomes the shebang of the shipped payload.

That is the definitive answer to a question the documentation currently answers two ways. The configuration metadata shipped with 2.21.3 states that the auto_legacy* modes are removed; the published documentation page for latest still lists auto_legacy as a deprecated alias for auto. Part VI flags the contradiction. The code settles it:

Read-only / Safewhat the value is checked against— Reads the installed ansible-core source on the controller. No host is contacted. This is the branch that decides whether discovery runs at all.

$ SP=$(python -c 'import ansible,os;print(os.path.dirname(ansible.__file__))')
grep -n auto_silent $SP/executor/module_common.py

456:            if not interpreter_out or interpreter_out in ['auto', 'auto_silent']:

So auto_legacy on 2.21.3 does not fall back to auto, and does not warn that it is deprecated. It is treated as a relative interpreter path named auto_legacy, the payload gets the shebang #!auto_legacy, and the module fails on the target with a message about the file not being found — a very long way from the setting that caused it.

Reading the result

The discovered value is stored as a fact and is available to you:

Read-only / Safewhat discovery chose, per host— Runs the debug module against every host in the pattern. It connects and executes a module on each host, so it is read-only but not free; on a large estate run it with a sensible fork count.

$ ansible all -m debug -a 'var=ansible_facts.discovered_interpreter_python' --one-line

web-a1.example.com | SUCCESS => {"ansible_facts.discovered_interpreter_python": "/usr/bin/python3.12"}
web-a2.example.com | SUCCESS => {"ansible_facts.discovered_interpreter_python": "/usr/bin/python3.12"}
web-a3.example.com | SUCCESS => {"ansible_facts.discovered_interpreter_python": "/usr/bin/python3.13"}
db01.example.com | SUCCESS => {"ansible_facts.discovered_interpreter_python": "/usr/bin/python3.9"}

Illustrative output

Three web servers that are supposed to be identical, and one of them is not. web-a3 has acquired a newer Python and discovery has moved onto it. Nothing has failed yet — it will fail the next time a play touches apt on that host, and it will look like a packaging problem.

This is the single most useful command in the part. It converts a class of future incident into a line of output you can diff.

Diagnosing with the sequence in mind

The sequence explains the symptoms. Work the cases in this order.

Every host fails identically, before any task, with an execution error. Suspect the value, not the hosts. ansible-config dump --only-changed and look at INTERPRETER_PYTHON. A typo or an auto_legacy inherited from an older estate produces exactly this.

One host in a group differs from its siblings. Compare discovered_interpreter_python across the group. A newer interpreter that arrived as a transitive dependency is the usual cause.

The interpreter exists but discovery says it does not. Check it as the connecting account without sudo, and check PATH. Discovery runs unprivileged and resolves through PATH; a check run as root is not the same check.

It works against localhost and fails everywhere else. The local connection skipped discovery and used the controller venv. Test against a real host before believing a role is portable.