Event-driven Ansible: sources, rules, actions · Ansible

Everything this course has taught so far is push automation. A person or a schedule decides that something should happen, and Ansible makes it happen.

Event-driven Ansible inverts the trigger. A long-running process watches a stream of events, and when an event matches a condition, an action fires.

event source  -->  rule (condition)  -->  action

That is the whole model. The rest of this lesson is what goes in each box, and the lesson after it is about why the arrow from action back to source is the dangerous part.

The shape of a rulebook

A rulebook is a YAML file containing one or more rulesets. A ruleset has a name, the hosts that actions requiring an inventory will use, a list of sources, and a list of rules:

---
- name: example_ruleset
  hosts: all
  sources:
    - name: range_source
      eda.builtin.range:
        limit: 5
  rules:
    - name: example_rule
      condition: event.i == 1
      action:
        debug:

Four parts, and the correspondence to a playbook is close enough to be useful and different enough to be worth stating precisely.

Playbook	Rulebook
Runs once and exits	Runs until stopped
`hosts:` is the blast radius	`hosts:` is only used by actions that need an inventory
Tasks execute in order	Rules fire when their condition matches, in no fixed order
Triggered by a person or a schedule	Triggered by an event
Fails, and you see the failure	Keeps running; a failure is an event among events

The last row is the operationally significant one. A playbook that fails stops and tells you. A rulebook that fails to do something useful goes on running, waiting for the next event, and there is nothing to notice unless you built something to notice it.

Sources

A source is a plugin that produces events. What matters for design is what kind of thing your source is:

Source class	Example	Event arrives when
Alerting webhook	A monitoring system posting an alert	A threshold is crossed — including repeatedly
Message queue	Kafka, AMQP	A producer publishes
Cloud or platform notification	An instance state change	Infrastructure changes
Source-control or CI webhook	A merge, a tag	Somebody pushes
Polling plugin	Watching a file or an API	The plugin looks and finds a difference

The distinction that determines whether your rules will behave is not the transport. It is whether the source can emit the same event many times in quick succession.

An alerting webhook can and will. A merge notification generally does not. That single property decides whether lesson 7 is optional reading or the reason your estate has an incident.

Conditions

A condition is a boolean expression over the event payload:

rules:
  - name: Respond to a disk alert
    condition: event.payload.alertname == "DiskWillFillIn4Hours"
    action:
      run_job_template:
        name: Reclaim log space
        organization: Default

Conditions can combine terms, match across multiple events, and reference facts the ruleset has accumulated. The craft here is the same as writing a when: expression, with one addition that has no playbook equivalent: precision in a condition is a safety control, because a condition that matches too broadly is an action that fires too often.

A condition that matches the alert name is usually too broad

alertname == "HighMemory" matches every firing of that alert, from every host, in every environment, forever — including the resolved notification if your alerting system sends one, and including the fifty firings that arrive in ninety seconds when a shared dependency wobbles.

At minimum a production condition should pin the environment, the severity, and whether the alert is firing rather than resolving:

condition: >-
  event.payload.status == "firing" and
  event.payload.labels.severity == "critical" and
  event.payload.labels.environment == "production" and
  event.payload.labels.alertname == "DiskWillFillIn4Hours"

Four terms rather than one. Each removes a class of unintended firing, and the environment term is the one that keeps a staging alert from remediating a production host.

Actions

There are eleven. Knowing the whole list matters because five of them change nothing, and starting with those is the recommended deployment path.

Action	What it does
`run_playbook`	Run an Ansible playbook
`run_module`	Run an Ansible module
`run_job_template`	Run a job template
`run_workflow_template`	Run a workflow template
`set_fact`	Post a fact to the running rule set in the rules engine
`post_event`	Post an event to a running rule set in the rules engine
`retract_fact`	Remove a fact from the running rule set
`print_event`	Write the event to stdout
`debug`	Debug `ansible-rulebook`
`shutdown`	Generate a shutdown event which terminates the `ansible-rulebook` process
`none`	No action, useful when writing tests

Four of them touch the fleet: run_playbook, run_module, run_job_template, run_workflow_template. The first two execute directly from the rulebook process; the last two hand the work to a platform, which means the run appears in the platform’s history with its credentials and permissions — a significant difference when you are deciding how much to trust an automated response.

The other seven stay inside the rules engine or write to stdout. They are how you build and test a rulebook without touching anything:

print_event is your alert-only deployment mode. The rulebook watches production events and prints what it would have responded to.
none is for tests where you want the rule to match and nothing to happen at all.
set_fact / retract_fact / post_event are the state machine. Facts persist in the ruleset, so a rule can record that it has acted and a later rule can condition on that.
shutdown stops the process — a kill switch expressible in the rulebook itself, which lesson 7 returns to.

Where event-driven automation genuinely fits

Being specific is worth more than enthusiasm here, because the honest list is shorter than the marketing list.

It fits when the trigger is genuinely unpredictable and the response is genuinely mechanical.

Fits well	Why
Enrich an alert with diagnostics before a human reads it	Read-only, timing matters, the work is identical every time
Open or update a ticket from an event	No fleet change, removes toil
Reconfigure automation when inventory changes	The trigger is a real event; the response is deterministic
Contain a host on a confirmed security signal	Minutes matter, and the action is narrow — isolate one host
Roll a config change when a source of truth changes	Event-shaped by construction

Fits badly	Why
Anything that happens on a timetable	It is a schedule. Use a schedule
Restarting a service that keeps failing	The restart treats the symptom, and see lesson 7
Responses that need judgement about impact	The rule has none
Anything triggered by a signal you do not fully trust	A bad signal becomes a fleet change

Facts, events, and why a rulebook is a state machine

The rules engine distinguishes two things, and the distinction explains most of the surprising behaviour people meet.

An event arrives, is matched against rules, and is gone. A fact is posted into the ruleset with set_fact and stays until retracted.

That gives a rulebook memory, and memory is what makes multi-step reactions possible: a rule that fires on an alert can set_fact to record that it acted, and a second rule can be conditioned on that fact — for example to escalate only if the same alert arrives again while the fact is still present.

Three consequences worth knowing before you rely on it.

Facts are per-ruleset, and they live in memory. Restart ansible-rulebook and the accumulated state is gone. A rulebook that depends on facts to avoid repeating itself will happily repeat itself after a restart — and a restart is exactly what happens when you deploy a change to the rulebook, which is exactly when you are watching least carefully.

Rules do not have an execution order in the way tasks do. Any rule whose condition is satisfied may fire. Two rules matching the same event both fire. This is not a defect to route around, but it does mean “the second rule will catch it” is not reasoning you can rely on.

A fact you forget to retract is permanent. A rule conditioned on the absence of a fact stops firing forever once that fact is set and never retracted. The symptom is automation that worked for a week and then silently did nothing, with no error anywhere, which is among the least pleasant things to debug in this part.

What happens if the rulebook process dies?

Nothing happens. Which is the problem.

A playbook that dies mid-run leaves a partial change, a non-zero exit code, and usually somebody watching. A rulebook that dies leaves a fleet where nothing is listening, and every event that arrives from that moment is discarded silently.

There is no partial state to notice and no exit code anybody is reading. The estate continues to believe it has automated remediation. It has a stopped process.

The requirements this creates are ordinary systems administration, which is the point:

Run it under something that restarts it. A systemd unit with Restart=on-failure, or a container under an orchestrator. Part VII of the Linux course covers the unit; nothing here is special.

Alert on the process being absent, not on it being unhealthy. The failure mode is absence.

Alert on silence. If the source normally delivers events hourly and has delivered none in six, either the world got quiet or the pipeline broke, and the second is far more likely. This is the check that catches a broken webhook, a rotated token, or a firewall change — none of which stop the process.

Remember what the restart does to your facts. Per the section above, accumulated state is lost, so a restarted rulebook may re-fire responses it had already suppressed. Which is a good reason for the rate limiting in lesson 7 to live somewhere other than in memory alone.

Knowledge check

Why this matters

Event-driven automation is the most oversold idea in this part and it has a real core, so it is worth being able to separate the two.

The real core is that some responses are worth making in seconds, and a human cannot. Containing a compromised host. Enriching an alert with diagnostics before the on-call engineer opens it. Reconfiguring automation when the inventory it depends on changes.

The oversold part is the promise of a self-healing estate. What a rulebook actually gives you is a mechanism for executing a decision you have already made, at a moment you did not choose, with no judgement applied. That is exactly as good as the decision and exactly as safe as the condition that triggers it.

Which is why the honest way to read this lesson is as a list of preconditions rather than a capability. Before an event-driven response touches a host, you need a condition precise enough that it will not fire on something else, a source you trust, a response narrow enough that firing wrongly is survivable, and a way to find out that the whole thing has quietly stopped.

Everything in the next lesson is about the fourth precondition on that list, and the one this lesson has not mentioned yet: what happens when the response makes the event more likely.