Validating configuration before it can break a service

The unprotected sequence is: render a template, write it over the live configuration file, notify a handler, restart the service. If the rendered file is invalid, the service does not come back, and it does not come back on every host in the batch simultaneously.

validate breaks that chain at the first link. The service’s own configuration checker runs against the temporary file, and the file is moved into place only if the checker exits zero.

An invalid configuration therefore never reaches /etc/. The task fails, nothing is notified, no handler fires, and the running service is untouched — still serving, on its previous configuration, while the run reports a red task on the canary.

The mechanics, in one task

- name: Deploy the nginx site configuration
ansible.builtin.template:
  src: site.conf.j2
  dest: /etc/nginx/conf.d/site.conf
  owner: root
  group: root
  mode: '0644'
  backup: true
  validate: 'nginx -t -c %s'
notify: Reload nginx

%s is substituted with the path of the temporary file. It is the only substitution the parameter performs, and the two ways to get it wrong are both common:

• Omitting it. validate: 'nginx -t' runs the checker against the installed configuration, which is still the old one. It passes, the new file is written unvalidated, and the parameter has produced a false sense of safety rather than safety.
• Quoting it wrong. The command is not run through a shell, so shell quoting does not apply and a path containing spaces is a problem the temporary path will not actually have.

Validators for a real deployment stack

The sshd entry is the one worth deploying first, because sshd is the service whose failure removes your ability to fix it. The man page describes -t as “Test mode. Only check the validity of the configuration file and sanity of the keys. This is useful for updating sshd reliably as configuration options may change.” That last sentence is the use case, written by the people who wrote the daemon.

sshd -T -f %s is the stronger relative: extended test mode, which checks validity and additionally prints the effective configuration. Using -T in validate works and discards the output; its real value is running it manually to see what a Match block actually resolves to.

For a service with no checker, the honest answer is that you have no validation, not that you should invent one. A grep for a keyword is not a parser and will pass files a parser rejects. Where a real checker does not exist, the protection has to come from the health gate after the restart and from a small first batch.

backup: true and what to do with the backup

backup defaults to false on both template and copy. Setting it writes the previous file alongside the new one with a timestamp before overwriting.

- name: Deploy the application configuration
ansible.builtin.template:
  src: myapp.conf.j2
  dest: /etc/myapp/myapp.conf
  mode: '0640'
  backup: true
  validate: '/usr/bin/myapp --check-config %s'
register: config_write
notify: Restart myapp

- name: Record the backup path in the run log
ansible.builtin.debug:
  msg: >-
    {{ inventory_hostname }}: previous configuration preserved at
    {{ config_write.backup_file }}
when: config_write.backup_file is defined

What validation does not prove

This is the part that matters most, because validate gives a real guarantee and people extend it past its edge.

It proves the file parses. It does not prove the configuration is correct. worker_processes 10000; parses. A proxy_pass to a hostname that does not resolve parses. A listen directive on a port another process already holds parses.

It validates the file, not the resulting configuration. For a service that assembles its configuration from an include glob, nginx -t -c %s validates the temporary file plus whatever the includes currently resolve to on that host. If your change is one drop-in among six and the interaction is the problem, the checker sees it — which is good. If your change is the whole file but a sibling drop-in is broken, the checker fails your task for someone else’s mistake, which is confusing until you know why.

It runs on the host, as the connection user. A validator needing root to read a key file behaves differently under become than without it. sshd -t checks key sanity, which means reading host keys, which means it needs the privileges to do so.

It says nothing about runtime resource acquisition. Binding a port, opening a file, connecting to a database, obtaining a certificate — all happen at start-up, all after validation has passed, and all are ordinary reasons a service with a perfectly valid configuration does not come back.

Knowledge check