Certificates, end to end · Ansible

A certificate has four failure modes and automation addresses them in descending order of how often they are handled:

It is missing or wrong — handled by everyone.
Its private key is readable by users who should not read it — handled by some.
The service is serving the old one because nothing reloaded — handled by fewer.
It expires on a Saturday — handled by almost nobody until it happens once.

This lesson is the sequence that addresses all four.

community.crypto is a collection, not part of ansible-core. It has to be installed before any of this runs:

Read-only / Safethe collection has to be present— Read-only in the sense that it touches only the controller collection path. Pin the version in requirements.yml; Part XXVII covers why.

ansible-galaxy collection list community.crypto
ansible-galaxy collection install community.crypto

The sequence

Three modules in order, each producing the input to the next. The order is not negotiable — a CSR needs a key, a certificate needs a CSR.

Configuration changekey, CSR, certificate— Parameter names verified against the module documentation for the latest collection release. The mode on the private key is the security-relevant line.

- name: Generate the private key
community.crypto.openssl_privatekey:
  path: /etc/ssl/private/{{ cert_common_name }}.key
  type: RSA
  size: 4096
  owner: root
  group: "{{ service_group }}"
  mode: '0640'

- name: Generate the certificate signing request
community.crypto.openssl_csr:
  path: /etc/ssl/csr/{{ cert_common_name }}.csr
  privatekey_path: /etc/ssl/private/{{ cert_common_name }}.key
  common_name: "{{ cert_common_name }}"
  organization_name: Example Organisation
  subject_alt_name: "{{ cert_sans }}"
  key_usage:
    - digitalSignature
    - keyEncipherment
  extended_key_usage:
    - serverAuth
  owner: root
  group: root
  mode: '0644'

- name: Sign the certificate against the internal CA
community.crypto.x509_certificate:
  path: /etc/ssl/certs/{{ cert_common_name }}.crt
  csr_path: /etc/ssl/csr/{{ cert_common_name }}.csr
  provider: ownca
  ownca_path: /etc/ssl/ca/internal-ca.crt
  ownca_privatekey_path: /etc/ssl/ca/internal-ca.key
  ownca_not_after: "+365d"
  owner: root
  group: root
  mode: '0644'
notify: reload web server

provider is required and takes acme, ownca or selfsigned. For a public certificate the shape is the same with provider: acme, preceded by acme_account and acme_certificate to complete the challenge; the deployment, permissions and reload half of this lesson is identical either way.

With subject_alt_name, values carry a prefix — DNS:, IP:, URI::

cert_sans:
  - "DNS:{{ cert_common_name }}"
  - 'DNS:app.example.com'
  - 'IP:192.0.2.10'

Permissions are the part that goes wrong

A certificate with the right content and the wrong mode is a finding, not a success

The private key is the asset. Everything else in the chain is public by design — the certificate is served to every client that connects.

The failure is a private key at 0644, readable by every user on the host, including any account a web application runs as and anything that achieves local execution. It is undetectable from the outside, the service works perfectly, and the certificate check on the dashboard is green.

The rule:

File	Owner	Group	Mode	Why
Private key	`root`	The service group	`0640`	The service reads it; nothing else can
Certificate	`root`	`root`	`0644`	Public, and served to clients anyway
CSR	`root`	`root`	`0644`	Contains no secret
CA private key	`root`	`root`	`0600`	Nothing but the signing process

openssl_privatekey documents that the key “will have 0600 mode if mode is not explicitly set”, which is a good default — and it becomes a problem in the other direction when a service that does not run as root cannot read its own key. 0640 with the service group is the shape that works, and it must be stated rather than left to the default.

A validation assertion, so the mode is checked rather than assumed:

Read-only / Safeassert the permissions, do not assume them— stat is read-only. This belongs in the validation play, because a mode set correctly in March can be changed by anything in the intervening months.

- name: Read the private key metadata
ansible.builtin.stat:
  path: /etc/ssl/private/{{ cert_common_name }}.key
register: key_stat
check_mode: false

- name: The private key is not readable beyond its group
ansible.builtin.assert:
  that:
    - key_stat.stat.exists
    - key_stat.stat.pw_name == 'root'
    - key_stat.stat.mode is match('06[04]0')
  fail_msg: >-
    Private key on {{ inventory_hostname }} is
    {{ key_stat.stat.mode | default('missing') }} owned by
    {{ key_stat.stat.pw_name | default('nobody') }}
  quiet: true

Reload, not restart

The handler reloads. It matters, and the reason generalises.

Service impact possiblehandlers/main.yml— A reload re-reads configuration without dropping established connections. A restart drops them.

- name: reload web server
ansible.builtin.systemd_service:
  name: nginx
  state: reloaded

A restart terminates the process and every connection it holds. Across a fleet of 40 web servers on a certificate renewal, that is 40 simultaneous connection drops for a change that did not require any of them.

Two caveats keep this from being a rule you apply blindly.

Not every service picks up a certificate on reload. Some read certificate material only at startup. Confirm it for the specific service rather than assuming; the validation step below is what tells you, because it checks the certificate the service is actually serving.

A reload does not survive a bad certificate. If the new file is malformed the reload fails and the service continues with the previous one — which is the safe outcome, and it means the play must notice the failed reload rather than treating it as a no-op.

Expiry is a fact, not a calendar reminder

The fourth failure mode. x509_certificate_info reads a certificate and returns not_after as an ASN.1 time, expired as a boolean, and — the useful one — valid_at, which the documentation describes as returning, for each named time specification, “a boolean whether the certificate is valid at that point in time or not”.

Read-only / Saferenewal scheduled from the certificate itself— valid_at asks the question directly: will this still be valid in 30 days? The answer is a boolean, not a date to subtract.

- name: Read the deployed certificate
community.crypto.x509_certificate_info:
  path: /etc/ssl/certs/{{ cert_common_name }}.crt
  valid_at:
    in_two_weeks: "+14d"
    in_a_month: "+30d"
register: cert_info
check_mode: false

- name: The certificate is valid now
ansible.builtin.assert:
  that:
    - not cert_info.expired
  fail_msg: >-
    EXPIRED certificate on {{ inventory_hostname }}:
    not_after {{ cert_info.not_after }}
  quiet: true

- name: The certificate is not inside the renewal window
ansible.builtin.assert:
  that:
    - cert_info.valid_at.in_a_month
  fail_msg: >-
    {{ inventory_hostname }} certificate expires within 30 days
    (not_after {{ cert_info.not_after }}) - schedule renewal
  quiet: true

The two assertions are deliberately different findings. The first is an incident. The second is a work item with a month of lead time, and it is the one that prevents the first.

Check the certificate the service is serving, not the file on disk

x509_certificate_info reads a file. A service can be serving something else: it was reloaded before the new file was written, the reload failed, it reads from a different path than the one the role manages, or the file is a bundle whose leaf is not the first entry.

The file check is necessary and it is not sufficient. The complete validation reads the file and connects:

- name: The served certificate matches what we deployed
  community.crypto.get_certificate:
    host: "{{ ansible_host }}"
    port: 443
  register: served
  delegate_to: localhost
  check_mode: false

Comparing the served certificate’s not_after against the file’s is what closes the gap between “the right certificate is on disk” and “the right certificate is being served” — which is the same gap as template succeeded, restart succeeded, health check failed.

Why the modules are idempotent, and the case where they are not

These modules do not regenerate on every run, and understanding what makes them decide is what stops a scheduled compliance play from issuing a new key every hour.

Each module compares the existing file against the parameters it was given. openssl_privatekey checks whether a key exists at path with the requested type and size; if so, it reports ok and does nothing. openssl_csr compares the CSR’s subject, SANs and key usage against the requested values, and regenerates only on a mismatch. x509_certificate checks whether the certificate matches its CSR and is still valid.

Two consequences.

Changing a parameter regenerates. Adding a SAN rewrites the CSR, which makes the certificate no longer match it, which re-signs. That chain is correct and it is worth knowing it is a chain, because a one-line change to a variable produces a new certificate on every host in the group.

force: true breaks the chain deliberately, and on a private key it is destructive in the way this course cares about: the old key is gone, every certificate issued against it is orphaned, and any client pinning it fails. It belongs in a deliberate rotation, not in a role that runs on a schedule.

The case where idempotence does not save you is expiry. A certificate approaching its not_after still matches its CSR, so x509_certificate reports ok and changes nothing — right up until it expires, at which point it still matches its CSR. Nothing in the generation sequence renews on expiry by itself. Renewal is driven by the valid_at check above, feeding a decision to regenerate — which is why expiry monitoring is part of the lifecycle rather than a separate monitoring concern someone else owns.

What happens if the certificate is renewed but the service is never reloaded?

Everything looks correct and the service serves the old certificate until it restarts for an unrelated reason — which might be next week, or might be the night the old one expires.

Three causes, and they need different fixes:

The handler did not fire. x509_certificate reported ok because the certificate still matched its CSR, so nothing notified. This is the expiry case above: renewal has to be driven by the valid_at check, not by the generation task’s change status.
The handler fired and failed. The reload returned non-zero and the play carried on, or the failure was inside a rescue. The validation play catches it, and only if it checks the served certificate rather than the file.
The service does not re-read certificates on reload. Some do not. The fix is a restart, scheduled deliberately with the batching the fleet needs, rather than 40 at once.

The general defence is the same in all three: a certificate change is not complete until something has connected to the service and confirmed what it is serving. File on disk, handler notified, handler succeeded — none of those is that confirmation, and the first is the one people check.

Knowledge check

Why this matters

Certificate expiry is the most predictable outage in infrastructure. The date is known at issuance, printed in the certificate, and readable by anything that cares to look.

It still takes services down constantly, and the reason is structural rather than technical: the deployment and the expiry are separated by a year. Whoever deployed it has moved on, the calendar reminder was in a personal calendar, the wiki page is stale, and the renewal became somebody’s problem at 22:00 on a Saturday because that is when the clock ran out.

Every part of the mechanism to prevent this exists and is in this lesson. valid_at turns “when does this expire” into a boolean the play already knows how to act on. The check costs one task in a play that is already running. What it buys is that expiry arrives as a work item with thirty days of notice, in a report somebody reads, rather than as an incident.

The same argument covers the private key mode, which nothing external will ever tell you about, and the served-certificate check, which is the only thing that distinguishes a renewal that worked from one that only wrote a file.