One role, many environments

The premise of having a staging environment is that the automation which runs there is the automation that will run in production. Byte for byte, task for task. If staging runs different code, the staging run proves something about code that will never touch a customer.

That premise is broken by a single line, and it is a line that looks entirely reasonable when you write it:

# roles/webserver/tasks/main.yml
- name: Enable strict TLS settings
  ansible.builtin.template:
    src: tls-strict.conf.j2
    dest: /etc/nginx/conf.d/tls.conf
  when: environment_name == 'production'

Read that as a test-coverage statement rather than a configuration statement. It says: this task has never executed anywhere except production. The template has never been rendered on a machine you could afford to break. The first time it runs is the time it matters.

Why the branch is worse than it looks

Three separate problems compound.

The production path has no test. Every other task in the role has been run hundreds of times in staging. This one has been run in production only, by definition, and every change to it goes straight from a code review to the fleet.

The staging result is misleading rather than merely incomplete. A green staging run reports success for a role in which the riskiest task was skipped. skipped=1 is in the recap, and nobody reads the recap looking for a skip that should have been an ok.

The branches multiply. One when: environment_name == 'production' invites the next. After a year the role has six, they interact, and the number of combinations that have actually been executed anywhere is one: the combination production happens to be in.

Parameterise instead

The transformation is mechanical. The environment name decides a value; the role acts on the value.

# roles/webserver/defaults/main.yml
webserver_tls_profile: modern       # safe for every environment
webserver_worker_processes: 2

# roles/webserver/tasks/main.yml
- name: Install the TLS configuration
  ansible.builtin.template:
    src: "tls-{{ webserver_tls_profile }}.conf.j2"
    dest: /etc/nginx/conf.d/tls.conf
    mode: '0644'
    validate: 'nginx -t -c %s'
  notify: Reload nginx

# inventories/production/group_vars/webservers/30-tls.yml
webserver_tls_profile: strict

Now the task runs in every environment. What differs is which template gets rendered, and staging can be pointed at the strict profile whenever you want to prove it works.

The general rule: a role may branch on what a host is — its distribution, whether a service is present, what a fact reports. It may not branch on where the host lives. The first is a property the role can discover and legitimately must handle. The second is a decision the caller has already made by choosing an inventory.

The silent divergence you already have

Removing the conditionals is necessary and not sufficient. Environments still resolve different values, and a value that differs by accident is the same defect wearing a different hat.

Here is the same role and the same playbook against two inventories:

$ ansible-playbook -i inventories/production playbooks/report.yml

ok: [web-prod-01.example.com] => {
  "msg": "env=production workers=16 tls=False"
}
ok: [web-prod-02.example.com] => {
  "msg": "env=production workers=32 tls=False"
}

$ ansible-playbook -i inventories/staging playbooks/report.yml

ok: [web-stg-01.example.com] => {
  "msg": "env=staging workers=2 tls=False"
}

Two workers in staging, sixteen and thirty-two in production. Both runs green. If the change under test is anything to do with concurrency — connection pooling, file descriptor limits, a shared cache — staging did not test it.

Nobody decided this. Staging simply never grew a group_vars/webservers/ directory, and the fallback to the role default was invisible because a fallback is not an error.

Exercising the production parameter set in staging

The cheapest fix is to run the staging inventory with production’s parameter file forced in. Extra vars sit at the top of the precedence ladder, so -e @file overrides whatever staging’s own layers say:

$ ansible-playbook -i inventories/staging playbooks/report.yml -e @inventories/production/group_vars/webservers/20-limits.yml

TASK [webserver : Report the effective parameter set] **************************
ok: [web-stg-01.example.com] => {
  "msg": "env=staging workers=16 tls=False"
}

Note carefully what this is and is not.

It is a way to prove that the role tolerates the production parameter set: the template renders, the validation command passes, the service accepts the configuration, the second run reports no changes.

It is not a load test, and it is not evidence that 16 workers is correct for production’s hardware. Staging has different machines. What you have proven is that the automation handles the values, which is the part the repository is responsible for.

Knowledge check