Writing Effective Policy Rules

A policy rule is a piece of code. It has tests, it has a review path, and it has failure modes. The common mistake is to write a rule, ship it, and treat it as finished because the plan passes. The uncommon practice is to write the test cases first, watch them fail, then write the rule that makes them pass. The second practice scales to a rule set of 200+; the first does not.

Deny by default

A rule that asks “is this allowed?” is easier to misread than a rule that asks “is this denied?”. The deny-by-default structure makes the policy intent explicit and the rule legible to the engineer reading the failure.

package terraform.regions
import future.keywords.if
import future.keywords.contains

approved := {"eu-west-2", "eu-west-1", "us-east-1"}

deny contains msg if {
  rc := input.resource_changes[_]
  region := rc.change.after.region
  not approved[region]
  msg := sprintf("region %q is not in the approved list", [region])
}

The default is “deny”. The set of approved regions is the allow clause. A region not in the set fails. The rule does not need to enumerate every disallowed region — only the small allow list.

The right abstraction level

A rule can evaluate at three levels of abstraction:

1. Attribute level
   "AWS S3 bucket server_side_encryption_configuration is set"
   Specific. Easy to read. Easy to write. Brittle.

2. Property level
   "Taggable resources in this account have the required tags"
   Generalised. Survives schema drift. Less obvious.

3. Invariant level
   "Public storage is not exposed to the internet"
   Highest abstraction. Survives resource renames.
   Hardest to write correctly.

The right level depends on the rule. A rule that enforces an S3-specific behaviour is fine at the attribute level — the attribute is named and stable. A rule that enforces “no public exposure” is better at the invariant level, because public exposure can come from many resources (S3, IAM, security groups) and the attribute-level enumeration will miss one.

Test-first authoring

The workflow that scales:

1. Author the test cases as data
2. Run the rule against the tests; expect failures
3. Write the rule
4. Re-run; expect green
5. Commit rule and tests together
6. CI runs the tests on every PR to the policy repository

The test cases are data files in the policy repository. One folder per rule. Conftest and OPA call this testdata/.

policy/
  regions.rego
  regions_test.rego
  regions/
    positive.json   # known-bad plan; expect deny
    negative.json   # known-good plan; expect no deny

The test files are tiny Terraform plans. Each plan contains one resource that the rule should care about, with attributes chosen to test a single branch.

{
  "resource_changes": [
    {
      "address": "aws_instance.web",
      "type": "aws_instance",
      "change": {
        "after": { "region": "us-east-1" }
      }
    }
  ]
}

The test in regions_test.rego:

package terraform.regions

test_approved_region_allowed if {
  result := count(deny) == 0 with input as data.regions.negative
}

test_unapproved_region_denied if {
  result := count(deny) == 1 with input as data.regions.positive
}

Run the tests:

conftest test --policy policy/ --all-namespaces

region: PASS
2 / 2 tests passed

For Sentinel:

sentinel test

PASS - regions.sentinel
2 tests, 0 failures

What good test coverage looks like

A rule with adequate test coverage tests:

• Happy path — the canonical resource that should be allowed or denied.
• Negative path — the resource that the rule rejects, with the expected denial message.
• Edge of the deny list — values just inside and just outside the allow set ("eu-west-2" allowed, "eu-central-1" denied).
• Resource-type boundary — the rule must not trip on resources that are not in the selector (e.g. a non-aws resource when the rule is aws_*-specific).
• Plan-time vs apply-time — both create and update actions; both before and after states for update.
• Empty plan — the rule must not crash on an empty resource_changes array.

Six categories. Most shops that write their first three and stop end up rewriting the rule after a corner case finds it.

When the rule set has outgrown its tests

Five failure modes to recognise:

1. Rule passes the wrong plan. The author tested the rule on aws_instance and committed. A null_resource in the same plan has the region field and the rule silently approves. Mitigation: property-test with a synthesis tool, or accept that the rule is selective by resource type and document that.
2. False positive after provider upgrade. The provider changes the schema. The rule still matches, but the attribute no longer means what the rule author thought. The test catches this only if the test plan uses the new schema. Re-run the tests against a fresh plan output after every provider upgrade.
3. Rule author left; rule author not replaced. A rule set in a personal style is hard to extend. The team writes a style guide at policy-repo bootstrapping; reviews the guide at every onboarding.
4. Rule blocks a legitimate workload; a one-off override becomes the pattern. The rule’s failure mode has been found, and the workaround has been encoded somewhere. The exception should be in the rule file itself; the override flag is a regression.
5. Stale deny message. The rule denies “S3 must have SSE encryption” — but the message cites a different rule ID. Engineers cannot search for the failing rule. The error message must be stable, machine-parseable, and human-readable. Test the message too.

Authoring workflow in practice

A concrete sequence, end-to-end. The rule: tags must include Owner on every taggable resource.

1. Open the policy repository. Identify the file: policy/tags.rego.
2. Add a testdata/tags/positive.json with an aws_instance that has no Owner tag.
3. Add a testdata/tags/negative.json with the same resource type but with tags.Owner = "platform".
4. Write tags_test.rego. Tests expect: positive is denied; negative is allowed. The tests fail; the rule does not exist yet.
5. Write tags.rego as deny-by-default.
6. Run conftest test. The tests pass.
7. Open a PR. The PR adds tags.rego, tags_test.rego, and testdata/tags/*.json. The CI runs the tests.
8. Merge. The policy is live for the next run.

The author never has a moment where the rule is “live but untested”. The rule and its tests are merged as one commit.

Validation

The CI pipeline asserts the rule set is healthy:

conftest verify --policy policy/
opa test policy/ -v
sentinel test

For the policy repository itself, run a lint:

opa fmt --diff policy/

Diff:
--- policy/regions.rego.orig
+++ policy/regions.rego
@@
- approved := { "eu-west-2", "eu-west-1", "us-east-1" }
+ approved := {"eu-west-2", "eu-west-1", "us-east-1"}

A git diff step in CI catches formatter drift.

opa fmt --list policy/

policy/regions.rego

Any file in the list is unformatted; the CI step fails.

Security and performance

The policy repository is high-value. Treat it as such.

• Branch protection. The default branch is protected. PR reviews are required; at least one maintainer must approve.
• Signed commits. Each commit is signed. The git history is the audit trail.
• No external writers. CI is the only writer to the default branch; PRs are merged from forks only.
• Dependency pinning. Conftest, OPA, and Sentinel are pinned to a known version. The policy tests run on the pinned version.

Performance is dominated by terraform plan. The policy run on a 200-resource plan is in the low seconds. A rule set with 100 rules is still well under the timeout a CI step allows.

What comes next

A well-authored rule set solves most policy problems at the attribute and property levels. The next lessons cover the specific policy domains — tags, encryption, network — that the rule set is most likely to target.

Verification

conftest test --policy policy/ --output json

{
  "passed": 12,
  "failed": 0,
  "skipped": 0
}

For Sentinel:

sentinel test -verbose

PASS - regions.sentinel
  test_approved_region_allowed
  test_unapproved_region_denied
PASS - tags.sentinel
  ...
5 policies, 10 tests, 0 failures, 0 errors

Any non-zero failed count is a broken rule. CI fails the merge. The policy repository cannot reach the default branch in a failing state.