Lab 20: Write and prove a least-privilege secret policy

Objective

A policy that has never been tested is a claim, not a control. Teams write a policy, attach it to an application, watch the application work, and conclude that the policy is correct. What they have proved is that the allowed operation is allowed. They have proved nothing about the operations they believe are forbidden, and those are the ones an auditor and an attacker both care about.

In this lab you write a policy of four lines that grants exactly one capability on exactly one path. You then run four operations against it and demand a specific result from each: one success and three distinct refusals. The three refusals are not the same refusal repeated. One is a neighbouring path the token has no grant on at all. One is a write to the very path it may read. One is a directory listing, which fails for a reason that has nothing to do with the first two and which catches out almost everyone the first time.

Finally you go to the audit log and find the record of a denial, because a control you cannot evidence after the fact is not a control you can defend in an incident review.

Architecture

A single OpenBao container holds a KV version 2 secrets engine mounted at kv/ with two secrets under it. Two identities interact with it: the root token, used only to set the scene, and a scoped token carrying one policy, used for every test. The audit device writes JSON records to a host directory you can read directly.

The structure worth drawing is the one that produces the fourth outcome. A KV version 2 mount does not expose the paths its command line appears to use.

flowchart TD
    A["bao kv get kv/app/config"] --> B["GET /v1/kv/data/app/config"]
    C["bao kv put kv/app/config"] --> D["PUT /v1/kv/data/app/config"]
    E["bao kv list kv/app"] --> F["LIST /v1/kv/metadata/app"]
    B --> G["policy grants read on kv/data/app/config"]
    D --> G
    F --> H["no grant on kv/metadata/app"]

The command line hides a split. Reading and writing a secret both go to a path under kv/data/, where the policy has a grant, so the read succeeds and the write is refused on the capability rather than on the path. Listing goes somewhere else entirely, to kv/metadata/, where the policy has no grant of any kind. Read access and list access are separate decisions in this engine, and a policy written against the command line rather than the API grants neither reliably.

Requirements

• Docker with a working daemon, and permission to run containers as your user.
• OpenBao 2.6.x, pulled as the openbao/openbao container image.
• Roughly 65 minutes and about 200 MB of free disk in your home directory.
• No out-of-band access requirement. Nothing here touches SSH, the firewall, the primary interface, or /etc/fstab. Every container, network and directory is named with an rbpki- prefix.

Scenario

An internal service needs one database password. The request that arrives in your queue asks for read access to the team’s secret area. You are going to grant read access to one secret, and then demonstrate to the requesting team exactly what their credential can and cannot do, so that nobody discovers the boundary in production by walking into it.

Tasks

Task 1 — Record the starting state and prepare the server directory

LAB="$HOME/rbpki-lab20"
# The secret manager writes its storage as root inside the container, so hand
# the tree back before removing it from the host.
if [ -d "$LAB" ]; then
  docker run --rm -v "$LAB:/w" alpine:3.22 \
    sh -c 'rm -rf /w/* /w/.[!.]* 2>/dev/null || true'
fi
rm -rf "$LAB"
mkdir -p "$LAB/config" "$LAB/data" "$LAB/audit"
cd "$LAB"

docker ps -a --format '{{.Names}}' | sort > "$LAB/state.pre-lab"
docker network ls --format '{{.Name}}' | sort >> "$LAB/state.pre-lab"
chmod 777 "$LAB/data" "$LAB/audit"
grep -c . "$LAB/state.pre-lab"

Task 2 — Declare the audit device before anything else exists

cat > "$LAB/config/bao.hcl" <<'EOF'
audit "file" {
  type    = "file"
  path    = "file/"
  options = { file_path = "/openbao/audit/audit.log" }
}

storage "file" {
  path = "/openbao/data"
}

listener "tcp" {
  address     = "0.0.0.0:8200"
  tls_disable = true
}

disable_mlock = true
api_addr      = "http://127.0.0.1:8200"
EOF

docker network create rbpki-net-20
docker run -d --name rbpki-bao-20 --network rbpki-net-20 \
  --cap-add=IPC_LOCK --user root \
  -v "$LAB/config:/openbao/config" \
  -v "$LAB/data:/openbao/data" \
  -v "$LAB/audit:/openbao/audit" \
  openbao/openbao:latest server -config=/openbao/config/bao.hcl

sleep 6
docker inspect -f '{{.State.Status}}' rbpki-bao-20

The audit stanza is written first in the file purely as a reminder of the operational order: the device has to exist before the events you will want to read. HCL does not care about stanza order, but the person editing the file next does. Audit devices cannot be added through the API in this release, so if this stanza is wrong you will not be able to correct it at runtime.

Task 3 — Initialise, unseal and set the scene

docker exec -e BAO_ADDR=http://127.0.0.1:8200 rbpki-bao-20 \
  bao operator init -key-shares=1 -key-threshold=1 > "$LAB/init.raw" 2>&1

umask 077
grep 'Unseal Key 1:' "$LAB/init.raw" | awk '{print $NF}' > "$LAB/share-1"
grep 'Initial Root Token:' "$LAB/init.raw" | awk '{print $NF}' > "$LAB/root-token"
chmod 600 "$LAB/share-1" "$LAB/root-token"

docker exec -e BAO_ADDR=http://127.0.0.1:8200 rbpki-bao-20 \
  bao operator unseal "$(cat "$LAB/share-1")" | grep -E 'Sealed|Threshold'

BAO_TOKEN="$(cat "$LAB/root-token")"
export BAO_TOKEN

A single share with a threshold of one is a deliberate simplification. The ceremony is not the subject of this lab and a one-share split removes a step you have already rehearsed elsewhere. Never initialise a real server this way: one share means one person can bring the service up alone and one lost file can lose it for ever.

Task 4 — Create the secrets the policy will govern

docker exec -e BAO_ADDR=http://127.0.0.1:8200 -e BAO_TOKEN="$BAO_TOKEN" rbpki-bao-20 \
  bao secrets enable -path=kv -version=2 kv

docker exec -e BAO_ADDR=http://127.0.0.1:8200 -e BAO_TOKEN="$BAO_TOKEN" rbpki-bao-20 \
  bao kv put kv/app/config db_user=appuser db_password=lab-only-not-real

docker exec -e BAO_ADDR=http://127.0.0.1:8200 -e BAO_TOKEN="$BAO_TOKEN" rbpki-bao-20 \
  bao kv put kv/app/other other=value

The first command prints Success! Enabled the kv secrets engine at: kv/. The two writes each print the metadata block for the version they created. Two secrets now sit side by side under the same prefix, which is what makes the second test meaningful: the neighbouring secret is not hidden away somewhere obscure, it is the next entry in the same directory.

Task 5 — Write the policy

cat > "$LAB/app-read.hcl" <<'EOF'
path "kv/data/app/config" {
  capabilities = ["read"]
}
EOF

docker cp "$LAB/app-read.hcl" rbpki-bao-20:/tmp/app-read.hcl

docker exec -e BAO_ADDR=http://127.0.0.1:8200 -e BAO_TOKEN="$BAO_TOKEN" rbpki-bao-20 \
  bao policy write app-read /tmp/app-read.hcl

The command reports Success! Uploaded policy: app-read. Read the policy again and notice what is absent. There is no deny block, and there does not need to be one. Policies are deny by default, so an empty policy grants nothing at all, and every capability you do not name is already withheld.

Task 6 — Issue a token bound to the policy

docker exec -e BAO_ADDR=http://127.0.0.1:8200 -e BAO_TOKEN="$BAO_TOKEN" rbpki-bao-20 \
  bao token create -policy=app-read -ttl=30m -field=token > "$LAB/app-token" 2>&1
chmod 600 "$LAB/app-token"

APP_TOKEN="$(cat "$LAB/app-token")"
printf 'scoped token issued, first characters only: %.10s\n' "$APP_TOKEN"

The token carries two policies rather than one. app-read is the one you wrote; default is a built-in policy attached to every token, which cannot be removed and which grants a small set of self-service operations such as looking up and renewing the token itself. Any audit of an application’s effective permissions has to account for default as well as the policy someone deliberately attached.

Task 7 — Outcome one: prove the operation that must succeed

$ docker exec -e BAO_ADDR=http://127.0.0.1:8200 -e BAO_TOKEN="$APP_TOKEN" rbpki-bao-20 bao kv get kv/app/config

=== Secret Path ===
kv/data/app/config

======= Metadata =======
Key                Value
---                -----
created_time       2026-08-26T21:26:20.073241275Z
custom_metadata    <nil>
deletion_time      n/a
destroyed          false
version            1

======= Data =======
Key            Value
---            -----
db_password    lab-only-not-real
db_user        appuser

The header line is the most useful thing on the screen. The command asked for kv/app/config and the server answered about kv/data/app/config. That is the API path, and it is the path your policy must name. Write the command-line form into a policy and the grant matches nothing.

Task 8 — Outcomes two, three and four: prove the three refusals

Run all three and capture them together, because the matrix is the deliverable:

APP_TOKEN="$(cat "$HOME/rbpki-lab20/app-token")"
LAB="$HOME/rbpki-lab20"

{
  echo '## 2. a neighbouring path'
  docker exec -e BAO_ADDR=http://127.0.0.1:8200 -e BAO_TOKEN="$APP_TOKEN" rbpki-bao-20 \
    bao kv get kv/app/other 2>&1
  echo "exit=$?"
  echo '## 3. a write to the readable path'
  docker exec -e BAO_ADDR=http://127.0.0.1:8200 -e BAO_TOKEN="$APP_TOKEN" rbpki-bao-20 \
    bao kv put kv/app/config x=y 2>&1
  echo "exit=$?"
  echo '## 4. a list of the enclosing directory'
  docker exec -e BAO_ADDR=http://127.0.0.1:8200 -e BAO_TOKEN="$APP_TOKEN" rbpki-bao-20 \
    bao kv list kv/app 2>&1
  echo "exit=$?"
} > "$LAB/outcome-matrix.txt"

grep -E 'URL:|Code:|permission denied' "$LAB/outcome-matrix.txt"

$ docker exec -e BAO_ADDR=http://127.0.0.1:8200 -e BAO_TOKEN="$APP_TOKEN" rbpki-bao-20 bao kv get kv/app/other

Error reading kv/data/app/other: Error making API request.

URL: GET http://127.0.0.1:8200/v1/kv/data/app/other
Code: 403. Errors:

* 1 error occurred:
* permission denied

$ docker exec -e BAO_ADDR=http://127.0.0.1:8200 -e BAO_TOKEN="$APP_TOKEN" rbpki-bao-20 bao kv put kv/app/config x=y

Error writing data to kv/data/app/config: Error making API request.

URL: PUT http://127.0.0.1:8200/v1/kv/data/app/config
Code: 403. Errors:

* 1 error occurred:
* permission denied

The path in that URL is identical to the one the successful read used. The only difference is the HTTP verb, and the policy named read rather than create or update. This is the outcome that matters most in a compromise: a leaked read-only token cannot overwrite the secret it can see, so it cannot be used to lock the application out of its own database.

$ docker exec -e BAO_ADDR=http://127.0.0.1:8200 -e BAO_TOKEN="$APP_TOKEN" rbpki-bao-20 bao kv list kv/app

Error listing kv/metadata/app: Error making API request.

URL: GET http://127.0.0.1:8200/v1/kv/metadata/app?list=true
Code: 403. Errors:

* 1 error occurred:
* permission denied

Compare the three URLs. Two of them are under kv/data/; the third is under kv/metadata/. Read access does not imply list access in this engine, and it never will, because they are decisions about different paths. An application that needs to enumerate secrets needs a second, explicit grant of list on the metadata path, and granting it tells anyone reading the policy that this identity can see the names of every secret in that area.

Task 9 — Read the audit record of a denial, and prove the value never leaked

LAB="$HOME/rbpki-lab20"

docker exec rbpki-bao-20 sh -c "grep 'app/other' /openbao/audit/audit.log | tail -1" \
  > "$LAB/denied-record.json"
wc -c "$LAB/denied-record.json"

{
  echo '# does the secret value appear anywhere in the audit log?'
  docker exec rbpki-bao-20 sh -c "grep -c 'lab-only-not-real' /openbao/audit/audit.log"
} > "$LAB/value-leak-check.txt" 2>&1
cat "$LAB/value-leak-check.txt"

The grep returns 0. The audit device recorded every one of the four requests you made, including the one that returned the secret, and the value lab-only-not-real appears in none of them. Values in an audit record are replaced by a keyed HMAC, so a record can be correlated with another record about the same value without ever revealing it.

Here is the denial record from the second outcome, trimmed to the fields that matter:

{"time":"2026-08-26T21:26:20.555636512Z","type":"response",
 "auth":{"client_token":"hmac-sha256:da33377eba1c...",
         "accessor":"hmac-sha256:5eb6ce9e...",
         "display_name":"token",
         "policies":["app-read","default"],
         "token_policies":["app-read","default"],
         "policy_results":{"allowed":false},
         "token_type":"service","token_ttl":1800},
 "request":{"operation":"read","mount_point":"kv/","mount_type":"kv",
            "path":"kv/data/app/other","remote_address":"127.0.0.1"},
 "response":{"data":{"error":"hmac-sha256:b0a0f532..."}},
 "error":"1 error occurred:\n\t* permission denied\n\n"}

Three fields turn this from a log line into evidence. "policy_results":{"allowed":false} is the authorisation decision, recorded as a decision rather than inferred from an error string. "path":"kv/data/app/other" is the API path the caller actually reached for, which is what you want when someone claims they never touched a secret. And "policies":["app-read","default"] names the policy set in force at the moment of the decision, so a later change to app-read cannot rewrite what was true at the time.

Task 10 — Capture the deliverables

cd "$HOME/rbpki-lab20"
ls -l app-read.hcl outcome-matrix.txt denied-record.json value-leak-check.txt
grep -c 'permission denied' outcome-matrix.txt
grep -c 'allowed":false' denied-record.json

Validation

• The read in Task 7 exits 0 and prints a ======= Data ======= block containing db_password. If it returns 403, the policy path is wrong, almost always because it names kv/app/config instead of kv/data/app/config.
• grep -c 'permission denied' outcome-matrix.txt returns 3. A count of 2 means one operation unexpectedly succeeded, and the usual cause is that the commands were run with BAO_TOKEN still set to the root token rather than the scoped one.
• grep -c 'kv/metadata/app' outcome-matrix.txt returns at least 1, proving the list request really went to the metadata tree. If the string is absent, the list was never attempted.
• grep -c 'allowed":false' denied-record.json returns 1. An empty file means the audit device was not active, which you can confirm with bao audit list -detailed.
• cat value-leak-check.txt shows a count of 0. Any other number means a plaintext secret has reached the audit log and the log must be treated as secret material.

Expected Outcome

$HOME/rbpki-lab20/
├── audit/
│   └── audit.log
├── config/
│   └── bao.hcl
├── data/
├── app-read.hcl
├── app-token
├── denied-record.json
├── outcome-matrix.txt
├── root-token
├── share-1
├── state.pre-lab
└── value-leak-check.txt

You can now hand a requesting team a one-page answer that says what their credential does, backed by four terminal captures rather than by an assurance. You can also answer the question that follows it: prove that the refusal happened, and prove the log you are using as evidence does not itself contain the secret.

Troubleshooting

Every operation succeeds, including the ones that should fail. BAO_TOKEN is still the root token. The root policy bypasses everything. Re-export APP_TOKEN and pass it explicitly on each docker exec, as the commands in Task 8 do.

The read fails with a 403 as well. Print the policy back with bao policy read app-read and compare it character by character with the path in the successful URL. A missing data/ segment is the usual fault; a trailing slash is the next.

bao kv list kv/app returns an empty list instead of a 403. The token in use has a list grant from another policy, most often because it was created with -policy=app-read -policy=default plus something inherited. Create a fresh token with only app-read.

The audit log file is empty or missing. The container could not write to the mounted directory. Confirm with docker logs rbpki-bao-20, re-run the chmod 777 from Task 1, and remember that this release will not let you add the device through the API afterwards.

Cleanup

LAB="$HOME/rbpki-lab20"

# 1. Stop and forget the service.
docker rm -f rbpki-bao-20 || true
docker network rm rbpki-net-20 || true

# 2. Compare the host against the Task 1 capture.
docker ps -a --format '{{.Names}}' | sort > "$LAB/state.post-lab"
docker network ls --format '{{.Name}}' | sort >> "$LAB/state.post-lab"
diff "$LAB/state.pre-lab" "$LAB/state.post-lab" || true

# 3. Remove the lab directory, including tokens and the share.
# The secret manager writes its storage as root inside the container, so hand
# the tree back before removing it from the host.
if [ -d "$LAB" ]; then
  docker run --rm -v "$LAB:/w" alpine:3.22 \
    sh -c 'rm -rf /w/* /w/.[!.]* 2>/dev/null || true'
fi
rm -rf "$LAB"

To confirm restoration, run docker ps -a --format '{{.Names}}' | grep -c rbpki- and docker network ls --format '{{.Name}}' | grep -c rbpki-; both must return 0, and ls -d "$HOME/rbpki-lab20" must report that the directory is gone. The diff in step 2 prints nothing on a clean run.

Production notes

• Test the denials in the pipeline, not by hand. The four-outcome matrix in this lab is a script. Run it against every policy change in a non-production instance and fail the change if any of the three refusals stops being a refusal.
• Prefer a wildcard you can explain. A trailing * is a prefix match and is legal only as the final character of a path. A + matches within a single segment. kv/data/app/+/config is a narrower and far more reviewable grant than kv/data/app/*.
• Grant list reluctantly. Enumeration is how an attacker turns one stolen credential into a map of the estate. A workload that knows the name of the secret it needs does not need to list.
• Keep audit logs out of the application logging pipeline. They contain HMACed identifiers that are safe to retain, and they are the only record of a refusal. Ship them somewhere the application team cannot rotate away.

What You Learned

• A policy is proved by its refusals. One success and three specific refusals is the minimum evidence for a least-privilege grant.
• Policies are written against API paths. kv/data/app/config, not the kv/app/config the command line shows you.
• Read does not imply write, and read does not imply list. The write refusal shares a URL with the successful read; the list refusal has a different URL entirely, under kv/metadata/.
• Unset and deny are different tools. Unset withholds; deny vetoes across every attached policy, including over sudo.
• The audit log records the decision, not just the error. "policy_results":{"allowed":false} is the field to search for, and the secret value is never in the file beside it.