Audit without leaking: what to record and what must never be recorded

An audit trail for a secret manager has an obligation that most logging systems do not: it must prove who read which credential and when, while never becoming a second copy of those credentials. Those two requirements pull in opposite directions, and how a product resolves the tension tells you more about its design than any feature list.

The two obligations, stated as a boundary

The line runs between the shape of a request and its payload.

The left column is what an investigation needs. The right column is what turns the log server, its backups and its retention policy into an extension of the barrier you spent the whole of this part protecting. A log aggregator with a secret value in it has a completely different threat model from a log aggregator, and nobody who runs it has been told.

In OpenBao 2.6 you cannot enable a device through the API

Every tutorial written for HashiCorp Vault opens with an audit enable command. On a stock OpenBao v2.6.2 build that command fails, and the refusal is the lesson:

$ bao audit enable file file_path=/tmp/bao-audit.log
Error enabling audit device: Error making API request.

URL: PUT http://127.0.0.1:8200/v1/sys/audit/file
Code: 400. Errors:

* cannot enable audit device via API; use declarative, config-based audit device management instead

This is a deliberate default, not a regression. The server option unsafe_allow_api_audit_creation defaults to false, and has been required since v2.3.2 for API creation to work at all. The stated rationale is exactly the risk the name implies: enabling a device through the API allows the operator to create files at arbitrary locations on the host system or send network requests to arbitrary addresses, which may have unwanted effects. In other words, the audit-enable endpoint is a file-write and network-connect primitive dressed as an observability feature, and it is now off unless a human turns it on in the configuration file and reloads.

The replacement is a top-level audit stanza in the server configuration, alongside storage, listener and seal. This form was executed on v2.6.2 and produced a working device:

audit "file" {
  type    = "file"
  path    = "file/"
  options = { file_path = "/openbao/audit/audit.log" }
}

The upstream v2.6.2 documentation shows a two-label header, audit "file" "to-stdout", with options written as a nested block rather than an assignment. HCL accepts both shapes, so read the header as naming the device type and its mount path however your configuration expresses it, and confirm the result rather than trusting the syntax:

$ bao audit list -detailed
Path     Type    Description    Replication    Options
----     ----    -----------    -----------    -------
file/    file    n/a            replicated     file_path=/openbao/audit/audit.log

Declarative devices are created and removed on the active node during restarts and reload signals, no two blocks may share the same path, and a declarative device cannot be modified in place or duplicate an existing API-created device. Keep the stanza identical across every server in the cluster, because it is now part of the configuration that must not drift.

Reading a real record

The record below was written when a token carrying a read grant on one path attempted to read a different path. Several fields are omitted here for length; nothing shown has been altered.

{"time":"2026-08-26T21:26:20.555636512Z","type":"response",
 "auth":{"client_token":"hmac-sha256:da33377eba1c...",
         "accessor":"hmac-sha256:5eb6ce9e...",
         "display_name":"token",
         "policies":["app-read","default"],
         "policy_results":{"allowed":false},
         "token_type":"service","token_ttl":1800},
 "request":{"operation":"read","mount_point":"kv/","mount_type":"kv",
            "path":"kv/data/app/other","remote_address":"127.0.0.1"},
 "response":{"data":{"error":"hmac-sha256:b0a0f532..."}},
 "error":"1 error occurred:\n\t* permission denied\n\n"}

Four things in that record deserve attention.

• The refusal is recorded as fully as a success. The policy_results object carries "allowed":false, and the error field carries the message the client received. A denied read is the single most useful event in the whole stream, because it is what both a misconfigured application and an attacker probing paths produce.
• The path is exact and unredacted. It reads kv/data/app/other, which is what makes the previous lesson’s data and metadata split diagnosable from the log alone.
• The token appears as a keyed hash, and so does the accessor. Anyone reading the log learns which session acted without being able to replay it.
• The secret value is absent. On the run that produced this file, searching the whole audit log for the stored value returned zero matches. The value never reached the log in clear text.

flowchart LR
    Q["Request\ntoken plus path"] --> E["Policy evaluation"]
    E --> A["Audit device"]
    Q --> H["HMAC-SHA256\nkeyed hash"]
    H --> A
    A --> F["Record: identity hash,\npath, outcome, error"]
    E -->|"allowed or denied"| F

The diagram separates the two paths a request takes. The request itself goes to policy evaluation and then, if permitted, to the engine. In parallel, the sensitive parts of the request and response are passed through a keyed hash before anything is written, so the record that lands on disk contains the shape of the request and a non-reversible stand-in for its secrets. The outcome of the policy decision is attached to that record whether the answer was yes or no.

Fail-closed is a correctness guarantee and an availability risk

OpenBao will not respond to requests when no enabled audit device can record them. If a device is blocked, requests hang until the blocked device can write again. That is the right default for a system whose entire value proposition is attributable access, and it converts a full disk on the audit volume into a total outage of a tier-zero service.

Both facts have to be operated at once. The audit destination needs the same capacity planning, monitoring and alerting as the data path, because it is the data path now.

Production discipline

1. Put the audit stanza in the configuration on day one. Enable it before the first secret exists, so no credential was ever written or read without a record.
2. Leave unsafe_allow_api_audit_creation at its default. If a genuine need arises, turn it on temporarily, do the work, and reload the configuration to turn it off again.
3. Ship the records off the node, and protect them differently from application logs. The audit stream is attributable access history for every credential in the estate. Its retention and access rules belong to security, not to whoever owns the log platform.
4. Alert on denials, not only on volume. A rising rate of records carrying an allowed value of false is either a deployment that changed a path or somebody enumerating your paths. Both deserve a look, and both are invisible if you only chart request counts.
5. Monitor the audit destination as a tier-zero dependency. Free space, write latency and the timestamp of the most recent record. A stalled device does not raise an error; it stops the service.

Cross-course references

• Linux for Production Sysadmins - Part XXXI (Audit and Security Logging) covers the host-level auditing whose records sit alongside these and answer a different question: who was on the machine when the credential was read.
• Kubernetes for Production Sysadmins - Part XCIV (Audit Logging) covers the API server audit policy, the same record-everything-but-the-payload problem at cluster scope.
• Observability for Production Sysadmins - Part XL (Log Retention) covers the retention and capacity planning that turn an audit stream into evidence you still have when you need it.

Quiz