The secret lifecycle - seven stages and seven owners

A credential is never only a value. It is a value plus a set of facts: where it came from, who holds copies, what breaks if it changes, when it stops working, and how to make it stop working early. Those facts are produced by seven stages. An estate that runs credentials without producing them still works perfectly well, right up to the first incident that asks a question nobody recorded the answer to.

The seven stages and what each one produces

Each stage answers exactly one question, and the answer is an artefact rather than an intention. If the artefact does not exist somewhere an on-call engineer can read at three in the morning, the stage has not happened.

flowchart LR
    G["Generate"] --> D["Distribute"]
    D --> U["Use"]
    U --> S["Store"]
    S --> R["Rotate"]
    R --> D
    U --> V["Revoke"]
    V --> X["Destroy"]
    S --> X

The diagram shows the ordinary path along the top and the two shortcuts that matter under pressure. Rotation loops back into distribution, because a rotated value has to reach every consumer again. Revocation cuts across from use straight to destruction, because an incident does not wait for the next scheduled rotation. An estate that has never exercised either arrow has a lifecycle diagram, not a lifecycle.

An owner is a party that can act without asking

Ownership is frequently confused with three other things, and the confusion is what leaves stages unowned in practice.

• The owner is not the custodian. A secret manager, a CI variable store or a Kubernetes Secret object holds the value. Holding is a storage service, not accountability. A custodian answers the question “can I read it”; an owner answers the question “may this change today”.
• The owner is not the consumer. The service that presents the credential to a database has every incentive to leave it alone. Consumers make excellent objectors and poor owners.
• The owner is not a person. Individuals leave, take leave and sleep. An owner is a rota or a team with a paging address, a budget line and the authority to break something on purpose during a maintenance window.

The working definition is narrow on purpose. A stage is owned when one named rota can change it without seeking permission, is paged when it fails, and is the party that pays for the failure. Anything softer collapses at the moment it is tested.

The failure signature of each unowned stage

Unowned stages do not fail silently. Each one produces a characteristic symptom that an experienced operator can read backwards to the missing owner.

• Generate unowned. The strength policy exists in a wiki and is advisory. Somebody types a memorable passphrase into a field labelled “API key” and it works, because nothing measures it.
• Distribute unowned. Copies propagate by whatever route was convenient. Nobody can enumerate the copies, so nobody can bound the exposure when one route turns out to be logged.
• Use unowned. No inventory maps the value to its consumers. Rotation is now blocked by fear rather than by risk, because the honest answer to “what breaks if we change this” is unknown.
• Store unowned. The value rests in three places with three different retention rules, and the shortest-lived copy is the one everybody remembers.
• Rotate unowned. Age becomes unbounded. Blast radius grows monotonically with age, so the credential quietly becomes the most valuable thing in the estate.
• Revoke unowned. The incident has no stop button. The response degrades into asking a vendor to do something, on a weekend, without a support contract that covers it.
• Destroy unowned. The retired value stays valid inside a backup, a snapshot or an archived pipeline log, and remains usable long after the change ticket was closed as done.

Read that list as a diagnostic rather than as a warning. Each symptom points at exactly one missing owner, which means an engineer who hears “we cannot rotate that one” during a planning session has learned something specific: the use stage has no owner and no consumer inventory exists. Likewise, a team that can rotate quickly but cannot say who else holds a value has a distribution gap rather than a rotation gap, and improving the rotation tooling will not help it. Matching the symptom to the stage is what turns a vague sense that secrets management is weak into a piece of work somebody can be assigned.

The classic composite is the database password issued years ago for a service that has since been decommissioned. It is still present in a CI variable store, still accepted by the database, owned by nobody, and consumed by nothing anybody can name. Every stage failed, and no monitor noticed, because nothing was broken.

The corrective artefact is unglamorous and short. A record that travels with the value, rather than a page in a wiki that drifts away from it, gives every stage somewhere to write its answer.

id: reporting-db-readonly
owner_rota: platform-data-oncall
paging_address: platform-data@example.com
generated: "2026-08-26 by openbao database engine"
consumers:
  - reporting-api
  - nightly-extract-job
transports:
  - openbao agent, memory-backed mount on web-01 and web-02
rotation_procedure: secrets-pki-rb-11
rotation_last_run: "2026-08-19"
expiry_enforced_by: postgresql role validity

What a fully owned lifecycle looks like when it works

A dynamic database credential is the clearest available example of all seven stages implemented by machinery rather than by intention. The credential is generated on request, distributed only to the caller that authenticated, used for a bounded window, stored nowhere permanent, rotated by being replaced rather than edited, revoked by a single API call, and destroyed because the underlying database role ceases to exist.

$ bao lease lookup database/creds/app-readonly/xoHI541EXoFgKn1OTiusOdd9
expire_time     2026-08-26T21:25:43.891514518Z
issue_time      2026-08-26T21:23:43.891514368Z
renewable       true
ttl             1m59s

Read that output as a lifecycle record rather than as a status line. It states when the value came into existence, when it stops working without any further human action, and whether the holder may extend it. Revoking the same lease removes the database role outright, so the next connection attempt fails at the database rather than at a policy layer.

LEASE=database/creds/app-readonly/xoHI541EXoFgKn1OTiusOdd9
bao lease lookup "$LEASE"
bao lease revoke "$LEASE"

Those three lines cover the revoke and destroy stages for that credential, and the effect is observable from outside the issuing system. That external observability is what makes the stage genuinely owned rather than merely documented, and it is the property a static value in a vault cannot offer.

Most estates cannot move every credential onto that model, and the useful conclusion is not that they should try. It is that the dynamic case shows what the seven artefacts look like when they are complete, and every static credential can be measured against it. Ask which of the seven a given value has, and the gap becomes a work list rather than a feeling. A static database password with a named rota, a consumer list, a rehearsed rotation procedure and a documented revocation path is operationally close to the dynamic case, even though nothing about it is automated. The automation is a delivery mechanism for the artefacts; the artefacts are the point.

Production discipline

1. Write the owner down beside the value, not in a wiki. The secret store entry carries an owner field naming a rota and a paging address. A wiki page is a copy that drifts.
2. Make every stage produce a dated artefact. An undated rotation procedure is a plan; the same procedure with a last run date is a control.
3. Exercise revocation on a healthy credential quarterly. The revocation path is the only stage whose first use is otherwise guaranteed to be during an incident.
4. Treat an unknown consumer list as a blocking finding. If nobody can say what breaks when a value changes, the value is already unrotatable and the risk is already accepted by default.
5. Close the loop on destruction explicitly. Record which copies were removed and which residual copies, such as backups, will expire on their own and when.

Cross-course references

• Linux for Production Sysadmins - Part LXXII (Secrets) covers the on-host end of these stages, where file ownership and modes decide which local accounts become unrecorded holders of a value.
• Git, CI/CD & GitOps for Infrastructure Engineers - Part XCIII (CredRotation) covers rotation cadence by credential class, which is the rotate stage expressed as a schedule rather than as an intention.
• Kubernetes for Production Sysadmins - Part LXV (SecretsSec) covers what changes when the cluster itself is the custodian and namespace membership becomes an implicit holder list.

Quiz