Envelope encryption in practice: key-wrapping keys and data keys

Envelope encryption is the industry name for a two-tier scheme in which one key protects another key, and only the inner key ever touches your data. NIST does not use the word envelope at all. Its vocabulary is key wrapping, and it is worth adopting, because the NIST terms make a distinction that the marketing term hides and that matters when something goes wrong.

Three terms that are not interchangeable

Earlier in this course you met symmetric encryption and authenticated encryption as primitives. Here they acquire roles.

• Key-encrypting key. A key used to encrypt or decrypt other keys, providing confidentiality for those keys. That is all it promises.
• Key-wrapping key. A symmetric key that provides both confidentiality and integrity protection for other keys. The approved constructions are the deterministic authenticated encryption modes in SP 800-38F: AES-KW, AES-KWP and TKW.
• Data key. The key that actually encrypts your bytes. It is generated fresh, used on your own hardware, and destroyed the moment it is no longer needed.

The distinction is operationally live. A wrapped key carries an integrity tag, so an unwrap of a corrupted or substituted blob fails loudly. A merely encrypted key does not, so the same corruption yields a key-shaped pile of bytes that decrypts your data into garbage, and the failure surfaces hundreds of layers away from its cause. When a design says “we encrypt the DEK with the KEK”, ask which of the two it actually built.

Why two tiers exist at all

The reason is not that the wrapping key is too precious to use. It is that symmetric keys wear out from volume, and the two tiers let you retire the key that does the volume without touching the key that does the trust.

A 256-bit data key that encrypts millions of messages becomes exhausted and starts producing ciphertext with subtle patterns. The remedy is to use a data key once, or a small number of times, and then generate another. A wrapping key, by contrast, performs one short operation per data key rather than one per message, so it is used orders of magnitude less often and is almost never reused enough to approach that limit.

Two consequences follow. First, a data key is cheap and disposable by design, so a scheme that reuses one across an entire tenant has thrown away the property it was built for. Second, the wrapping key is long-lived on purpose, which is why lesson four has to be so careful about what rotating it does.

The encrypt sequence, in order

sequenceDiagram
    participant App as Application
    participant KMS as Key service
    participant Obj as Object store
    App->>KMS: generate data key under wrapping key id
    KMS-->>App: plaintext data key plus wrapped data key
    App->>App: encrypt payload with the plaintext data key
    App->>App: destroy the plaintext data key
    App->>Obj: store ciphertext, wrapped key, nonce, key id

The order in that diagram is the point. The application asks the key service for a fresh data key and receives two things: the key in clear, for immediate use, and an encrypted copy of the same key, safe to store next to the data. It encrypts the payload locally with an authenticated mode, then removes the plaintext key from memory as soon as it possibly can. Only then does it write. Everything the object needs in order to be readable later travels with the object; nothing needs to be remembered elsewhere.

There is a useful variant for processes that must be able to write but must never be able to read. Ask for a wrapped data key without the plaintext copy, and the caller receives only the encrypted key. A batch writer, an archiver or an edge collector can then be provisioned so that it can prepare protected material for later processing while holding nothing that could decrypt it.

What must be stored with the ciphertext

object header (not secret, stored in clear)
  wrapping key identifier   which key can unwrap the next field
  wrapped data key          the encrypted data key, integrity protected
  algorithm identifier      which AEAD mode and key size were used
  nonce or IV               unique per encryption under this data key
  authentication tag        emitted by the AEAD mode over ciphertext
  binding context           optional key/value pairs also authenticated

object body
  ciphertext                the payload, encrypted under the data key

None of these fields is a secret, and none of them may be omitted. The wrapping key identifier tells a future reader which key to call. The wrapped data key is the only surviving copy of the data key, and the key service does not hold a spare. The nonce must be unique for every encryption performed under a given data key, which is the strongest practical argument for short-lived data keys: uniqueness is easy to guarantee over hundreds of operations and hard to guarantee over billions.

The binding context deserves a note. Most services let you attach additional authenticated data to a wrap, so that the unwrap only succeeds when the caller supplies the same values. Bind the things that identify the object, such as a tenant identifier and an object path. A wrapped data key lifted from one record then cannot be replayed against another, because the unwrap fails before any key is returned.

The decrypt sequence, in order

1. read the object header from storage
2. call the key service unwrap operation with:
     the wrapped data key
     the same binding context used at encrypt time
3. receive the plaintext data key
4. decrypt the body with the data key, nonce and tag
5. destroy the plaintext data key
6. return the plaintext payload to the caller

Step five is skipped more often than any other, and it is the one that decides whether a memory-disclosure bug in step six is an availability incident or a confidentiality incident. Step two is the one that fails in production: a context that was built from a mutable field, such as a display name or a path that a migration later rewrote, will not reproduce, and the unwrap will refuse. Bind on immutable identifiers only.

Caching data keys without pretending it is free

A per-object data key means one key service call per object write, and one per object read unless you cache. At scale that is a real bill and a real latency budget, and caching is a legitimate answer. It is also, precisely, the decision to keep key material in your process memory for longer, which is the thing lesson one said delegation had removed.

Bound a cache on three axes at once, because any single bound can be defeated by a workload that moves along a different one.

• Time. A maximum age, measured from generation rather than from last use, so a busy key cannot live forever.
• Volume. A maximum number of bytes and a maximum number of messages encrypted under one cached key, chosen well below the exhaustion threshold discussed above.
• Scope. Never share a cached key across tenants, and never across binding contexts, or the cache has quietly undone the isolation that the context was added to provide.

Cache only on the encrypt path if you can. Read paths usually tolerate the extra call better than write paths, and a cache that holds decryption keys for popular objects is a cache that a memory disclosure turns directly into a data breach.

Production discipline

1. Write the header format down and version it. Add a version byte before you need one. The first time you change an algorithm, that byte is the difference between a migration and an archaeology project.
2. Test the read path against the oldest object you have. Encrypt-side changes are exercised constantly; decrypt-side compatibility with a five-year-old header is exercised only by the customer who asks for their oldest record.
3. Destroy plaintext data keys explicitly. Overwrite the buffer and drop the reference in the same function that created it, rather than trusting a garbage collector you do not control.
4. Back up the wrapped data key with the same guarantees as the ciphertext. They are one artefact. A replication rule that copies the body and not the header creates unreadable data that passes every integrity check on the storage tier.
5. Alert on unwrap failure rate, not just error rate. A slow climb in unwrap failures is the shape of a migration quietly rewriting a field that somebody bound a context to.

Cross-course references

• Kubernetes for Production Sysadmins - Part LXV (SecretsSec) covers the encryption-at-rest provider that wraps cluster Secrets with an external key, which is this pattern applied to the API server’s write path.
• Terraform for Production Sysadmins - Part XIX (Security) covers how ciphertext and key references end up in state files, and why the wrapped key beside a resource is not itself a secret.
• Linux for Production Sysadmins - Part XLVII (Backup) covers backup sets that must carry both the body and the header for a restore to be readable at all.

Quiz