Flat namespaces and the prefix convention

Why this matters in production

The key scheme is decided once and constrains everything afterwards — listing performance, index distribution, lifecycle rule granularity. It is worth ten minutes of thought and is usually given none.

Flat, with a convention

data/2026/08/18/sensor-a/reading-0001.json

That is one key, a single string. The slashes mean nothing to the store. What creates the appearance of hierarchy is the listing API:

aws s3api list-objects-v2 --bucket data \
    --prefix "data/2026/08/" --delimiter "/"

{
  "CommonPrefixes": [
    {"Prefix": "data/2026/08/17/"},
    {"Prefix": "data/2026/08/18/"}
  ],
  "Contents": []
}

--delimiter / makes the listing stop at the next slash and report the distinct prefixes it found. That is the entire mechanism behind every object-storage browser that shows folders.

What hierarchy does not provide

“Delete the 2025 directory” is a listing followed by up to a thousand deletes per batch request, repeated. On a prefix with millions of keys that is a job, not a command — which is what lifecycle rules exist for.

Designing key schemes

For listing: put the attributes you filter by earliest in the key.

good:  sensor-a/2026/08/18/reading-0001.json     ← list per sensor, per day
poor:  reading-0001-sensor-a-2026-08-18.json     ← cannot list per sensor

For distribution: keys hash to index shards, so a scheme where many keys share a long common prefix still distributes fine — the hash covers the whole key. This differs from AWS’s historical guidance about random prefixes, which addressed a partitioning behaviour RGW does not share.

For lifecycle: rules are prefix-scoped, so the key scheme must separate data with different retention.

{"Rules": [
  {"ID": "expire-temp", "Prefix": "temp/", "Status": "Enabled",
   "Expiration": {"Days": 7}},
  {"ID": "archive-old", "Prefix": "data/", "Status": "Enabled",
   "Expiration": {"Days": 2555}}
]}

Quiz

Production discipline

Design the key scheme against your listing patterns and your retention boundaries together, since both are prefix-based and both are fixed by what the key puts first. Validate that lifecycle rules can express the retention policy under the proposed scheme before ingest starts — rekeying afterwards means copying everything.

Cross-course references

• Kubernetes: label and annotation schemes constrain selection the same way and are equally hard to change
• Linux: file naming conventions that support globbing follow the same design pressure