Bucket configuration that a data copy does not carry

Why this matters in production

A restored bucket with the right objects and the wrong policy is either broken or exposed, and neither is obvious from an object count.

What lives outside the objects

`rclone copy` and `aws s3 sync` move objects. None of the above travels
with them.

Exporting the configuration

B=acme-data
OUT=/secure/rgw-config/$B
mkdir -p "$OUT"

aws --endpoint-url $EP s3api get-bucket-policy       --bucket $B > "$OUT/policy.json" 2>/dev/null
aws --endpoint-url $EP s3api get-bucket-acl          --bucket $B > "$OUT/acl.json"
aws --endpoint-url $EP s3api get-bucket-lifecycle-configuration --bucket $B \
  > "$OUT/lifecycle.json" 2>/dev/null
aws --endpoint-url $EP s3api get-bucket-cors         --bucket $B > "$OUT/cors.json" 2>/dev/null
aws --endpoint-url $EP s3api get-bucket-versioning   --bucket $B > "$OUT/versioning.json"
aws --endpoint-url $EP s3api get-object-lock-configuration --bucket $B \
  > "$OUT/objectlock.json" 2>/dev/null

# RGW-side metadata the S3 API does not expose
radosgw-admin bucket stats --bucket=$B      > "$OUT/bucket-stats.json"
radosgw-admin metadata get bucket:$B        > "$OUT/bucket-meta.json"
radosgw-admin user info --uid=$(python3 -c '
import json,sys; print(json.load(open(sys.argv[1]))["owner"])' "$OUT/bucket-stats.json") \
  > "$OUT/owner.json"
radosgw-admin quota show --quota-scope=bucket --bucket=$B > "$OUT/quota.json"

The owner export contains access keys. Store it with the same controls
as any credential.

Restoring it

Order matters. Some settings can only be applied at creation.

# object lock requires versioning and must be set at bucket creation
aws --endpoint-url $EP s3api create-bucket --bucket acme-data-restored \
  --object-lock-enabled-for-bucket

aws --endpoint-url $EP s3api put-bucket-versioning --bucket acme-data-restored \
  --versioning-configuration Status=Enabled
aws --endpoint-url $EP s3api put-bucket-policy --bucket acme-data-restored \
  --policy file://$OUT/policy.json
aws --endpoint-url $EP s3api put-bucket-lifecycle-configuration \
  --bucket acme-data-restored \
  --lifecycle-configuration file://$OUT/lifecycle.json

# then the objects
rclone copy external:acme-backup ceph:acme-data-restored --transfers 32

Verifying behaviour

# the policy is in effect
aws --endpoint-url $EP s3api get-bucket-policy --bucket acme-data-restored \
  | python3 -m json.tool | head

# an unauthorised principal is still denied
aws --endpoint-url $EP --profile other-tenant \
  s3 ls s3://acme-data-restored 2>&1 | grep -qi 'denied' \
  && echo "policy enforcing" || echo "POLICY NOT ENFORCING"

diff <(aws --endpoint-url $EP s3api get-bucket-lifecycle-configuration \
        --bucket acme-data-restored | python3 -m json.tool) \
     <(python3 -m json.tool < "$OUT/lifecycle.json")

Quiz

Production discipline

Export bucket policy, ACLs, lifecycle, CORS, versioning, and object lock configuration alongside the objects — none of it travels with a data copy. Create the bucket with object lock enabled if the original had it; it cannot be added afterwards.

Cross-course references

• Kubernetes: restoring PVC data without the StorageClass and access mode is incomplete
• Linux: a lost restrictive policy fails open, which is why negative tests matter