Skip to main content
RunBook Academy

← All break/fix scenarios in Ceph

intermediateceph-rgw~20 min

RGW bucket access denied for an S3 client

Reported symptoms

  • ●S3 client receives AccessDenied on a valid request
  • ●Other RGW users can access the same bucket
  • ●radosgw-admin log show lists the access denied errors

Evidence

  • · radosgw-admin user info shows the user has caps but no bucket policy
  • · radosgw-admin bucket list --uid=<user> confirms the bucket exists
  • · radosgw-admin zone get shows the data pool is healthy
Diagnosis and resolutionclick to reveal

Root cause

The bucket policy denies the request. Either IAM policy, bucket policy, or per-user ACL is denying access.

Remediation

Inspect each layer: IAM policy, bucket policy, user caps. Identify the denial; correct the policy.

Verification

The S3 client can s3 ls and s3 cp successfully; the application proceeds.

Prevention

Audit policies quarterly; document who has access; use least privilege.

Architecture

The student examines the broken architecture diagram and identifies the responsible component. The cluster is running Ceph Tentacle (20.2.x); the incident is one a production operator must diagnose from the evidence presented.

Symptoms

  • S3 client receives AccessDenied on a valid request
  • Other RGW users can access the same bucket
  • radosgw-admin log show lists the access denied errors

Evidence

  • radosgw-admin user info shows the user has caps but no bucket policy
  • radosgw-admin bucket list —uid=<user> confirms the bucket exists
  • radosgw-admin zone get shows the data pool is healthy

Student investigation

The student follows the methodology: define the symptom, determine the impact, gather evidence, identify the component, form a hypothesis, test safely, restore, validate.

Progressive hints

  • Hint 1: check the cluster state with ceph -s first.
  • Hint 2: read ceph health detail and identify the affected PGs / OSDs / daemons.
  • Hint 3: use ceph osd tree, ceph pg dump, or ceph mds stat as the next-level diagnostic.
  • Hint 4: the root cause is documented in the frontmatter root_cause field.

Validation

The student runs the verification steps and confirms the symptom cleared.

Root cause

See the frontmatter root_cause field.

Remediation

Apply the fix from the frontmatter remediation field.

Prevention

Apply the prevention measures from the frontmatter prevention field.