librados — the API every Ceph service is built on

Why this matters in production

RBD, CephFS, and RGW are all librados applications. Understanding what librados provides explains what those services can and cannot do, and occasionally an application genuinely belongs at this level.

What librados offers

Beyond read and write:

• Atomic compound operations. Several operations on one object applied as a unit — read xattr, compare, write data, set xattr — all or nothing.
• Conditional operations. Write only if the object does not exist, or only if an xattr has a given value. This is how distributed locks are built.
• omap operations. Read, write, and range-scan the per-object key-value map.
• watch/notify. Register interest in an object and receive notifications when others notify on it. This is how RBD exclusive locks coordinate between clients.
• Object classes. Server-side code executed on the OSD holding the object, so computation happens where the data is.

A minimal Python example:

import rados

cluster = rados.Rados(conffile='/etc/ceph/ceph.conf')
cluster.connect()
ioctx = cluster.open_ioctx('mypool')

ioctx.write_full('config', b'{"version": 1}')
ioctx.set_xattr('config', 'owner', b'platform-team')
print(ioctx.read('config'))

ioctx.close()
cluster.shutdown()

When direct use is justified

• Applications whose natural unit is already an object and which need neither POSIX nor S3 semantics.
• Workloads needing atomic compound operations that no service layer exposes — distributed coordination, leader election, counters.
• Workloads that benefit from object classes, where moving computation to the data beats moving the data.
• Internal Ceph tooling and integrations.

Object classes

Object classes are the genuinely distinctive capability. A class is compiled server-side code, loaded by the OSD, callable by clients:

ioctx.execute('myobject', 'myclass', 'mymethod', input_data)

RGW uses this heavily — bucket index operations run as object class methods on the OSD holding the index, so listing a bucket does not require transferring the whole index to the gateway.

The practical takeaway

Most operators never write librados code, and that is correct. What matters operationally is recognising that the services are librados applications, so their behaviour inherits librados semantics — atomic per-object operations, no cross-object transactions, and coordination built on watch/notify rather than on a lock service.

Quiz

Production discipline

Use a service layer unless you can state a specific reason not to — RBD, CephFS, and RGW each solve real problems on top of librados that a direct integration would have to solve again. When direct librados use is genuinely right, plan enumeration explicitly: there is no built-in listing that scales, so the application must own its index. And carry the semantics forward into how you reason about the services: per-object atomicity, no cross-object transactions, and coordination through watch/notify rather than a lock service.

Cross-course references

• Ceph: Part XXXV (RBD Architecture) for what RBD builds on this.
• Ceph: Part XLV (RADOS Gateway) for RGW use of object classes.
• Ceph: Part XXXII (Least Privilege Capabilities) for cephx caps on a pool.