Client version compatibility

Why this matters in production

Clients are frequently on hardware or distributions with their own upgrade constraints, and the cluster upgrade can strand them.

Checking connected clients

ceph features

{
    "mon": [ { "features": "0x...", "release": "squid", "num": 3 } ],
    "osd": [ { "features": "0x...", "release": "squid", "num": 96 } ],
    "client": [
        { "features": "0x...", "release": "reef", "num": 42 },
        { "features": "0x...", "release": "pacific", "num": 3 }
    ]
}

The three Pacific clients are the ones to identify before upgrading.

# and the minimum the cluster requires
ceph osd get-require-min-compat-client
ceph osd dump | grep require_min_compat_client

What clients need

RELEASE=release
ceph osd set-require-min-compat-client ${RELEASE}

Raising the requirement disconnects clients below it, so it is
checked against ceph features first.

# what would be affected
ceph features | python3 -c '
import sys,json
d = json.load(sys.stdin)
for c in d.get("client", []):
    print(c.get("release"), c.get("num"))'

Identifying which clients

ceph daemon mon.$(hostname -s) sessions 2>/dev/null | head

# for RBD, watchers show the client addresses
POOL=rbd-vms
IMAGE=vm-disk-01
rbd status ${POOL}/${IMAGE}

# for CephFS
ceph tell mds.0 client ls | python3 -c '
import sys,json
for c in json.load(sys.stdin):
    print(c.get("id"), c.get("client_metadata", {}).get("ceph_version"),
          c.get("inst"))'

Identifying the specific hosts is what turns “three old clients” into an actionable list.

Handling clients that cannot be upgraded

The general escape: userspace clients ship with Ceph packages and are
decoupled from the kernel, so a host with an old kernel can still run a
current client.

Sequencing

1. inventory the connected clients and their versions
2. identify any below the target's requirement
3. upgrade or replace those clients first
4. verify ceph features shows no client below the requirement
5. upgrade the cluster
6. raise require-min-compat-client only after all clients are current

ceph features | python3 -c '
import sys,json
d=json.load(sys.stdin)
old=[c for c in d.get("client",[]) if c.get("release") in ("pacific","octopus","nautilus")]
print("clients needing attention:", sum(c["num"] for c in old))'

Quiz

Production discipline

Inventory connected client versions with ceph features and identify the specific hosts before upgrading. Where a client fleet cannot be upgraded, keep require-min-compat-client at their level and use userspace clients, which ship with Ceph and are decoupled from the kernel.

Cross-course references

• Kubernetes: client version skew constraints drive the same staged migration
• Linux: userspace implementations decouple capability from kernel version generally