Choosing and configuring a time source

Intermediate⏱ ~16 minchronytimedatectl

What you'll learn

Compare the available time synchronisation implementations
Design a time source hierarchy for a cluster
Configure sources for accuracy and resilience
Verify the hierarchy is working as intended

Prerequisites

None — start here.

Verified against Ceph Tentacle 20.2.x · Ceph Squid 19.2.x (supported previous) · cephadm matches the verified Ceph release · podman 4.x · csi-rbd and csi-cephfs current · RBD / CephFS / RGW current (matches Ceph release) · Linux kernel 5.15+ (5.10 minimum) · Ubuntu 24.04 LTS (Ceph host baseline) · Debian 12 (Bookworm) (Ceph host baseline) · Rocky Linux / RHEL / AlmaLinux 9.x (Ceph host baseline) · Proxmox VE 9.x (cross-course integration) · Kubernetes 1.31+ (cross-course integration) · 2026-08-18

Why this matters in production

Ceph’s tolerance is tight enough that the implementation and the source structure matter. A default installation pointing at public pool servers across a slow link is not reliably within 50 milliseconds.

The implementations

	chrony	ntpd	systemd-timesyncd
Accuracy	best	good	adequate
Convergence after boot	fast	slow	moderate
Handles intermittent connectivity	well	poorly	moderately
Can serve time to others	yes	yes	no
Suits virtual machines	well	poorly	adequately
Configuration complexity	moderate	higher	minimal

chrony for Ceph hosts. Its fast convergence and handling of the irregular conditions found on virtual and rebooting hosts are exactly what the tight tolerance needs.

systemd-timesyncd is a simple SNTP client. It is adequate for edge and lightly-loaded virtual machines and cannot serve time to other hosts, so it does not suit a local time server role.

Source hierarchy

external stratum 1/2 sources
        ↓
local time servers (2 or 3, running chrony, peered)
        ↓
every Ceph host

# on the local time servers
server ntp1.public.example iburst
server ntp2.public.example iburst
server ntp3.public.example iburst
peer timeserver-02 iburst
allow 10.20.0.0/16
local stratum 10

# on every Ceph host
server timeserver-01 iburst
server timeserver-02 iburst
server timeserver-03 iburst
makestep 1.0 3
rtcsync

Local servers give three benefits: low and consistent latency, resilience to internet connectivity loss through local stratum, and the whole cluster converging on the same time rather than independently on nearby ones.

Why three sources

With two sources that disagree, chrony cannot tell which is wrong. With three, a single bad source is outvoted. This is the same reasoning as quorum and it is why two is a worse configuration than one in some respects.

Verifying

chronyc sources -v
chronyc sourcestats
chronyc tracking

MS Name/IP address    Stratum Poll Reach LastRx Last sample
^* timeserver-01            3    6   377     41  +12us[  +18us] +/-  1.2ms
^+ timeserver-02            3    6   377     38  -31us[  -31us] +/-  1.4ms
^+ timeserver-03            3    6   377     44  +8us [   +8us] +/-  1.3ms

^* is the selected source, ^+ are acceptable alternatives, Reach 377 means all recent polls succeeded. Anything else warrants investigation.

Quiz

Knowledge check · 4 questions

Q1. Why is chrony preferred over systemd-timesyncd for Ceph hosts?

Faster convergence after boot and better handling of the irregular conditions Ceph's tight tolerance is sensitive to; it can also serve time It is packaged on more distributions than systemd-timesyncd, so a mixed fleet can standardise on one time implementation without any exceptions systemd-timesyncd implements only single-server SNTP polling and therefore cannot discipline a clock continuously between polls chrony uses less memory and fewer threads, which matters on dense OSD hosts where every megabyte is committed to BlueStore caches
Q2. Configuring exactly two time sources can leave a host worse placed than configuring one.

True False
Q3. Design a time source structure for a production cluster.

A 40-host cluster currently has every host configured with three public pool servers directly. Occasional MON_CLOCK_SKEW warnings appear, and an internet outage last month caused widespread drift.
Q4. What does the `local stratum` directive provide on a local time server?

Passing score: 75%. Answers are checked in this browser.

Production discipline

Deploy local time servers rather than pointing every Ceph host at public pools; the consistency and the outage resilience both matter at Ceph’s tolerance. Configure at least three independent sources so a single bad one is outvoted — two is a worse configuration than three by a meaningful margin.

Cross-course references

Kubernetes: cluster-wide time consistency matters equally for certificates and leases
Linux: NTP hierarchy design is standard infrastructure practice