Time concepts - UTC, monotonic, and the wall clock

Linux has multiple clocks. They look the same to a casual user but behave very differently. Misunderstanding them is the cause of subtle bugs in production: certificate validation, log correlation, distributed databases, and Kerberos all break when clocks skew. This lesson is the foundation.

Wall-clock time vs monotonic time

Linux exposes two fundamentally different kinds of time:

• Wall-clock time (CLOCK_REALTIME): the date and time as humans think about it. Subject to NTP adjustments, daylight saving, manual changes, and jumps backwards or forwards.
• Monotonic time (CLOCK_MONOTONIC): a counter that always increases. Never goes backwards. Not adjusted by NTP. Cannot be set by user. Perfect for measuring durations.

# Read the wall clock
date
timedatectl

# Read the monotonic clock (seconds since boot, with
# sub-second precision)
cat /proc/uptime

In Python or C, you would use clock_gettime(CLOCK_REALTIME) for the wall clock and clock_gettime(CLOCK_MONOTONIC) for elapsed time. In shell, you would use date +%s.%N for wall or /proc/uptime for monotonic.

The hardware clock

The hardware clock (RTC) is a battery-backed clock on the motherboard. It runs even when the host is off. On boot, Linux reads the RTC and uses it to set the system clock.

hwclock              # read the hardware clock
hwclock --systohc    # write system time to hardware clock
hwclock --hctosys    # write hardware clock to system time

In modern Linux, the hardware clock is usually UTC. Setting it to local time is a Windows convention; do not do this on Linux.

Why monotonic time matters

Consider measuring “how long did this query take”:

START=$(date +%s.%N)
do_something
END=$(date +%s.%N)
echo $END - $START

If NTP adjusts the system clock between START and END, END - START may be wrong (negative or wildly large). Use /proc/uptime or clock_gettime(CLOCK_MONOTONIC) for elapsed time:

# Read monotonic time
cat /proc/uptime
# Outputs: <uptime_seconds> <idle_seconds>

Or in a script:

START=$(awk '{print $1}' /proc/uptime)
do_something
END=$(awk '{print $1}' /proc/uptime)
echo "$END - $START" | bc

This is immune to wall-clock adjustments.

Time zones

Linux stores the wall clock as UTC internally. Display in a local time zone is a presentation choice.

date                      # local time
date -u                   # UTC
TZ=America/New_York date  # explicit time zone for this command
timedatectl list-timezones

/etc/localtime is a symlink (or copy) of the time zone data. /etc/timezone (Debian) contains the time zone name.

Set the time zone:

sudo timedatectl set-timezone America/New_York
sudo ln -sf /usr/share/zoneinfo/Europe/London /etc/localtime

What breaks when clocks skew

The most common threshold is 5 minutes: TLS certificates typically reject anything more than 5 minutes off.

Time skew in production

Every host should be within milliseconds of the true time. With NTP:

• Local network: <1 ms skew.
• Internet to public NTP: <50 ms skew.
• Cross-datacenter: <10 ms skew.

100 ms skew causes visible problems with logging. >5 seconds skew causes authentication failures. >5 minutes skew causes TLS failures.

The two clocks: how they interact

Boot:
  1. Read hardware clock (RTC)
  2. Set system clock (CLOCK_REALTIME) from RTC
  3. NTP starts and adjusts the system clock

Runtime:
  - NTP adjusts CLOCK_REALTIME in small steps ("slewing")
    or occasional jumps (large error)
  - NTP never adjusts CLOCK_MONOTONIC
  - CLOCK_MONOTONIC continues from boot time + adjustments

Shutdown:
  - Write system clock to hardware clock
  - Hardware clock keeps time while host is off

Modern systemd timesync and chrony both use slewing for small errors and stepping for large errors. The slewing is small (<1 second per step) so it does not affect most applications.

Knowledge check