Keepalived health checks - service-aware failover

A health check turns “is the host up?” into “is the service working?”. This lesson covers keepalived’s health check mechanisms.

Why health checks

Without health checks, keepalived fails over when the host is unreachable. The host is up but the service is down: the VIP is held by a broken host.

With health checks, keepalived fails over when the service is broken, even if the host is up. The VIP is held by a host that can serve the workload.

HTTP check

vrrp_script check_http {
    script "/usr/bin/curl -sf http://localhost/health || exit 1"
    interval 2
    weight -20
    fall 3
    rise 2
}

The script returns 0 if the HTTP check succeeds, non-zero if it fails. If the script fails 3 times in a row (fall), the priority is reduced by 20.

TCP check

For a TCP service (e.g. PostgreSQL on port 5432):

vrrp_script check_tcp {
    script "/usr/bin/nc -z localhost 5432"
    interval 2
    weight -20
    fall 3
    rise 2
}

TCP check only verifies the port is open. It does not verify the service is functional.

Process check

For a process that should be running:

vrrp_script check_process {
    script "/usr/bin/pgrep -x myapp"
    interval 2
    weight -20
    fall 3
    rise 2
}

pgrep -x matches the exact process name.

File check

For a file that should exist (e.g. a lock file):

vrrp_script check_file {
    script "/usr/bin/test -f /var/run/myapp.lock"
    interval 2
    weight -20
}

Custom script

For complex checks, write a script:

#!/bin/bash
# /usr/local/bin/check-app-health.sh

# Check process
pgrep -x myapp || exit 1

# Check port
nc -z localhost 8080 || exit 1

# Check HTTP
curl -sf http://localhost/health || exit 1

# All checks passed
exit 0

Reference in keepalived:

vrrp_script check_app {
    script "/usr/local/bin/check-app-health.sh"
    interval 5
    weight -20
    fall 3
    rise 2
}

Weight and priority

When a check fails, the priority is reduced by weight (e.g. -20). If the backup has higher effective priority (after weight reduction), it takes over.

For example:

• Master priority: 100. With -20 weight on check failure: effective 80.
• Backup priority: 90. Always 90.

If the master’s check fails, master effective = 80, backup = 90. Backup takes over.

Multiple checks

You can have multiple checks; the weights add:

vrrp_script check_http { ... weight -10 }
vrrp_script check_db { ... weight -20 }

vrrp_instance VI_1 {
    priority 100
    track_script {
        check_http
        check_db
    }
}

If both fail, priority is reduced by 30.

Knowledge check