Lab: Keepalived failover - configure, test, document

This lab configures keepalived for floating-IP failover between two hosts and tests the failover. By the end you will have a working setup and a tested procedure.

Tasks

Task 1: Install keepalived and the service it checks

The health check in Task 2 curls http://localhost/health. If nothing serves that path the check fails on both hosts from the moment keepalived starts, both get the weight -20 penalty, and the failover test in Task 8 cannot work — see the arithmetic below. Install the web server first, on both hosts:

sudo apt install -y keepalived nginx curl

# A /health endpoint that returns 200 and nothing else
sudo tee /etc/nginx/conf.d/health.conf >/dev/null <<'EOF'
server {
    listen 80 default_server;
    location /health { return 200 "ok\n"; add_header Content-Type text/plain; }
}
EOF
sudo nginx -t && sudo systemctl reload nginx

# Must return 200 on BOTH hosts before keepalived starts
curl -sf -o /dev/null -w '%{http_code}\n' http://localhost/health

Task 2: Configure the master

On host1 (will be master):

sudo tee /etc/keepalived/keepalived.conf <<EOF
global_defs {
    enable_script_security
    script_user keepalived_script
}

vrrp_script check_service {
    script "/usr/bin/curl -sf http://localhost/health || exit 1"
    interval 2
    weight -20
    fall 3
    rise 2
}

vrrp_instance VI_1 {
    state MASTER
    interface eth0
    virtual_router_id 51
    priority 100
    advert_int 1
    authentication {
        auth_type PASS
        auth_pass mysecret
    }
    virtual_ipaddress {
        10.0.0.100/24
    }
    track_script {
        check_service
    }
}
EOF

Two things about auth_pass. VRRP’s PASS authentication sends the value in clear text on the wire, so it prevents misconfiguration — two clusters on one segment adopting each other’s VIP — and provides no security against anyone who can capture traffic. And it is limited to 8 characters: keepalived silently truncates anything longer, so a 20-character “strong” password on one host and a different 20-character one that shares the first 8 will authenticate to each other.

Treat virtual_router_id as the thing that must be unique per segment, and put the real access control in the firewall (VRRP is IP protocol 112, multicast 224.0.0.18).

Task 3: Configure the backup

On host2 (will be backup):

sudo tee /etc/keepalived/keepalived.conf <<EOF
global_defs {
    enable_script_security
    script_user keepalived_script
}

vrrp_script check_service {
    script "/usr/bin/curl -sf http://localhost/health || exit 1"
    interval 2
    weight -20
    fall 3
    rise 2
}

vrrp_instance VI_1 {
    state BACKUP
    interface eth0
    virtual_router_id 51
    priority 90
    advert_int 1
    authentication {
        auth_type PASS
        auth_pass mysecret
    }
    virtual_ipaddress {
        10.0.0.100/24
    }
    track_script {
        check_service
    }
}
EOF

Task 4: Start keepalived

On both hosts:

sudo systemctl enable --now keepalived

Task 5: Verify

On host1 (master):

ip addr show eth0 | grep 10.0.0.100

Should show the VIP.

On host2 (backup):

ip addr show eth0 | grep 10.0.0.100

Should NOT show the VIP.

Task 6: Test failover

Stop keepalived on host1:

ssh host1 "sudo systemctl stop keepalived"

On host2, the VIP should now be held:

ip addr show eth0 | grep 10.0.0.100

Verify the service works:

curl -I http://10.0.0.100/health

Task 7: Test preemption

Start keepalived on host1 again:

ssh host1 "sudo systemctl start keepalived"

By default, host1 (higher priority) takes back the VIP.

Task 8: Test health check failover

First confirm the starting state is the one the arithmetic assumes — both checks healthy, host1 holding the VIP:

# On both hosts
curl -sf -o /dev/null -w '%{http_code}\n' http://localhost/health   # 200 on both

Then break the service on host1 only:

sudo systemctl stop nginx

host1’s check now fails 3 times (fall 3 at interval 2, so about 6 seconds), its priority drops to 80, host2’s 90 wins, and the VIP migrates. Watch it happen rather than sleeping:

# On host2
sudo journalctl -u keepalived -f     # expect "Entering MASTER STATE"

Verify:

ip addr show eth0 | grep 10.0.0.100    # on host2
curl -I http://10.0.0.100/health       # service works

Restart the service on host1; the VIP returns.

Task 9: Document

KEEPALIVED TEST REPORT
Date: 2026-08-09
VIP: 10.0.0.100
Hosts: host1 (master, priority 100), host2 (backup, priority 90)
Auth: PASS shared secret
Health check: HTTP /health

Tests:
- Initial state: VIP on host1
- Failover: stopped keepalived on host1, VIP on host2
- Preemption: restarted host1, VIP back to host1
- Health check: stopped service, VIP on host2

Findings:
- All tests pass
- Health check triggers failover as expected
- Preemption works as expected