Mastering Keepalived: Build High‑Availability LVS Clusters with VRRP

Keepalived Overview

Keepalived is essentially a service for IPVS; it does not require shared storage. IPVS consists of rules, and Keepalived’s main task is to invoke ipvsadm to generate those rules and automatically move user‑requested addresses to available LVS nodes. Thus its HA is highly targeted, unlike generic HA solutions such as corosync.

Keepalived runs as a service on multiple LVS hosts. The active node is called Master and the standby node Backup. The Master continuously announces its heartbeat to the Backup using the VRRP protocol. When the Backup stops receiving the Master’s announcements, it takes over the VIP and the IPVS rules, becoming the new Master.

In addition to monitoring and migrating LVS resources, Keepalived can configure LVS directly without using ipvsadm, and it also provides health‑checking for backend application servers.

In short, Keepalived is an implementation of the VRRP (Virtual Router Redundancy Protocol).

VRRP Working Principle

VRRP allows automatic failover of routers without manual reconfiguration. It provides a virtual IP (VIP) and a virtual MAC (VMAC) that move with the active router, ensuring clients’ ARP tables are updated automatically.

Routers elect a Master based on priority; the Master sends gratuitous ARP packets announcing its virtual MAC.

The Master periodically sends VRRP advertisements with its configuration and status.

If the Master fails, the Backup routers re‑elect a new Master according to priority.

During state transition, the new Master sends a gratuitous ARP with the virtual MAC and VIP, updating other devices’ ARP caches.

If a Backup has higher priority, its preemptive or non‑preemptive mode determines whether it takes over.

VRRP also supports authentication (none, simple 8‑character string, or MD5; Keepalived does not support MD5).

Keepalived Software Architecture

After starting, Keepalived creates a master process and two child processes: the VRRP stack (implementing VRRP) and the Checkers (performing health checks on IPVS backend servers). Failed checks trigger removal of the backend server’s IP from IPVS rules, and successful checks add it back. Notifications can be sent via SMTP.

The master process (Control Plane) parses the configuration file, applies settings, and coordinates the child processes. A WatchDog kernel module monitors the child processes; if a child stops sending heartbeats through an internal Unix socket, the master restarts it.

Installation and Configuration

Prerequisites: disable SELinux, clear iptables rules, turn off the firewall (or allow multicast), synchronize time on all nodes (e.g., systemctl start chronyd), and ensure the network interface used by Keepalived has multicast enabled ( ip link set multicast on dev ens33).

Installing Keepalived

Install via yum: yum install -y keepalived. The binary is /usr/sbin/keepalived, the configuration file is /etc/keepalived/keepalived.conf, and the systemd service file is /usr/lib/systemd/system/keepalived.service.

Sample Configuration File

# Global configuration
global_defs {
   notification_email {
     [email protected]
   }
   notification_email_from [email protected]
   smtp_server 192.168.200.1
   smtp_connect_timeout 30
   router_id LVS_DEVEL
   vrrp_mcast_group4 224.0.0.18
   vrrp_mcast_group6 ff02::12
}

vrrp_instance VI_1 {
    state MASTER
    interface ens33
    virtual_router_id 51
    use_vmac XX:XX:XX:XX:XX
    track_interface { eth0 ens33 }
    priority 100
    advert_int 1
    authentication {
        auth_type PASS
        auth_pass 1111
    }
    virtual_ipaddress {
        192.168.200.16/24 dev eth1
        192.168.200.17/24 dev label eth1:1
    }
    virtual_routes { 192.110.0/24 dev eth2 }
    nopreempt|preempt
    preempt delay 300
    track_script { }
    notify_master ""
    notify_backup ""
    notify_fault ""
}

virtual_server 10.10.10.2 1358 {
    delay_loop 6
    lb_algo rr|wrr|lc|wlc|lblc|sh|dh
    lb_kind NAT|DR|TUN
    nat_mask 255.255.255.0
    persistence_timeout 50
    protocol TCP
    sorry_server 192.168.200.200 1358
    real_server 192.168.200.2 1358 {
        weight 1
        MISC_CHECK {}
        SMTP_CHEKC {}
        TCP_CHECK {
          connect_port <PORT>
          bindto <IP>
          connect_timeout 3
        }
        SSL_GET {}
        notify_up ""
        notify_down "/PATH/SCRIPTS.sh 参数"
        HTTP_GET {
            url { path /testurl/test.jsp digest 640205b7b0fc66c1ea91c463fac6334d status_code 200 }
            url { path /testurl2/test.jsp digest 640205b7b0fc66c1ea91c463fac6334d }
            url { path /testurl3/test.jsp digest 640205b7b0fc66c1ea91c463fac6334d }
            connect_port <PORT>
            bindto <IP>
            connect_timeout 3
            nb_get_retry 3
            delay_before_retry 3
        }
    }
    real_server 192.168.200.3 1358 { … }
}

Configuring Master and Backup Nodes

On Srv01 (Master):

global_defs {
   notification_email { [email protected] }
   notification_email_from [email protected]
   smtp_server 127.0.0.1
   smtp_connect_timeout 30
   router_id srv01
}
vrrp_instance VI_1 {
    state MASTER
    interface ens33
    virtual_router_id 51
    priority 100
    advert_int 1
    authentication { auth_type PASS auth_pass 1111 }
    virtual_ipaddress { 172.16.42.111/24 brd 172.16.42.111 dev ens33 label ens33:0 }
    preempt delay 60
}

On Srv02 (Backup): same as above but state BACKUP, priority 90, and router_id srv02.

Start both nodes; the virtual interface ens33:0 with the VIP is created automatically.

Testing Failover

Check status with systemctl status keepalived and view logs ( cat /var/log/message) to see role changes. Use tcpdump -i ens33 -nn host 224.0.0.18 to observe VRRP advertisements. Stop Keepalived on the Master to verify that the Backup takes over the VIP.

Configure LVS in keepalived.conf (example shown) and use ipvsadm -Ln to view IPVS rules. Test load balancing with a curl loop; the round‑robin algorithm distributes requests between the two real servers.

Sample Notification Script

#!/bin/bash
if [ $1 == "up" ]; then
  echo "Srv02 is UP" > /tmp/notify.txt
elif [ $1 == "down" ]; then
  echo "Srv02 is DOWN" > /tmp/notify.txt
fi