CÁCH THỨC HOẠT ĐỘNG CỦA LOAD BALANCER

IPVS Load-balancers with Keepalived

Keepalived: 1.2.7 – Failover and monitoring daemon for LVS clusters

We are using 2 hosts for load-balancing purposes: lb1v.itim.vn and lb2v.itim.vn
If service MASTER LB is down, BACKUP LB must change state to MASTER
(Main schema will be the same for bigger number of IPVS load-balancers)
Each Load-Balancer (LB below) acts as a MASTER and BACKUP for different services. (Read more about support of VRRP protocol in keepalived here: http://www.keepalived.org/documentation.html and in Documentation from Keepalived package.)
At one moment VIP address of service is UP only on a MASTER in the one broadcast-domain. VIP address on LB always on interface, that can answer on ARP requests, for example:
```
    virtual_ipaddress {
        123.30.175.40 dev eth1 label eth1:2
    }
```
IPVS – it’s a very fast kernel module. Keepalived works as service management system for IPVS.

Configs in /etc/keepalived/
- keepalived.conf – instances, VIP addresses, interfaces
- /etc/keepalived/services/<domain_name>.conf – services, pools of real servers

How to change service

1) Change service config /etc/keepalived/services/<domain_name>.conf with pool of real servers for Virtual IP
2) Change vrrp instance (if need) in keepalived.conf on MASTER and BACKUP
2) Restart keepalived process on MASTER; sure that VIP is up and service is ok.
3) Restart keepalived process on BACKUP; sure that service instance is up as a BACKUP

How to add new service

1) Create new instance for service in keepalived.conf – example: takataka.vn instance on BACKUP

# takataka.vn BACKUP instance
vrrp_instance VI_2 {
    state BACKUP
    interface eth0
    lvs_sync_daemon_interface eth0
    virtual_router_id 52
    priority 100
    advert_int 1
    authentication {
        auth_type PASS
        auth_pass <pass-for-instance>
    }
    track_interface {
    eth0
    eth1
    eth1:2
    }
    virtual_ipaddress {
        123.30.175.40 dev eth1 label eth1:2
    }
    debug
}

2) Create virtual service in /etc/keepalived/services/<domain_name>.conf – example: service and pool of real servers for takataka.vn

#takataka.vn
virtual_server 123.30.175.40 80 {
    delay_loop 6
    lb_algo rr
    lb_kind DR
#    persistence_timeout 50
    protocol TCP

    real_server 172.16.16.37 80 {
        weight 10
        inhibit_on_failure
    }

    real_server 172.16.16.38 80 {
        weight 10
        inhibit_on_failure
   }

    real_server 172.16.16.39 80 {
        weight 10
        inhibit_on_failure
    }
}

3) add VIP of service on each real server to the interface, that do not answer on ARP requests (for example lo in /etc/network/interfaces):
```
auto lo:1
iface lo:1 inet static
        address 123.30.175.34
        netmask 255.255.255.255
```

and up it: ifconfig lo:N up

4) Restart keepalived on BACKUP LB; sure that service UP as MASTER and VIP is UP
5) Restart keepalived on MASTER LB; sure that service on BACKUP LB changed state from MASTER to BACKUP; sure that service on MASTER LB is up as MASTER and VIP is UP

LB dataflow diagram

Sysctl

rp.filter must be disabled!!!

Without it load-balancing will NOT work!!!

Firewall and autoban script

UPD. In case of grow of connections number, execution of ipvsadm -lnc takes to much time. So for now ipvs_overlimit.sh disabled on load balancers.

All scripts placed in /etc/itim/firewall/

firewall – main script
ipvs_overlimit.sh – script, that detect ip adresses, with more that 100 ESTABLISHED connections in IPVS connection table.

If script detects bad IP, it writes detected IP into static_blacklist and restart firewall script.

Firewall script reads IP addresses from static_blacklist into dedicated ipset and blocks traffic from IPs in this ipset.

Script runs each minute on lb1v by cron.

Due to big number of records in connections table, ipvs_overlimit.sh is DISABLED now

static_blacklist – list of IP adresses for ban.