{"id":688,"date":"2018-07-08T00:01:22","date_gmt":"2018-07-07T17:01:22","guid":{"rendered":"https:\/\/lagonet.vn\/?p=688"},"modified":"2018-07-08T00:01:22","modified_gmt":"2018-07-07T17:01:22","slug":"cach-thuc-hoat-dong-cua-load-balancer","status":"publish","type":"post","link":"https:\/\/kb.lagonet.vn\/?p=688","title":{"rendered":"C\u00c1CH TH\u1ee8C HO\u1ea0T \u0110\u1ed8NG C\u1ee6A LOAD BALANCER"},"content":{"rendered":"<h2 id=\"IPVSLoad-balancerswithKeepalived\">IPVS Load-balancers with Keepalived<a id=\"section_1\" class=\"anchor\" title=\"Edit this section\" href=\"https:\/\/trac.coccoc.com\/coccoc\/wiki\/NCC\/net-structure\/Load-Balancers?action=edit&amp;section=1\"><\/a><\/h2>\n<p><strong>Keepalived<\/strong>:\u00a0<strong>1.2.7<\/strong>\u00a0&#8211; Failover and monitoring daemon for LVS clusters<\/p>\n<ul>\n<li>We are using 2 hosts for load-balancing purposes:\u00a0<a class=\"ext-link\" href=\"http:\/\/racktables.itim.vn\/index.php?page=object&amp;object_id=150\"><span class=\"icon\">\u200b<\/span>lb1v.itim.vn<\/a>\u00a0and\u00a0<a class=\"ext-link\" href=\"http:\/\/racktables.itim.vn\/index.php?page=object&amp;object_id=151\"><span class=\"icon\">\u200b<\/span>lb2v.itim.vn<\/a><\/li>\n<li>If service MASTER LB is down, BACKUP LB must change state to MASTER<\/li>\n<li><em>(Main schema will be the same for bigger number of IPVS load-balancers)<\/em><\/li>\n<li>Each Load-Balancer (LB below) acts as a MASTER and BACKUP for different services. (Read more about support of\u00a0<strong>VRRP<\/strong>\u00a0protocol in keepalived here:\u00a0<a class=\"ext-link\" href=\"http:\/\/www.keepalived.org\/documentation.html\"><span class=\"icon\">\u200b<\/span>http:\/\/www.keepalived.org\/documentation.html<\/a>\u00a0and in Documentation from Keepalived package.)<\/li>\n<li>At one moment VIP address of service is UP only on a MASTER in the one broadcast-domain. VIP address on LB always on interface, that can answer on ARP requests, for example:\n<pre class=\"wiki\">    virtual_ipaddress {\n        123.30.175.40 dev eth1 label eth1:2\n    }\n<\/pre>\n<\/li>\n<li>IPVS &#8211; it&#8217;s a very fast kernel module. Keepalived works as service management system for IPVS.<\/li>\n<\/ul>\n<ul>\n<li>Configs in \/etc\/keepalived\/\n<ul>\n<li><strong>keepalived.conf<\/strong>\u00a0&#8211; instances, VIP addresses, interfaces<\/li>\n<li><strong>\/etc\/keepalived\/services\/&lt;domain_name&gt;.conf<\/strong>\u00a0&#8211; services, pools of real servers<\/li>\n<\/ul>\n<\/li>\n<\/ul>\n<h2 id=\"Howtochangeservice\">How to change service<a id=\"section_2\" class=\"anchor\" title=\"Edit this section\" href=\"https:\/\/trac.coccoc.com\/coccoc\/wiki\/NCC\/net-structure\/Load-Balancers?action=edit&amp;section=2\"><\/a><\/h2>\n<ul>\n<li>1) Change service config \/etc\/keepalived\/services\/&lt;domain_name&gt;.conf with pool of real servers for Virtual IP<\/li>\n<li>2) Change vrrp instance (if need) in\u00a0<strong>keepalived.conf<\/strong>\u00a0on MASTER and BACKUP<\/li>\n<li>2) Restart keepalived process on MASTER; sure that VIP is up and service is ok.<\/li>\n<li>3) Restart keepalived process on BACKUP; sure that service instance is up as a BACKUP<\/li>\n<\/ul>\n<h2 id=\"Howtoaddnewservice\">How to add new service<a id=\"section_3\" class=\"anchor\" title=\"Edit this section\" href=\"https:\/\/trac.coccoc.com\/coccoc\/wiki\/NCC\/net-structure\/Load-Balancers?action=edit&amp;section=3\"><\/a><\/h2>\n<ul>\n<li>1) Create new instance for service in\u00a0<strong>keepalived.conf<\/strong>\u00a0&#8211; example: takataka.vn instance on BACKUP\n<pre class=\"wiki\"># takataka.vn BACKUP instance\nvrrp_instance VI_2 {\n    state BACKUP\n    interface eth0\n    lvs_sync_daemon_interface eth0\n    virtual_router_id 52\n    priority 100\n    advert_int 1\n    authentication {\n        auth_type PASS\n        auth_pass &lt;pass-for-instance&gt;\n    }\n    track_interface {\n    eth0\n    eth1\n    eth1:2\n    }\n    virtual_ipaddress {\n        123.30.175.40 dev eth1 label eth1:2\n    }\n    debug\n}\n<\/pre>\n<\/li>\n<\/ul>\n<ul>\n<li>2) Create virtual service in\u00a0<strong>\/etc\/keepalived\/services\/&lt;domain_name&gt;.conf<\/strong>\u00a0&#8211; example: service and pool of real servers for takataka.vn\n<pre class=\"wiki\">#takataka.vn\nvirtual_server 123.30.175.40 80 {\n    delay_loop 6\n    lb_algo rr\n    lb_kind DR\n#    persistence_timeout 50\n    protocol TCP\n\n    real_server 172.16.16.37 80 {\n        weight 10\n        inhibit_on_failure\n    }\n\n    real_server 172.16.16.38 80 {\n        weight 10\n        inhibit_on_failure\n   }\n\n    real_server 172.16.16.39 80 {\n        weight 10\n        inhibit_on_failure\n    }\n}\n<\/pre>\n<\/li>\n<\/ul>\n<ul>\n<li>3) add VIP of service on each real server to the interface, that do not answer on ARP requests (for example\u00a0<strong>lo<\/strong>\u00a0in\u00a0<strong>\/etc\/network\/interfaces<\/strong>):\n<pre class=\"wiki\">auto lo:1\niface lo:1 inet static\n        address 123.30.175.34\n        netmask 255.255.255.255\n<\/pre>\n<\/li>\n<\/ul>\n<p>and up it:\u00a0<em>ifconfig lo:N up<\/em><\/p>\n<ul>\n<li>4) Restart keepalived on BACKUP LB; sure that service UP as\u00a0<strong>MASTER<\/strong>\u00a0and VIP is UP<\/li>\n<li>5) Restart keepalived on MASTER LB; sure that service on BACKUP LB changed state from\u00a0<strong>MASTER<\/strong>\u00a0to\u00a0<strong>BACKUP<\/strong>; sure that service on MASTER LB is up as\u00a0<strong>MASTER<\/strong>\u00a0and VIP is UP<\/li>\n<\/ul>\n<h2 id=\"LBdataflowdiagram\">LB dataflow diagram<a id=\"section_4\" class=\"anchor\" title=\"Edit this section\" href=\"https:\/\/trac.coccoc.com\/coccoc\/wiki\/NCC\/net-structure\/Load-Balancers?action=edit&amp;section=4\"><\/a><\/h2>\n<figure id=\"attachment_689\" aria-describedby=\"caption-attachment-689\" style=\"width: 1283px\" class=\"wp-caption alignnone\"><img loading=\"lazy\" decoding=\"async\" class=\"size-full wp-image-689\" src=\"https:\/\/lagonet.vn\/wp-content\/uploads\/2018\/07\/LB-diag.jpeg\" alt=\"Load balancer structure\" width=\"1283\" height=\"1211\" \/><figcaption id=\"caption-attachment-689\" class=\"wp-caption-text\">Load balancer structure<\/figcaption><\/figure>\n<h3 id=\"Sysctl\">Sysctl<a id=\"section_5\" class=\"anchor\" title=\"Edit this section\" href=\"https:\/\/trac.coccoc.com\/coccoc\/wiki\/NCC\/net-structure\/Load-Balancers?action=edit&amp;section=5\"><\/a><\/h3>\n<p><strong>rp.filter\u00a0<\/strong>must be disabled!!!<\/p>\n<p><strong>Without it load-balancing will NOT work!!!<\/strong><\/p>\n<div class=\"wikipage searchable\">\n<div id=\"wikipage\">\n<h2 id=\"Firewallandautobanscript\">Firewall and autoban script<a id=\"section_6\" class=\"anchor\" title=\"Edit this section\" href=\"https:\/\/trac.coccoc.com\/coccoc\/wiki\/NCC\/net-structure\/Load-Balancers?action=edit&amp;section=6\"><\/a><\/h2>\n<p>UPD. In case of grow of connections number, execution of ipvsadm -lnc takes to much time. So for now\u00a0<strong>ipvs_overlimit.sh<\/strong>\u00a0disabled on load balancers.<\/p>\n<p>All scripts placed in \/etc\/itim\/firewall\/<\/p>\n<ul>\n<li><strong>firewall<\/strong>\u00a0&#8211; main script<\/li>\n<li><strong>ipvs_overlimit.sh<\/strong>\u00a0&#8211; script, that detect ip adresses, with more that 100 ESTABLISHED connections in IPVS connection table.<\/li>\n<\/ul>\n<blockquote>\n<blockquote><p><em>If script detects bad IP, it writes detected IP into\u00a0<strong>static_blacklist<\/strong>\u00a0and restart firewall script.<\/em><\/p><\/blockquote>\n<\/blockquote>\n<blockquote>\n<blockquote><p><em>Firewall script reads IP addresses from\u00a0<strong>static_blacklist<\/strong>\u00a0into dedicated ipset and blocks traffic from IPs in this ipset.<\/em><\/p><\/blockquote>\n<\/blockquote>\n<blockquote>\n<blockquote><p><em>Script runs each minute on lb1v by cron.<\/em><\/p><\/blockquote>\n<p><strong>Due to big number of records in connections table, ipvs_overlimit.sh is DISABLED now<\/strong><\/p><\/blockquote>\n<ul>\n<li><strong>static_blacklist<\/strong>\u00a0&#8211; list of IP adresses for ban.<\/li>\n<\/ul>\n<\/div>\n<\/div>\n<div id=\"attachments\" class=\"collapsed\">\n<h3 class=\"foldable\"><\/h3>\n<\/div>\n","protected":false},"excerpt":{"rendered":"<p>IPVS Load-balancers with Keepalived Keepalived:\u00a01.2.7\u00a0&#8211; Failover and monitoring daemon for LVS clusters We are using 2 hosts for load-balancing purposes:\u00a0\u200blb1v.itim.vn\u00a0and\u00a0\u200blb2v.itim.vn [&hellip;]<\/p>\n","protected":false},"author":1,"featured_media":0,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"site-sidebar-layout":"default","site-content-layout":"","ast-site-content-layout":"","site-content-style":"default","site-sidebar-style":"default","ast-global-header-display":"","ast-banner-title-visibility":"","ast-main-header-display":"","ast-hfb-above-header-display":"","ast-hfb-below-header-display":"","ast-hfb-mobile-header-display":"","site-post-title":"","ast-breadcrumbs-content":"","ast-featured-img":"","footer-sml-layout":"","theme-transparent-header-meta":"","adv-header-id-meta":"","stick-header-meta":"","header-above-stick-meta":"","header-main-stick-meta":"","header-below-stick-meta":"","astra-migrate-meta-layouts":"default","ast-page-background-enabled":"default","ast-page-background-meta":{"desktop":{"background-color":"var(--ast-global-color-4)","background-image":"","background-repeat":"repeat","background-position":"center center","background-size":"auto","background-attachment":"scroll","background-type":"","background-media":"","overlay-type":"","overlay-color":"","overlay-opacity":"","overlay-gradient":""},"tablet":{"background-color":"","background-image":"","background-repeat":"repeat","background-position":"center center","background-size":"auto","background-attachment":"scroll","background-type":"","background-media":"","overlay-type":"","overlay-color":"","overlay-opacity":"","overlay-gradient":""},"mobile":{"background-color":"","background-image":"","background-repeat":"repeat","background-position":"center center","background-size":"auto","background-attachment":"scroll","background-type":"","background-media":"","overlay-type":"","overlay-color":"","overlay-opacity":"","overlay-gradient":""}},"ast-content-background-meta":{"desktop":{"background-color":"var(--ast-global-color-5)","background-image":"","background-repeat":"repeat","background-position":"center center","background-size":"auto","background-attachment":"scroll","background-type":"","background-media":"","overlay-type":"","overlay-color":"","overlay-opacity":"","overlay-gradient":""},"tablet":{"background-color":"var(--ast-global-color-5)","background-image":"","background-repeat":"repeat","background-position":"center center","background-size":"auto","background-attachment":"scroll","background-type":"","background-media":"","overlay-type":"","overlay-color":"","overlay-opacity":"","overlay-gradient":""},"mobile":{"background-color":"var(--ast-global-color-5)","background-image":"","background-repeat":"repeat","background-position":"center center","background-size":"auto","background-attachment":"scroll","background-type":"","background-media":"","overlay-type":"","overlay-color":"","overlay-opacity":"","overlay-gradient":""}},"footnotes":""},"categories":[6],"tags":[],"class_list":["post-688","post","type-post","status-publish","format-standard","hentry","category-networking"],"_links":{"self":[{"href":"https:\/\/kb.lagonet.vn\/index.php?rest_route=\/wp\/v2\/posts\/688","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/kb.lagonet.vn\/index.php?rest_route=\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/kb.lagonet.vn\/index.php?rest_route=\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/kb.lagonet.vn\/index.php?rest_route=\/wp\/v2\/users\/1"}],"replies":[{"embeddable":true,"href":"https:\/\/kb.lagonet.vn\/index.php?rest_route=%2Fwp%2Fv2%2Fcomments&post=688"}],"version-history":[{"count":0,"href":"https:\/\/kb.lagonet.vn\/index.php?rest_route=\/wp\/v2\/posts\/688\/revisions"}],"wp:attachment":[{"href":"https:\/\/kb.lagonet.vn\/index.php?rest_route=%2Fwp%2Fv2%2Fmedia&parent=688"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/kb.lagonet.vn\/index.php?rest_route=%2Fwp%2Fv2%2Fcategories&post=688"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/kb.lagonet.vn\/index.php?rest_route=%2Fwp%2Fv2%2Ftags&post=688"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}