Keepalived+LVS+nginx搭建nginx高可用集群
1. 环境准备
1. VMware;
2. 4台CentOs7虚拟主机：192.168.122.248,192.168.122.68, 192.168.122.110, 192.168.122.167
3. 系统服务：LVS, Keepalived
4. Web服务器：nginx
5. 集群搭建：LVS DR模式
2. 软件安装
在四台虚拟机上，我们以如下方式搭建集群：
192.168.122.248 lvs+keepalived-1
192.168.122.68 lvs+keepalived-2
192.168.122.110 nginx-1
192.168.122.167 nginx-2
这里我们使用192.168.122.248和192.168.122.68两台机器作为lvs+keepalived的工作机器，也就是说这两台机器的作用主要是进行负载均衡和故障检测和下线的；我们使用192.168.122.110和192.168.122.167两台机器作为应用服务器，主要是对外提供服务的。这四台服务器作为整个后端集群服务，并且对外提供的虚拟ip是192.168.122.120。需要说明的是，这里的keepalived所检测的服务是两台lvs服务器，这两台服务器，一台作为master服务器，一台作为backup服务器，两者在负载均衡的配置上是完全一样的。在正常情况下，客户端请求虚拟ip的时候，lvs会将该请求转发到master服务器上，然后master服务器根据配置的负载均衡策略选择一台应用服务器，并且将请求发送给该应用服务器进行处理。如果在某个时刻，lvs的master服务器由于故障宕机了，keepalived就会检测到该故障，并且进行故障下线，然后将backup机器上线用于提供服务，从而实现故障转移的功能。
2.1 lvs+keepalived安装
在192.168.122.248和192.168.122.68上安装ipvs和keepalived：
[root@lvskeepalived-1 ~]# yum install ipvsadm keepalived -y Loaded plugins: fastestmirror Loading mirror speeds from cached hostfile * base: mirrors.aliyun.com * extras: mirrors.aliyun.com * updates: mirrors.aliyun.com base | 3.6 kB 00:00:00 epel | 5.3 kB 00:00:00 extras | 3.4 kB 00:00:00 updates | 3.4 kB 00:00:00 (1/2): epel/x86_64/updateinfo | 999 kB 00:00:00 (2/2): epel/x86_64/primary_db | 6.8 MB 00:00:01 Package ipvsadm-1.27-7.el7.x86_64 already installed and latest version Resolving Dependencies --> Running transaction check ---> Package keepalived.x86_64 0:1.3.5-8.el7_6.5 will be installed --> Processing Dependency: libnetsnmpmibs.so.31()(64bit) for package: keepalived-1.3.5-8.el7_6.5.x86_64 --> Processing Dependency: libnetsnmpagent.so.31()(64bit) for package: keepalived-1.3.5-8.el7_6.5.x86_64 --> Processing Dependency: libnetsnmp.so.31()(64bit) for package: keepalived-1.3.5-8.el7_6.5.x86_64 --> Running transaction check ---> Package net-snmp-agent-libs.x86_64 1:5.7.2-38.el7_6.2 will be installed ---> Package net-snmp-libs.x86_64 1:5.7.2-38.el7_6.2 will be installed --> Finished Dependency Resolution Dependencies Resolved =================================================================================================================================================================================================== Package Arch Version Repository Size =================================================================================================================================================================================================== Installing: keepalived x86_64 1.3.5-8.el7_6.5 updates 331 k Installing for dependencies: net-snmp-agent-libs x86_64 1:5.7.2-38.el7_6.2 updates 705 k net-snmp-libs x86_64 1:5.7.2-38.el7_6.2 updates 749 k Transaction Summary =================================================================================================================================================================================================== Install 1 Package (+2 Dependent packages) Total download size: 1.7 M Installed size: 6.0 M Downloading packages: (1/3): net-snmp-libs-5.7.2-38.el7_6.2.x86_64.rpm | 749 kB 00:00:00 (2/3): keepalived-1.3.5-8.el7_6.5.x86_64.rpm | 331 kB 00:00:00 (3/3): net-snmp-agent-libs-5.7.2-38.el7_6.2.x86_64.rpm | 705 kB 00:00:01 --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- Total 1.1 MB/s | 1.7 MB 00:00:01 Running transaction check Running transaction test Transaction test succeeded Running transaction Installing : 1:net-snmp-libs-5.7.2-38.el7_6.2.x86_64 1/3 Installing : 1:net-snmp-agent-libs-5.7.2-38.el7_6.2.x86_64 2/3 Installing : keepalived-1.3.5-8.el7_6.5.x86_64 3/3 Verifying : 1:net-snmp-agent-libs-5.7.2-38.el7_6.2.x86_64 1/3 Verifying : keepalived-1.3.5-8.el7_6.5.x86_64 2/3 Verifying : 1:net-snmp-libs-5.7.2-38.el7_6.2.x86_64 3/3 Installed: keepalived.x86_64 0:1.3.5-8.el7_6.5 Dependency Installed: net-snmp-agent-libs.x86_64 1:5.7.2-38.el7_6.2 net-snmp-libs.x86_64 1:5.7.2-38.el7_6.2 Complete!
在192.168.122.110和192.168.122.167上安装nginx：
[root@nginx-1 ~]# yum install nginx -y Loaded plugins: fastestmirror Loading mirror speeds from cached hostfile * base: mirrors.aliyun.com * extras: mirrors.aliyun.com * updates: mirrors.aliyun.com base | 3.6 kB 00:00:00 epel | 5.3 kB 00:00:00 extras | 3.4 kB 00:00:00 updates | 3.4 kB 00:00:00 (1/2): epel/x86_64/updateinfo | 999 kB 00:00:00 (2/2): epel/x86_64/primary_db | 6.8 MB 00:00:01 Resolving Dependencies --> Running transaction check ---> Package nginx.x86_64 1:1.12.2-3.el7 will be installed --> Processing Dependency: nginx-all-modules = 1:1.12.2-3.el7 for package: 1:nginx-1.12.2-3.el7.x86_64 --> Processing Dependency: nginx-filesystem = 1:1.12.2-3.el7 for package: 1:nginx-1.12.2-3.el7.x86_64 --> Processing Dependency: nginx-filesystem for package: 1:nginx-1.12.2-3.el7.x86_64 --> Processing Dependency: libprofiler.so.0()(64bit) for package: 1:nginx-1.12.2-3.el7.x86_64 --> Running transaction check ---> Package gperftools-libs.x86_64 0:2.6.1-1.el7 will be installed ---> Package nginx-all-modules.noarch 1:1.12.2-3.el7 will be installed --> Processing Dependency: nginx-mod-http-geoip = 1:1.12.2-3.el7 for package: 1:nginx-all-modules-1.12.2-3.el7.noarch --> Processing Dependency: nginx-mod-http-image-filter = 1:1.12.2-3.el7 for package: 1:nginx-all-modules-1.12.2-3.el7.noarch --> Processing Dependency: nginx-mod-http-perl = 1:1.12.2-3.el7 for package: 1:nginx-all-modules-1.12.2-3.el7.noarch --> Processing Dependency: nginx-mod-http-xslt-filter = 1:1.12.2-3.el7 for package: 1:nginx-all-modules-1.12.2-3.el7.noarch --> Processing Dependency: nginx-mod-mail = 1:1.12.2-3.el7 for package: 1:nginx-all-modules-1.12.2-3.el7.noarch --> Processing Dependency: nginx-mod-stream = 1:1.12.2-3.el7 for package: 1:nginx-all-modules-1.12.2-3.el7.noarch ---> Package nginx-filesystem.noarch 1:1.12.2-3.el7 will be installed --> Running transaction check ---> Package nginx-mod-http-geoip.x86_64 1:1.12.2-3.el7 will be installed ---> Package nginx-mod-http-image-filter.x86_64 1:1.12.2-3.el7 will be installed --> Processing Dependency: gd for package: 1:nginx-mod-http-image-filter-1.12.2-3.el7.x86_64 --> Processing Dependency: libgd.so.2()(64bit) for package: 1:nginx-mod-http-image-filter-1.12.2-3.el7.x86_64 ---> Package nginx-mod-http-perl.x86_64 1:1.12.2-3.el7 will be installed ---> Package nginx-mod-http-xslt-filter.x86_64 1:1.12.2-3.el7 will be installed --> Processing Dependency: libxslt.so.1(LIBXML2_1.0.11)(64bit) for package: 1:nginx-mod-http-xslt-filter-1.12.2-3.el7.x86_64 --> Processing Dependency: libxslt.so.1(LIBXML2_1.0.18)(64bit) for package: 1:nginx-mod-http-xslt-filter-1.12.2-3.el7.x86_64 --> Processing Dependency: libexslt.so.0()(64bit) for package: 1:nginx-mod-http-xslt-filter-1.12.2-3.el7.x86_64 --> Processing Dependency: libxslt.so.1()(64bit) for package: 1:nginx-mod-http-xslt-filter-1.12.2-3.el7.x86_64 ---> Package nginx-mod-mail.x86_64 1:1.12.2-3.el7 will be installed ---> Package nginx-mod-stream.x86_64 1:1.12.2-3.el7 will be installed --> Running transaction check ---> Package gd.x86_64 0:2.0.35-26.el7 will be installed --> Processing Dependency: libjpeg.so.62(LIBJPEG_6.2)(64bit) for package: gd-2.0.35-26.el7.x86_64 --> Processing Dependency: libjpeg.so.62()(64bit) for package: gd-2.0.35-26.el7.x86_64 --> Processing Dependency: libfontconfig.so.1()(64bit) for package: gd-2.0.35-26.el7.x86_64 --> Processing Dependency: libXpm.so.4()(64bit) for package: gd-2.0.35-26.el7.x86_64 --> Processing Dependency: libX11.so.6()(64bit) for package: gd-2.0.35-26.el7.x86_64 ---> Package libxslt.x86_64 0:1.1.28-5.el7 will be installed --> Running transaction check ---> Package fontconfig.x86_64 0:2.13.0-4.3.el7 will be installed --> Processing Dependency: fontpackages-filesystem for package: fontconfig-2.13.0-4.3.el7.x86_64 --> Processing Dependency: dejavu-sans-fonts for package: fontconfig-2.13.0-4.3.el7.x86_64 ---> Package libX11.x86_64 0:1.6.5-2.el7 will be installed --> Processing Dependency: libX11-common >= 1.6.5-2.el7 for package: libX11-1.6.5-2.el7.x86_64 --> Processing Dependency: libxcb.so.1()(64bit) for package: libX11-1.6.5-2.el7.x86_64 ---> Package libXpm.x86_64 0:3.5.12-1.el7 will be installed ---> Package libjpeg-turbo.x86_64 0:1.2.90-6.el7 will be installed --> Running transaction check ---> Package dejavu-sans-fonts.noarch 0:2.33-6.el7 will be installed --> Processing Dependency: dejavu-fonts-common = 2.33-6.el7 for package: dejavu-sans-fonts-2.33-6.el7.noarch ---> Package fontpackages-filesystem.noarch 0:1.44-8.el7 will be installed ---> Package libX11-common.noarch 0:1.6.5-2.el7 will be installed ---> Package libxcb.x86_64 0:1.13-1.el7 will be installed --> Processing Dependency: libXau.so.6()(64bit) for package: libxcb-1.13-1.el7.x86_64 --> Running transaction check ---> Package dejavu-fonts-common.noarch 0:2.33-6.el7 will be installed ---> Package libXau.x86_64 0:1.0.8-2.1.el7 will be installed --> Finished Dependency Resolution Dependencies Resolved =================================================================================================================================================================================================== Package Arch Version Repository Size =================================================================================================================================================================================================== Installing: nginx x86_64 1:1.12.2-3.el7 epel 531 k Installing for dependencies: dejavu-fonts-common noarch 2.33-6.el7 base 64 k dejavu-sans-fonts noarch 2.33-6.el7 base 1.4 M fontconfig x86_64 2.13.0-4.3.el7 base 254 k fontpackages-filesystem noarch 1.44-8.el7 base 9.9 k gd x86_64 2.0.35-26.el7 base 146 k gperftools-libs x86_64 2.6.1-1.el7 base 272 k libX11 x86_64 1.6.5-2.el7 base 606 k libX11-common noarch 1.6.5-2.el7 base 164 k libXau x86_64 1.0.8-2.1.el7 base 29 k libXpm x86_64 3.5.12-1.el7 base 55 k libjpeg-turbo x86_64 1.2.90-6.el7 base 134 k libxcb x86_64 1.13-1.el7 base 214 k libxslt x86_64 1.1.28-5.el7 base 242 k nginx-all-modules noarch 1:1.12.2-3.el7 epel 16 k nginx-filesystem noarch 1:1.12.2-3.el7 epel 17 k nginx-mod-http-geoip x86_64 1:1.12.2-3.el7 epel 23 k nginx-mod-http-image-filter x86_64 1:1.12.2-3.el7 epel 27 k nginx-mod-http-perl x86_64 1:1.12.2-3.el7 epel 36 k nginx-mod-http-xslt-filter x86_64 1:1.12.2-3.el7 epel 26 k nginx-mod-mail x86_64 1:1.12.2-3.el7 epel 54 k nginx-mod-stream x86_64 1:1.12.2-3.el7 epel 76 k Transaction Summary =================================================================================================================================================================================================== Install 1 Package (+21 Dependent packages) Total download size: 4.3 M Installed size: 14 M Downloading packages: (1/22): dejavu-fonts-common-2.33-6.el7.noarch.rpm | 64 kB 00:00:00 (2/22): fontconfig-2.13.0-4.3.el7.x86_64.rpm | 254 kB 00:00:00 (3/22): fontpackages-filesystem-1.44-8.el7.noarch.rpm | 9.9 kB 00:00:00 (4/22): gperftools-libs-2.6.1-1.el7.x86_64.rpm | 272 kB 00:00:00 (5/22): libX11-1.6.5-2.el7.x86_64.rpm | 606 kB 00:00:00 (6/22): gd-2.0.35-26.el7.x86_64.rpm | 146 kB 00:00:00 (7/22): libX11-common-1.6.5-2.el7.noarch.rpm | 164 kB 00:00:00 (8/22): libXau-1.0.8-2.1.el7.x86_64.rpm | 29 kB 00:00:00 (9/22): libXpm-3.5.12-1.el7.x86_64.rpm | 55 kB 00:00:00 (10/22): libjpeg-turbo-1.2.90-6.el7.x86_64.rpm | 134 kB 00:00:00 (11/22): libxslt-1.1.28-5.el7.x86_64.rpm | 242 kB 00:00:00 (12/22): dejavu-sans-fonts-2.33-6.el7.noarch.rpm | 1.4 MB 00:00:00 (13/22): libxcb-1.13-1.el7.x86_64.rpm | 214 kB 00:00:00 (14/22): nginx-all-modules-1.12.2-3.el7.noarch.rpm | 16 kB 00:00:00 (15/22): nginx-1.12.2-3.el7.x86_64.rpm | 531 kB 00:00:00 (16/22): nginx-filesystem-1.12.2-3.el7.noarch.rpm | 17 kB 00:00:00 (17/22): nginx-mod-http-geoip-1.12.2-3.el7.x86_64.rpm | 23 kB 00:00:00 (18/22): nginx-mod-http-image-filter-1.12.2-3.el7.x86_64.rpm | 27 kB 00:00:00 (19/22): nginx-mod-http-xslt-filter-1.12.2-3.el7.x86_64.rpm | 26 kB 00:00:00 (20/22): nginx-mod-mail-1.12.2-3.el7.x86_64.rpm | 54 kB 00:00:00 (21/22): nginx-mod-stream-1.12.2-3.el7.x86_64.rpm | 76 kB 00:00:00 (22/22): nginx-mod-http-perl-1.12.2-3.el7.x86_64.rpm | 36 kB 00:00:00 --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- Total 3.0 MB/s | 4.3 MB 00:00:01 Running transaction check Running transaction test Transaction test succeeded Running transaction Installing : fontpackages-filesystem-1.44-8.el7.noarch 1/22 Installing : dejavu-fonts-common-2.33-6.el7.noarch 2/22 Installing : dejavu-sans-fonts-2.33-6.el7.noarch 3/22 Installing : fontconfig-2.13.0-4.3.el7.x86_64 4/22 Installing : 1:nginx-filesystem-1.12.2-3.el7.noarch 5/22 Installing : libX11-common-1.6.5-2.el7.noarch 6/22 Installing : gperftools-libs-2.6.1-1.el7.x86_64 7/22 Installing : libXau-1.0.8-2.1.el7.x86_64 8/22 Installing : libxcb-1.13-1.el7.x86_64 9/22 Installing : libX11-1.6.5-2.el7.x86_64 10/22 Installing : libXpm-3.5.12-1.el7.x86_64 11/22 Installing : libxslt-1.1.28-5.el7.x86_64 12/22 Installing : libjpeg-turbo-1.2.90-6.el7.x86_64 13/22 Installing : gd-2.0.35-26.el7.x86_64 14/22 Installing : 1:nginx-mod-mail-1.12.2-3.el7.x86_64 15/22 Installing : 1:nginx-mod-stream-1.12.2-3.el7.x86_64 16/22 Installing : 1:nginx-mod-http-xslt-filter-1.12.2-3.el7.x86_64 17/22 Installing : 1:nginx-mod-http-image-filter-1.12.2-3.el7.x86_64 18/22 Installing : 1:nginx-mod-http-perl-1.12.2-3.el7.x86_64 19/22 Installing : 1:nginx-mod-http-geoip-1.12.2-3.el7.x86_64 20/22 Installing : 1:nginx-all-modules-1.12.2-3.el7.noarch 21/22 Installing : 1:nginx-1.12.2-3.el7.x86_64 22/22 Verifying : 1:nginx-all-modules-1.12.2-3.el7.noarch 1/22 Verifying : fontconfig-2.13.0-4.3.el7.x86_64 2/22 Verifying : 1:nginx-1.12.2-3.el7.x86_64 3/22 Verifying : 1:nginx-mod-mail-1.12.2-3.el7.x86_64 4/22 Verifying : 1:nginx-mod-stream-1.12.2-3.el7.x86_64 5/22 Verifying : fontpackages-filesystem-1.44-8.el7.noarch 6/22 Verifying : libjpeg-turbo-1.2.90-6.el7.x86_64 7/22 Verifying : 1:nginx-mod-http-xslt-filter-1.12.2-3.el7.x86_64 8/22 Verifying : 1:nginx-mod-http-image-filter-1.12.2-3.el7.x86_64 9/22 Verifying : dejavu-fonts-common-2.33-6.el7.noarch 10/22 Verifying : libxcb-1.13-1.el7.x86_64 11/22 Verifying : libXpm-3.5.12-1.el7.x86_64 12/22 Verifying : 1:nginx-mod-http-perl-1.12.2-3.el7.x86_64 13/22 Verifying : libxslt-1.1.28-5.el7.x86_64 14/22 Verifying : libX11-1.6.5-2.el7.x86_64 15/22 Verifying : dejavu-sans-fonts-2.33-6.el7.noarch 16/22 Verifying : gd-2.0.35-26.el7.x86_64 17/22 Verifying : 1:nginx-mod-http-geoip-1.12.2-3.el7.x86_64 18/22 Verifying : libXau-1.0.8-2.1.el7.x86_64 19/22 Verifying : gperftools-libs-2.6.1-1.el7.x86_64 20/22 Verifying : libX11-common-1.6.5-2.el7.noarch 21/22 Verifying : 1:nginx-filesystem-1.12.2-3.el7.noarch 22/22 Installed: nginx.x86_64 1:1.12.2-3.el7 Dependency Installed: dejavu-fonts-common.noarch 0:2.33-6.el7 dejavu-sans-fonts.noarch 0:2.33-6.el7 fontconfig.x86_64 0:2.13.0-4.3.el7 fontpackages-filesystem.noarch 0:1.44-8.el7 gd.x86_64 0:2.0.35-26.el7 gperftools-libs.x86_64 0:2.6.1-1.el7 libX11.x86_64 0:1.6.5-2.el7 libX11-common.noarch 0:1.6.5-2.el7 libXau.x86_64 0:1.0.8-2.1.el7 libXpm.x86_64 0:3.5.12-1.el7 libjpeg-turbo.x86_64 0:1.2.90-6.el7 libxcb.x86_64 0:1.13-1.el7 libxslt.x86_64 0:1.1.28-5.el7 nginx-all-modules.noarch 1:1.12.2-3.el7 nginx-filesystem.noarch 1:1.12.2-3.el7 nginx-mod-http-geoip.x86_64 1:1.12.2-3.el7 nginx-mod-http-image-filter.x86_64 1:1.12.2-3.el7 nginx-mod-http-perl.x86_64 1:1.12.2-3.el7 nginx-mod-http-xslt-filter.x86_64 1:1.12.2-3.el7 nginx-mod-mail.x86_64 1:1.12.2-3.el7 nginx-mod-stream.x86_64 1:1.12.2-3.el7 Complete!
需要注意的是，在两台nginx服务器上需要将防火墙关闭，否则lvs+keepalived的两台机器就无法将请求发送到两台nginx服务器上来：
[root@nginx-1 ~]# systemctl disable firewalld.service
查看两台负载均衡机器是否支持lvs：
# 如果看到如下结果，则说明是支持的 [root@lvskeepalived-1 ~]# sudo lsmod |grep ip_vs ip_vs 145497 0 nf_conntrack 137239 1 ip_vs libcrc32c 12644 3 xfs,ip_vs,nf_conntrack
如果上述命令没有任何结果，则执行sudo ipvsadm命令启动ipvs之后，再通过上述命令进行查看即可。启动ipvs之后，我们就可以在/etc/keepalived/目录下编辑keepalived.conf文件，我们以192.168.122.248机器作为master机器，master节点配置如下：
# Global Configuration global_defs { lvs_id director1 # 指定lvs的id } # VRRP Configuration vrrp_instance LVS { state MASTER # 指定当前节点为master节点 interface eth0 # 这里是网卡的名称，通过ifconfig或者ip addr可以查看 virtual_router_id 51 # 这里指定的是虚拟路由id，master节点和backup节点需要指定一样的 priority 151 # 指定了当前节点的优先级，数值越大优先级越高，master节点要高于backup节点 advert_int 1 # 指定发送VRRP通告的间隔，单位是秒 authentication { auth_type PASS # 鉴权，默认通过 auth_pass 123456 # 鉴权访问密码 } virtual_ipaddress { 192.168.122.120 # 指定了虚拟ip } } # Virtual Server Configuration - for www server # 后台真实主机的配置 virtual_server 192.168.122.120 80 { delay_loop 1 # 健康检查的时间间隔 lb_algo rr # 负载均衡策略，这里是轮询 lb_kind DR # 调度器类型，这里是DR persistence_time 1 # 指定了持续将请求打到同一台真实主机的时间长度 protocol TCP # 指定了访问后台真实主机的协议类型 # Real Server 1 configuration # 指定了真实主机1的ip和端口 real_server 192.168.122.110 80 { weight 1 # 指定了当前主机的权重 TCP_CHECK { connection_timeout 10 # 指定了进行心跳检查的超时时间 nb_get_retry 3 # 指定了心跳超时之后的重复次数 delay_before_retry 3 # 指定了在尝试之前延迟多长时间 } } # Real Server 2 Configuration real_server 192.168.122.167 80 { weight 1 # 指定了当前主机的权重 TCP_CHECK { connection_timeout 10 # 指定了进行心跳检查的超时时间 nb_get_retry 3 # 指定了心跳超时之后的重复次数 delay_before_retry 3 # 指定了在尝试之前延迟多长时间 } } }
上面是master节点上keepalived的配置，对于backup节点，其配置与master几乎是一致的，只是其state和priority参数不同。如下是backup节点的完整配置：
# Global Configuration global_defs { lvs_id director2 # 指定lvs的id } # VRRP Configuration vrrp_instance LVS { state BACKUP # 指定当前节点为master节点 interface eth0 # 这里的ens33是网卡的名称，通过ifconfig或者ip addr可以查看 virtual_router_id 51 # 这里指定的是虚拟路由id，master节点和backup节点需要指定一样的 priority 150 # 指定了当前节点的优先级，数值越大优先级越高，master节点要高于backup节点 advert_int 1 # 指定发送VRRP通告的间隔，单位是秒 authentication { auth_type PASS # 鉴权，默认通过 auth_pass 123456 # 鉴权访问密码 } virtual_ipaddress { 192.168.122.120 # 指定了虚拟ip } } # Virtual Server Configuration - for www server # 后台真实主机的配置 virtual_server 192.168.122.120 80 { delay_loop 1 # 健康检查的时间间隔 lb_algo rr # 负载均衡策略，这里是轮询 lb_kind DR # 调度器类型，这里是DR persistence_time 1 # 指定了持续将请求打到同一台真实主机的时间长度 protocol TCP # 指定了访问后台真实主机的协议类型 # Real Server 1 configuration # 指定了真实主机1的ip和端口 real_server 192.168.122.110 80 { weight 1 # 指定了当前主机的权重 TCP_CHECK { connection_timeout 10 # 指定了进行心跳检查的超时时间 nb_get_retry 3 # 指定了心跳超时之后的重复次数 delay_before_retry 3 # 指定了在尝试之前延迟多长时间 } } # Real Server 2 Configuration real_server 192.168.122.167 80 { weight 1 # 指定了当前主机的权重 TCP_CHECK { connection_timeout 10 # 指定了进行心跳检查的超时时间 nb_get_retry 3 # 指定了心跳超时之后的重复次数 delay_before_retry 3 # 指定了在尝试之前延迟多长时间 } } }
将master和backup配置成完全一样的原因是，在master宕机时，可以根据backup的配置进行服务的无缝切换。
在lvs+keepalived机器配置完成之后，我们下面配置两台应用服务器的nginx配置。这里我们是将nginx作为应用服务器，在其配置文件中配置返回状态码为200，并且会将当前主机的ip返回，如下是其配置：
worker_processes auto; # pid /run/nginx.pid; events { worker_connections 786; } http { server { listen 80; # 这里是直接返回200状态码和一段文本 location / { default_type text/html; return 200 "Hello, Nginx! Server nginx-1\n"; } } }
worker_processes auto; # pid /run/nginx.pid; events { worker_connections 786; } http { server { listen 80; # 这里是直接返回200状态码和一段文本 location / { default_type text/html; return 200 "Hello, Nginx! Server nginx-2\n"; } } }
可以看到，两台机器返回的文本中主机ip是不一样的。nginx配置完成后，可以通过如下命令进行启动：
[root@nginx-1 nginx]# sudo nginx
在启动nginx之后，我们需要配置虚拟ip，这是因为我们使用的lvs调度器是DR模式，前面我们讲到过，这种模式下，对客户端的响应是真实服务器直接返回给客户端的，而真实服务器需要将响应报文中的源ip修改为虚拟ip，这里配置的虚拟ip就是起这个作用的。我们编辑/etc/init.d/lvsrs文件，写入如下内容：
#!/bin/bash ifconfig lo:0 192.168.122.120 netmask 255.255.255.255 broadcast 192.168.122.120 up route add -host 192.168.122.120 dev lo:0 echo "0" > /proc/sys/net/ipv4/ip_forward echo "1" > /proc/sys/net/ipv4/conf/lo/arp_ignore echo "2" > /proc/sys/net/ipv4/conf/lo/arp_announce echo "1" > /proc/sys/net/ipv4/conf/all/arp_ignore echo "2" > /proc/sys/net/ipv4/conf/all/arp_announce exit 0
- lo：表示当前主机真实网卡的名称；
- 172.16.28.120：表示虚拟ip；
编写完成后运行该脚本文件即可。然后将两台lvs+keepalived机器上的keepalived服务启动起来即可：
[root@lvskeepalived-1 keepalived]# service keepalived start Redirecting to /bin/systemctl start keepalived.service [root@lvskeepalived-1 keepalived]# service keepalived status Redirecting to /bin/systemctl status keepalived.service ● keepalived.service - LVS and VRRP High Availability Monitor Loaded: loaded (/usr/lib/systemd/system/keepalived.service; disabled; vendor preset: disabled) Active: active (running) since Tue 2019-08-27 15:01:20 CST; 10s ago Process: 1507 ExecStart=/usr/sbin/keepalived $KEEPALIVED_OPTIONS (code=exited, status=0/SUCCESS) Main PID: 1508 (keepalived) CGroup: /system.slice/keepalived.service ├─1508 /usr/sbin/keepalived -D ├─1509 /usr/sbin/keepalived -D └─1510 /usr/sbin/keepalived -D Aug 27 15:01:23 lvskeepalived-1 Keepalived_vrrp[1510]: Sending gratuitous ARP on eth0 for 192.168.122.120 Aug 27 15:01:23 lvskeepalived-1 Keepalived_vrrp[1510]: Sending gratuitous ARP on eth0 for 192.168.122.120 Aug 27 15:01:23 lvskeepalived-1 Keepalived_vrrp[1510]: Sending gratuitous ARP on eth0 for 192.168.122.120 Aug 27 15:01:23 lvskeepalived-1 Keepalived_vrrp[1510]: Sending gratuitous ARP on eth0 for 192.168.122.120 Aug 27 15:01:28 lvskeepalived-1 Keepalived_vrrp[1510]: Sending gratuitous ARP on eth0 for 192.168.122.120 Aug 27 15:01:28 lvskeepalived-1 Keepalived_vrrp[1510]: VRRP_Instance(LVS) Sending/queueing gratuitous ARPs on eth0 for 192.168.122.120 Aug 27 15:01:28 lvskeepalived-1 Keepalived_vrrp[1510]: Sending gratuitous ARP on eth0 for 192.168.122.120 Aug 27 15:01:28 lvskeepalived-1 Keepalived_vrrp[1510]: Sending gratuitous ARP on eth0 for 192.168.122.120 Aug 27 15:01:28 lvskeepalived-1 Keepalived_vrrp[1510]: Sending gratuitous ARP on eth0 for 192.168.122.120 Aug 27 15:01:28 lvskeepalived-1 Keepalived_vrrp[1510]: Sending gratuitous ARP on eth0 for 192.168.122.120 [root@lvskeepalived-1 keepalived]#
最后可以通过如下命令查看配置的lvs+keepalived的策略：
[root@lvskeepalived-1 keepalived]# ipvsadm -ln IP Virtual Server version 1.2.1 (size=4096) Prot LocalAddress:Port Scheduler Flags -> RemoteAddress:Port Forward Weight ActiveConn InActConn TCP 192.168.122.120:80 rr -> 192.168.122.110:80 Route 1 0 0 -> 192.168.122.167:80 Route 1 0 0
2.2 集群测试
根据上述步骤，我们配置完成了一个lvs+keepalived+nginx的集群。在浏览器中，我们可以访问http://192.168.122.120即可看到如下响应：
Hello, Nginx! Server nginx-1
多次刷新浏览器之后，可以看到浏览器中显示的文本切换如下，这是因为lvs的负载均衡策略产生的：
[root@test2 ~]# curl 192.168.122.120 Hello, Nginx! Server nginx-1 [root@test2 ~]# curl 192.168.122.120 Hello, Nginx! Server nginx-2 [root@test2 ~]# curl 192.168.122.120 Hello, Nginx! Server nginx-1 [root@test2 ~]# curl 192.168.122.120 Hello, Nginx! Server nginx-2 [root@test2 ~]# curl 192.168.122.120 Hello, Nginx! Server nginx-1 [root@test2 ~]# curl 192.168.122.120 Hello, Nginx! Server nginx-2 [root@test2 ~]# curl 192.168.122.120 Hello, Nginx! Server nginx-1 [root@test2 ~]# curl 192.168.122.120 Hello, Nginx! Server nginx-2 [root@test2 ~]# curl 192.168.122.120 Hello, Nginx! Server nginx-1 [root@test2 ~]# curl 192.168.122.120 Hello, Nginx! Server nginx-2 [root@test2 ~]# curl 192.168.122.120 Hello, Nginx! Server nginx-1 [root@test2 ~]# curl 192.168.122.120 Hello, Nginx! Server nginx-2 [root@test2 ~]# curl 192.168.122.120 Hello, Nginx! Server nginx-1 [root@test2 ~]# curl 192.168.122.120 Hello, Nginx! Server nginx-2 [root@test2 ~]# curl 192.168.122.120 Hello, Nginx! Server nginx-1 [root@test2 ~]# curl 192.168.122.120 Hello, Nginx! Server nginx-2 [root@test2 ~]# curl 192.168.122.120 Hello, Nginx! Server nginx-1 [root@test2 ~]# curl 192.168.122.120 Hello, Nginx! Server nginx-2 [root@test2 ~]# curl 192.168.122.120 Hello, Nginx! Server nginx-1 [root@test2 ~]# curl 192.168.122.120 Hello, Nginx! Server nginx-2 [root@test2 ~]# curl 192.168.122.120 Hello, Nginx! Server nginx-1 [root@test2 ~]# curl 192.168.122.120 Hello, Nginx! Server nginx-2 [root@test2 ~]# curl 192.168.122.120 Hello, Nginx! Server nginx-1 [root@test2 ~]# curl 192.168.122.120 Hello, Nginx! Server nginx-2 [root@test2 ~]# curl 192.168.122.120 Hello, Nginx! Server nginx-1 [root@test2 ~]# curl 192.168.122.120 Hello, Nginx! Server nginx-2 [root@test2 ~]# curl 192.168.122.120 Hello, Nginx! Server nginx-1 [root@test2 ~]# curl 192.168.122.120 Hello, Nginx! Server nginx-2 You have new mail in /var/spool/mail/root [root@test2 ~]# curl 192.168.122.120 Hello, Nginx! Server nginx-1 [root@test2 ~]# curl 192.168.122.120 Hello, Nginx! Server nginx-2 [root@test2 ~]# curl 192.168.122.120 Hello, Nginx! Server nginx-1 [root@test2 ~]# curl 192.168.122.120 Hello, Nginx! Server nginx-2 [root@test2 ~]# curl 192.168.122.120 Hello, Nginx! Server nginx-1 [root@test2 ~]# curl 192.168.122.120 Hello, Nginx! Server nginx-2 [root@test2 ~]# curl 192.168.122.120 Hello, Nginx! Server nginx-1 [root@test2 ~]# curl 192.168.122.120 Hello, Nginx! Server nginx-2 [root@test2 ~]# curl 192.168.122.120 Hello, Nginx! Server nginx-1 [root@test2 ~]# curl 192.168.122.120 Hello, Nginx! Server nginx-2 [root@test2 ~]# curl 192.168.122.120 Hello, Nginx! Server nginx-1 [root@test2 ~]# curl 192.168.122.120 Hello, Nginx! Server nginx-2 [root@test2 ~]# curl 192.168.122.120 Hello, Nginx! Server nginx-1 [root@test2 ~]# curl 192.168.122.120 Hello, Nginx! Server nginx-2 [root@test2 ~]# curl 192.168.122.120 Hello, Nginx! Server nginx-1 [root@test2 ~]#
原文链接：https://www.cnblogs.com/jiangwenhui/p/11418662.html
本文来源 互联网收集，文章内容系作者个人观点，不代表 本站 对观点赞同或支持。如需转载，请注明文章来源,如您发现有涉嫌抄袭侵权的内容，请联系本站核实处理。
© 版权声明
文章版权归作者所有，未经允许请勿转载。
相关文章
B站导航 ，集网址、资源、资讯于一体的 综合网址导航站，简约优雅的设计风格，全面的前端用户功能，简单的模块化配置，欢迎您的体验