Confd实现Nginx配置文件自动管理

Confd是一个轻量级的配置管理工具。通过查询Etcd，结合配置模板引擎，保持本地配置最新，同时具备定期探测机制，配置变更自动reload。其后端支持的数据类型有：etcd、consul、vault、environment variables、redis、zookeeper、dynamodb、stackengine、rancher。不过一般使用Confd和etcd的配合使用比较多。

前端服务器：
服务器IP 主机名 安装组件 备注
192.168.27.211 Client1 etcd+confd+nginx+keepalived 192.168.27.110（Vip）
(http://nginx.jerry.com)
192.168.27.212 Client2 etcd+confd+nginx+keepalived
192.168.27.213 Client3 etcd+confd+nginx+keepalived
192.168.27.210 master ansible 堡垒机

后端服务器（web站）：
服务器IP 功能
192.168.26.210 web1
192.168.26.211 web2
192.168.26.212 web3

安装etcd集群确保正常略（ansible k8s -m shell -a'etcdctl endpoint health'）。

后端web服务器安装配置略（注意VIP域名映射关系）：
简略介绍安装keepalived安装配置：
[root@client1 ~]# yum install keepalived -y


192.168.27.211：
[root@client1 keepalived]# cat keepalived.conf
! Configuration File for keepalived

global_defs {
router_id nginx1
}
vrrp_script chk_http_port {
script "/etc/keepalived/chk_nginx.sh"
interval 2
weight 2

}
vrrp_instance VI_1 {
state MASTER
interface ens160
virtual_router_id 20
priority 100
advert_int 1
authentication {
auth_type PASS
auth_pass jerry520
}
track_script {
chk_http_port
}
virtual_ipaddress {
192.168.27.110/22
}
}

192.168.27.212：
! Configuration File for keepalived

global_defs {
router_id nginx2
}
vrrp_script chk_http_port {
script "/etc/keepalived/chk_nginx.sh"
interval 2
weight 2

}
vrrp_instance VI_1 {
state BACKUP
interface ens160
virtual_router_id 20
priority 99
advert_int 1
authentication {
auth_type PASS
auth_pass jerry520
}
track_script {
chk_http_port
}
virtual_ipaddress {
192.168.27.110/22
}
}
192.168.27.213:
! Configuration File for keepalived

global_defs {
router_id nginx3
}
vrrp_script chk_http_port {
script "/etc/keepalived/chk_nginx.sh"
interval 2
weight 2

}
vrrp_instance VI_1 {
state BACKUP
interface ens160
virtual_router_id 20
priority 98
advert_int 1
authentication {
auth_type PASS
auth_pass jerry520
}
track_script {
chk_http_port
}
virtual_ipaddress {
192.168.27.110/22
}
}
Nginx检测脚本：三台服务器上都需要配置（一样的）vim /etc/keepalived/chk_nginx.sh
[root@client1 keepalived]# cat chk_nginx.sh
#!/bin/bash
A=ps -C nginx --no-header |wc -l
if [ $A -eq 0 ];then
echo 'nginx server is died'
/etc/init.d/keepalived stop
fi

nginx安装：
yum install nginx -y
nginx.conf配置文件：三台服务器保持一样 vim /etc/nginx/nginx.conf
user nginx ;
worker_processes 1;
events {
worker_connections 1024;
}
http {
include mime.types;
default_type application/octet-stream;
sendfile on;
keepalive_timeout 65;
upstream nginx.jerry.com {
server 192.168.26.210:80;
server 192.168.26.211:80;
server 192.168.26.212:80;

}
server {
listen 80;
server_name nginx.jerry.com;
location / {
root html;
index index.html index.htm;
proxy_pass http://nginx.jerry.com;
}
error_page 500 502 503 504 /50x.html;
location = /50x.html {
root html;
}
}
}
分别安装配置好KEEPALIved和nginx（转发器）并启动运行观察效果：

Confd安装配置：

[root@master etc]# ansible k8s -m copy -a"src=/etc/confd dest=/etc/"

[root@master bin]# ansible k8s -m copy -a"src=/usr/bin/confd dest=/usr/bin/confd"
[root@master bin]# ansible k8s -m shell -a"cd /usr/bin;chmod +x confd "
[root@master conf.d]# ansible k8s -m shell -a'ls /usr/bin/confd -l'



创建配置目录
mkdir -p /etc/confd/{conf.d,templates}
conf.d # 资源模板，下面文件必须以toml后缀
templates # 配置文件模板，下面文件必须以tmpl后缀

创建confd配置文件:
[root@client1 confd]# cat conf.d/sync_nginx.toml
[template]
prefix = "/nginx/www"
src = "nginx.conf.tmpl"
dest = "/etc/nginx/conf.d/mynginx.conf"
owner = "nginx"
mode = "0644"
keys = [
"/server_name",
"/upstream",
]
reload_cmd = "/usr/sbin/nginx -s reload"
创建模板文件：
upstream {{getv "/server_name"}}.jerry.com {
{{ range getvs "/upstream/*"}}
server {{.}};
{{end}}
}
server {
listen 80;
server_name {{getv "/server_name"}}.jerry.com;
location / {
root html;
index index.html index.htm;
proxy_pass http://{{getv "/server_name"}}.jerry.com;
}
error_page 500 502 503 504 /50x.html;
location = /50x.html {
root html;
}
}

[root@client1 templates]# confd -watch -backend="etcdv3" -node http://192.168.27.211:2379

[root@client1 conf.d]# etcdctl put /nginx/www/upstream/serverweb1 "192.168.26.210"
[root@client1 conf.d]# etcdctl put /nginx/www/upstream/serverweb2 "192.168.26.211"
[root@client1 conf.d]# etcdctl put /nginx/www/upstream/serverweb3 "192.168.26.212"
[root@client1 conf.d]# etcdctl put /nginx/www/server_name "nginx"
观察集群中每一个结点NGINX反向代理配置文件变化：
27.212:

27.213:

27.211:

我们再来一次观察下变化，这次通过27.212更换键值/nginx/www/server_name 我们把值改为httpd(原来为nginx)观察27.211上NGINX配置文件是否更改变化。
[root@client2 conf.d]# etcdctl put /nginx/www/server_name "httpd"

配置文件瞬间更改：

测试：假如要增删改后端服务器(把后端WEB服务192.168.26.210进行删除操作).
[root@client2 conf.d]# etcdctl del /nginx/www/upstream/serverweb1

配置文件中已经将后端服务器192.168.26.210无感知地移出并重新加载配置

访问也变了

用公网IP：192.168.27.100(vip)访问（负载均衡采用轮询）:



通过域名（http://nginx.jerry.com）访问后端站点（负载均衡采用轮询）：
记得做公网域名解析或更换本地HOST文件：

参考文献：https://github.com/kelseyhightower/confd/blob/master/docs/quick-start-guide.md
https://github.com/kelseyhightower/confd/blob/master/docs/template-resources.md
https://github.com/kelseyhightower/confd/blob/master/docs/templates.md