k8s高可用架构

地址sealos, 让kubernetes高可用不再需要keepalived haproxy和ansible,sealyun定制超级版kubeadm通过ipvs代理多个master，优雅解决k8s高可用问题。

ip      role10.103.97.200   master010.103.97.201   master010.103.97.202   master010.103.97.2     virtulIP

apiserver.cluster.local apiserver解析名

在每个节点上初始化

tar zxvf kube1.14.0.tar.gz && cd kube/shell && sh init.sh

用下载的kubeadm替换掉包内的kubeadm:

cp kubeadm /usr/bin/kubeadm

cat kubeadm-config.yaml :

apiVersion: kubeadm.k8s.io/v1beta1kind: ClusterConfigurationkubernetesVersion: v1.14.0controlPlaneEndpoint: "apiserver.cluster.local:6443" # 使用解析名去访问APIserverapiServer:        certSANs:        - 127.0.0.1        - apiserver.cluster.local        - 172.20.241.205        - 172.20.241.206        - 172.20.241.207        - 172.20.241.208        - 10.103.97.2          # 虚拟IP等都加入到证书中---apiVersion: kubeproxy.config.k8s.io/v1alpha1kind: KubeProxyConfigurationmode: "ipvs"ipvs:        excludeCIDRs:         - "10.103.97.2/32" # 如果不加这个k8s会定时清理用户创建的IPVS规则，导致代理失败

在 master0 10.103.97.200 上

echo "10.103.97.200 apiserver.cluster.local" >> /etc/hostskubeadm init --config=kubeadm-config.yaml --experimental-upload-certs  mkdir ~/.kube && cp /etc/kubernetes/admin.conf ~/.kube/configkubectl apply -f https://docs.projectcalico.org/v3.6/getting-started/kubernetes/installation/hosted/kubernetes-datastore/calico-networking/1.7/calico.yaml # 安装calico

执行完会输出一些日志，里面包含join需要用的 命令

在 master1 10.103.97.201 上

# 注意，在安装之前解析到master0, 安装成功后解析改成自己,因为kubelet kube-proxy配置的都是这个解析名,如果不改解析master0宕机整个集群就不可用了echo "10.103.97.200 apiserver.cluster.local" >> /etc/hosts kubeadm join 10.103.97.200:6443 --token 9vr73a.a8uxyaju799qwdjv \    --discovery-token-ca-cert-hash sha256:7c2e69131a36ae2a042a339b33381c6d0d43887e2de83720eff5359e26aec866 \    --experimental-control-plane \    --certificate-key f8902e114ef118304e561c3ecd4d0b543adc226b7a07f675f56564185ffe0c07 sed "s/10.103.97.200/10.103.97.201/g" -i /etc/hosts  # 解析改也自己本机地址

在 master2 10.103.97.202 上，同master1

echo "10.103.97.200 apiserver.cluster.local" >> /etc/hostskubeadm join 10.103.97.200:6443 --token 9vr73a.a8uxyaju799qwdjv \    --discovery-token-ca-cert-hash sha256:7c2e69131a36ae2a042a339b33381c6d0d43887e2de83720eff5359e26aec866 \    --experimental-control-plane \    --certificate-key f8902e114ef118304e561c3ecd4d0b543adc226b7a07f675f56564185ffe0c07  sed "s/10.103.97.200/10.103.97.201/g" -i /etc/hosts

通过虚拟IP join到master上, 这个 命令 会在node节点上创建一条ipvs规则，virturl server就是虚拟IP， realserver就是三个master。 然后再以一个static pod起一个守护进程守护这些规则，一旦哪个apiserver不可访问了就清除realserver, apiserver通了之后再次添加回来

echo "10.103.97.2 apiserver.cluster.local" >> /etc/hosts   # using vipkubeadm join 10.103.97.2:6443 --token 9vr73a.a8uxyaju799qwdjv \    --master 10.103.97.200:6443 \    --master 10.103.97.201:6443 \    --master 10.103.97.202:6443 \    --discovery-token-ca-cert-hash sha256:7c2e69131a36ae2a042a339b33381c6d0d43887e2de83720eff5359e26aec866 Architecture

  +----------+                       +---------------+  virturl server: 10.103.97.2:6443  | mater0   |< ----------------------| ipvs nodes    |    real servers:  +----------+                      |+---------------+            10.103.97.200:6443                                    |                             10.103.97.201:6443  +----------+                      |                             10.103.97.202:6443  | mater1   |<---------------------+  +----------+                      |                                    |  +----------+                      |  | mater2   |<---------------------+  +----------+

集群每个node节点都会创建一条ipvs规则用于代理所有的master节点。类似kube-proxy的ipvs实现。

然后起一个守护进程就健康检查

apiservers /etc/kubernetes/manifests/sealyun-lvscare.yaml