kubenetes 集群二进制安装之安装etcd集群

kuberntes 系统使用 etcd 存储集群配置和一些状态信息。如果单一部署，etcd崩溃以后会导致集群异常甚至集群崩溃无法恢复。所以单点部署是否风险的。
这里记录下配置etcd集群，避免单点故障(临时将master02作为集群的etcd3节点加入测试)。

etcd01: 10.1.14.23
etcd02: 10.1.14.24

1，yum 安装etcd

yum install  etcd  -y

2,配置etcd.conf配置文件

[root@etcd01 system]# cat  /etc/etcd/etcd.conf 
# [member]
 ETCD_NAME=etcd1
 ETCD_DATA_DIR="/var/lib/etcd"
 ETCD_LISTEN_PEER_URLS="https://10.1.14.23:2380"
 ETCD_LISTEN_CLIENT_URLS="https://10.1.14.23:2379"

 #[cluster]
 ETCD_INITIAL_ADVERTISE_PEER_URLS="https://10.1.14.23:2380"
 ETCD_INITIAL_CLUSTER_TOKEN="etcd-cluster"
 ETCD_ADVERTISE_CLIENT_URLS="https://10.1.14.23:2379"

说明：
这里要注意集群节点名称，每个服务都需要编辑加入对应etcd集群节点名称

3,配置etcd.service
vim /usr/lib/systemd/system/etcd.service

[Unit]
Description=Etcd Server
After=network.target
After=network-online.target
Wants=network-online.target
Documentation=https://github.com/coreos

[Service]
Type=notify
WorkingDirectory=/var/lib/etcd/
EnvironmentFile=-/etc/etcd/etcd.conf
ExecStart=/usr/bin/etcd \
  --name ${ETCD_NAME} \
  --cert-file=/etc/kubernetes/ssl/kubernetes.pem \
  --key-file=/etc/kubernetes/ssl/kubernetes-key.pem \
  --peer-cert-file=/etc/kubernetes/ssl/kubernetes.pem \
  --peer-key-file=/etc/kubernetes/ssl/kubernetes-key.pem \
  --trusted-ca-file=/etc/kubernetes/ssl/ca.pem \
  --peer-trusted-ca-file=/etc/kubernetes/ssl/ca.pem \
  --initial-advertise-peer-urls ${ETCD_INITIAL_ADVERTISE_PEER_URLS} \
  --listen-peer-urls ${ETCD_LISTEN_PEER_URLS} \
  --listen-client-urls ${ETCD_LISTEN_CLIENT_URLS},http://127.0.0.1:2379 \
  --advertise-client-urls ${ETCD_ADVERTISE_CLIENT_URLS} \
  --initial-cluster-token ${ETCD_INITIAL_CLUSTER_TOKEN} \
  --initial-cluster etcd1=https://10.1.14.23:2380,etcd2=https://10.1.14.24:2380  \
  --initial-cluster-state new \
  --data-dir=${ETCD_DATA_DIR}
Restart=on-failure
RestartSec=5
LimitNOFILE=65536

[Install]
WantedBy=multi-user.target

修改完以后启动etcd服务：

systemctl   daemon-reload
systemctl   enable etcd
systemctl   start  etcd

4，集群验证
（1）验证集群状态

[root@etcd01 system]# etcdctl   --ca-file=/etc/kubernetes/ssl/ca.pem   --cert-file=/etc/kubernetes/ssl/kubernetes.pem   --key-file=/etc/kubernetes/ssl/kubernetes-key.pem   cluster-health
member 511f5710ab727fc6 is healthy: got healthy result from https://10.1.14.24:2379
member ef80ec0164ea7efe is healthy: got healthy result from https://10.1.14.23:2379
cluster is healthy

（2）通过集群节点etcd1设置一个值，到etcd2获取，如果获取正常，则存储也没问题：

[root@etcd01 system]#  etcdctl   --ca-file=/etc/kubernetes/ssl/ca.pem   --cert-file=/etc/kubernetes/ssl/kubernetes.pem   --key-file=/etc/kubernetes/ssl/kubernetes-key.pem  set  a 123456

[root@etcd02 system]# etcdctl   --ca-file=/etc/kubernetes/ssl/ca.pem   --cert-file=/etc/kubernetes/ssl/kubernetes.pem   --key-file=/etc/kubernetes/ssl/kubernetes-key.pem   get a
123456

（3）可以通过etcdctl member list查看谁是当前集群中的leader：

[root@etcd01 system]#  etcdctl   --ca-file=/etc/kubernetes/ssl/ca.pem   --cert-file=/etc/kubernetes/ssl/kubernetes.pem   --key-file=/etc/kubernetes/ssl/kubernetes-key.pem member list
511f5710ab727fc6: name=etcd2 peerURLs=https://10.1.14.24:2380 clientURLs=https://10.1.14.24:2379 isLeader=false
ef80ec0164ea7efe: name=etcd1 peerURLs=https://10.1.14.23:2380 clientURLs=https://10.1.14.23:2379 isLeader=true

验证leader切换，如何发挥集群作用。假设我这里再增加了一台，如果停掉当前leader的节点，其他etcd 集群节点其中一个会变成leader。

[root@etcd02 etcd]# etcdctl   --ca-file=/etc/kubernetes/ssl/ca.pem   --cert-file=/etc/kubernetes/ssl/kubernetes.pem   --key-file=/etc/kubernetes/ssl/kubernetes-key.pem   member  list
511f5710ab727fc6: name=etcd2 peerURLs=https://10.1.14.24:2380 clientURLs=https://10.1.14.24:2379 isLeader=false
e71a8bd23a404b12: name=etcd3 peerURLs=https://10.1.14.22:2380 clientURLs=https://10.1.14.22:2379 isLeader=true
ef80ec0164ea7efe: name=etcd1 peerURLs=https://10.1.14.23:2380 clientURLs=https://10.1.14.23:2379 isLeader=false
[root@etcd02 etcd]# etcdctl   --ca-file=/etc/kubernetes/ssl/ca.pem   --cert-file=/etc/kubernetes/ssl/kubernetes.pem   --key-file=/etc/kubernetes/ssl/kubernetes-key.pem   member  list
511f5710ab727fc6: name=etcd2 peerURLs=https://10.1.14.24:2380 clientURLs=https://10.1.14.24:2379 isLeader=false
e71a8bd23a404b12: name=etcd3 peerURLs=https://10.1.14.22:2380 clientURLs=https://10.1.14.22:2379 isLeader=false
ef80ec0164ea7efe: name=etcd1 peerURLs=https://10.1.14.23:2380 clientURLs=https://10.1.14.23:2379 isLeader=true

5，创建kubenetes 网段，用于后续的flannel请求调用
（1）创建网段

  
etcdctl  \
  --ca-file=/etc/kubernetes/ssl/ca.pem \
  --cert-file=/etc/kubernetes/ssl/kubernetes.pem \
  --key-file=/etc/kubernetes/ssl/kubernetes-key.pem \
  mk /21yunwei/network/config '{"Network":"172.50.0.0/16"}' 

2，查看网络配置

[root@etcd01 etcd]# etcdctl    --ca-file=/etc/kubernetes/ssl/ca.pem   --cert-file=/etc/kubernetes/ssl/kubernetes.pem   --key-file=/etc/kubernetes/ssl/kubernetes-key.pem  ls   -r
/21yunwei
/21yunwei/network
/21yunwei/network/config
[root@etcd01 etcd]# etcdctl    --ca-file=/etc/kubernetes/ssl/ca.pem   --cert-file=/etc/kubernetes/ssl/kubernetes.pem   --key-file=/etc/kubernetes/ssl/kubernetes-key.pem  get /21yunwei/network/config
{"Network":"172.50.0.0/16"}

通过其他etcd集群节点如果可以查看到这个网段，说明网段创建成功。

转载请注明：21运维 » kubenetes 集群二进制安装之安装etcd集群