k8s怎么设置集群（k8s二进制安装-etcd安装）

开始是一个master，两个node节点，后面再扩。

使用centos7系统，前面配置host，免密登录等基础操作省略。。。

集群中，etcd之间，k8s的api-server之间，https通信都需要证书。所以做一个私有的CA认证，cfssl是一个开源的证书管理工具， 使用json文件生成证书， 相比openssl更方便使用，这里用master节点

wget https://pkg.cfssl.org/R1.2/cfssl_linux-amd64 wget https://pkg.cfssl.org/R1.2/cfssljson_linux-amd64 wget https://pkg.cfssl.org/R1.2/cfssl-certinfo_linux-amd64 chmod x cfssl_linux-amd64 cfssljson_linux-amd64 cfssl-certinfo_linux-amd64 mv cfssl_linux-amd64 /usr/local/bin/cfssl mv cfssljson_linux-amd64 /usr/local/bin/cfssljson mv cfssl-certinfo_linux-amd64 /usr/local/bin/cfssl-certinfo

1.自签名证书颁发机构(CA)

mkdir /root/k8sbinary/TLS/{etcd,k8s} -p cd TLS/etcd/

ca-config.json和ca-csr.json这两个文件就是CA机构，下面用这两个文件组成的CA机构，为不同的域名颁发CA证书。

[root@FNSHB109 etcd]# cat ca-config.json { "signing": {表示该证书可用于签名其它证书；生成的 ca.pem 证书中CA=TRUE "default": { "expiry": "87600h" }, "profiles": { "www": { "expiry": "87600h", "usages": [ "signing", "key encipherment", "server auth",表示client可以用该 CA 对server提供的证书进行验证； "client auth"表示server可以用该CA对client提供的证书进行验证； ] } } } } [root@FNSHB109 etcd]# cat ca-csr.json { "CN": "etcd CA",# Common Name，kube-apiserver 从证书中提取该字段作为请求的用户名 (User Name) "key": { "algo": "rsa", "size": 2048 }, "names": [ { "C": "CN",# Country， 国家 "L": "Beijing",# Locality，地区，城市 "ST": "Beijing"State，州，省 } ] }

生成证书，initca代表用ca-csr这个请求文件初始化一个ca，初始化创建CA认证中心，将会生成证书的请求文件（ca.csr），证书对应的key（ca-key.pem）和证书本身（ca.pem）。

cfssl gencert -initca ca-csr.json | cfssljson -bare ca –

使用自签CA签发Etcd HTTPS证书，下面需要提供者，提供一个etcd-csr.json文件，代表为哪个域名颁发证书。

[root@FNSHB109 etcd]# cat etcd-csr.json { "CN": "etcd", "hosts": [ "135.251.205.109", "135.251.206.138", "135.251.206.137" ], "key": {#加密算法和它的长度 "algo": "rsa", "size": 2048 }, "names": [ { "C": "CN", "L": "BeiJing", "ST": "BeiJing" } ] }

指定ca是谁，ca的key是谁，配置文件还是ca-config.json，-profile=www是定义在config文件里面的，生成以etcd开头的证书。

cfssl gencert -ca=ca.pem -ca-key=ca-key.pem -config=ca-config.json -profile=www etcd-csr.json |cfssljson -bare etcd

下面这四个pem文件后面都会用到。

[root@FNSHB109 etcd]# ls *pem

ca-key.pem ca.pem etcd-key.pem etcd.pem

把四个证书分别放到新建的ssl文件中：

mkdir -p /etc/etcd/ssl mkdir -p /var/lib/etcd/default.etcd cd /root/k8sbinary/TLS/etcd cp ca*.pem /etc/etcd/ssl cp etcd*.pem /etc/etcd/ssl scp ca*.pem node1:/etc/etcd/ssl scp ca*.pem node2:/etc/etcd/ssl scp etcd*.pem node2:/etc/etcd/ssl scp etcd*.pem node1:/etc/etcd/ssl

wget https://github.com/etcd-io/etcd/releases/download/v3.5.2/etcd-v3.5.2-linux-amd64.tar.gz -e "https_proxy=http://135.245.192.7:8000" tar xf etcd-v3.5.2-linux-amd64.tar.gz cp etcd-v3.5.2-linux-amd64/etcd* /usr/local/bin scp etcd-v3.5.2-linux-amd64/etcd* node1:/usr/local/bin scp etcd-v3.5.2-linux-amd64/etcd* node1:/usr/local/bin

[root@FNSHB109 ~]# cat /etc/etcd/etcd.conf #[Member] ETCD_NAME="etcd1" ETCD_DATA_DIR="/var/lib/etcd/default.etcd" ETCD_LISTEN_PEER_URLS="https://135.251.205.109:2380" ETCD_LISTEN_CLIENT_URLS="https://135.251.205.109:2379" ETCD_TRUST_CA_FILE="/etc/etcd /ssl/ca.pem" ETCD_TRUST_CERT_FILE="/etc/etcd/ssl/etcd.pem" ETCD_TRUST_CERT_KEY="/etc/ etcd/ssl/etcd-key.pem" #[Clustering] ETCD_INITIAL_ADVERTISE_PEER_URLS="https://135.251.205.109:2380" ETCD_ADVERTISE_CLIENT_URLS="https://135.251.205.109:2379" ETCD_INITIAL_CLUSTER="etcd1=https://135.251.205.109:2380,etcd2=https://135.251.206.138:2380,etcd2=https://135.251.206.137:2380" ETCD_INITIAL_CLUSTER_TOKEN="etcd-cluster" ETCD_INITIAL_CLUSTER_STATE="new" [root@FNSHB109 ~]# cat /etc/systemd/system/etcd.service [Unit] Description=Etcd Service Documentation=https://coreos.com/etcd/docs/latest/ After=network.target After=network-online.target Wants=network-online.target [Service] Type=notify WorkingDirectory=/var/lib/etcd/ EnvironmentFile=-/etc/etcd/etcd.conf ExecStart=/usr/local/bin/etcd --cert-file=/etc/etcd/ssl/etcd.pem --peer-cert-file=/etc/etcd/ssl/etcd.pem --key-file=/etc/etcd/ssl/etcd-key.pem --peer-key-file=/etc/etcd/ssl/etcd-key.pem --trusted-ca-file=/etc/etcd/ssl/ca.pem --peer-trusted-ca-file=/etc/etcd/ssl/ca.pem --peer-client-cert-auth --client-cert-auth --logger=zap Restart=on-failure RestartSec=10 LimitNOFILE=65536 [Install] WantedBy=multi-user.target

scp /etc/etcd/etcd.conf node1:/etc/etcd

scp /etc/etcd/etcd.conf node2:/etc/etcd

scp /etc/systemd/system/etcd.service node1:/etc/systemd/system/etcd.service

scp /etc/systemd/system/etcd.service node2:/etc/systemd/system/etcd.service

ETCDCTL_API=3 etcdctl --write-out=table --cacert=/etc/etcd/ssl/ca.pem --cert=/etc/etcd/ssl/etcd.pem --key=/etc/etcd/ssl/etcd-key.pem --endpoints="https://135.251.205.109:2379,https://135.251.206.138:2379,https://135.251.206.137:2379" endpoint health