近期，公司网络重新规划，导致服务器IP地址变更，笔者根据网上的操作，经历了很多坑，最终将IP更换成功。

一、备份

cp -r ~/.kube ~/.kubebak
cp -r /etc/kubernetes /etc/kubernetesbak
cp -r /etc/ssl/etcd/ssl /etc/ssl/etcd/sslback
cp /etc/etcd.env /etc/etcd.envbak

二、更新host配置

 vim /etc/hosts 

三、更新etcd证书

笔者用的是外部etcd，证书需要单独更新，如果是内部证书请忽略。

1、安装cfssl

wget https://pkg.cfssl.org/R1.2/cfssl_linux-amd64 
wget https://pkg.cfssl.org/R1.2/cfssljson_linux-amd64 
wget https://pkg.cfssl.org/R1.2/cfssl-certinfo_linux-amd64
chmod +x cfssl*

2、获取默认配置

cfssl print-defaults config > ca-config.json
cfssl print-defaults csr > ca-csr.json

3、更改ca-config.json

  {
    "signing": {
        "default": {
            "expiry": "87600h"
        },
        "profiles": {
            "server": {
                "expiry": "87600h",
                "usages": [
                    "signing",
                    "key encipherment",
                    "server auth"
                ]
            },
            "client": {
                "expiry": "87600h",
                "usages": [
                    "signing",
                    "key encipherment",
                    "client auth"
                ]
            },
            "etcd": {
                "expiry": "87600h",
                "usages": [
                    "signing",
                    "key encipherment",
                    "server auth",
                    "client auth"
                ]
            }
        }
    }
}

4、更改ca-csr.json

{
  "CN": "etcd-ca",
  "key": {
    "algo": "rsa",
    "size": 2048
  },
  "names": [
    {
      "C": "CN",
      "ST": "hunan",
      "L": "changsha",
      "O": "etcd",
      "OU": "System"
    }
  ]
}

4、生成ca证书

 cfssl gencert -initca ca-csr.json | cfssljson -bare ca

结果如下:

此时会生成三个文件：ca.csr、ca-key.pem、ca.pem

5、签发证书

创建文件etcd-csr.json,它的内容如下：

{
    "CN": "etcd",
    "hosts": [
      "127.0.0.1",
      "0:0:0:0:0:0:0:1",
      "192.168.3.13",
      "lb.kubesphere.local",
      "kubesphere",
      "localhost",
      "etcd",
      "etcd.kube-system",
      "etcd.kube-system.svc",
      "etcd.kube-system.svc.cluster.local"
    ],
    "key": {
        "algo": "rsa",
        "size": 2048
    },
    "names": [
        {
            "C": "CN",
            "ST": "hunan",
            "L": "changsha",
            "O": "etcd",
            "OU": "System"
        }
    ]
}

执行签发证书命令:

 cfssl gencert -ca=ca.pem -ca-key=ca-key.pem -config=ca-config.json -profile=etcd etcd-csr.json | cfssljson -bare etcd

最终，服务器下将生成以下文件:

etcd.csr、 etcd-key.pem etcd.pem

6、修改etcd证书地址

vim /etc/etcd.env

修改内容如下:

# Environment file for etcd v3.4.13
ETCD_DATA_DIR=/var/lib/etcd
ETCD_ADVERTISE_CLIENT_URLS=https://192.168.3.16:2379
ETCD_INITIAL_ADVERTISE_PEER_URLS=https://192.168.3.16:2380
ETCD_INITIAL_CLUSTER_STATE=existing
ETCD_METRICS=basic
ETCD_LISTEN_CLIENT_URLS=https://192.168.3.16:2379,https://127.0.0.1:2379
ETCD_ELECTION_TIMEOUT=5000
ETCD_HEARTBEAT_INTERVAL=250
ETCD_INITIAL_CLUSTER_TOKEN=k8s_etcd
ETCD_LISTEN_PEER_URLS=https://192.168.3.16:2380
ETCD_NAME=etcd-kubesphere
ETCD_PROXY=off
ETCD_ENABLE_V2=true
ETCD_INITIAL_CLUSTER=etcd-kubesphere=https://192.168.3.16:2380
ETCD_AUTO_COMPACTION_RETENTION=8
ETCD_SNAPSHOT_COUNT=10000

# TLS settings
ETCD_TRUSTED_CA_FILE=/etc/ssl/etcd/ssl/ca.pem
ETCD_CERT_FILE=/etc/ssl/etcd/ssl/etcd.pem
ETCD_KEY_FILE=/etc/ssl/etcd/ssl/etcd-key.pem
ETCD_CLIENT_CERT_AUTH=true

ETCD_PEER_TRUSTED_CA_FILE=/etc/ssl/etcd/ssl/ca.pem
ETCD_PEER_CERT_FILE=/etc/ssl/etcd/ssl/etcd.pem
ETCD_PEER_KEY_FILE=/etc/ssl/etcd/ssl/etcd-key.pem
ETCD_PEER_CLIENT_CERT_AUTH=True

# CLI settings
ETCDCTL_ENDPOINTS=https://127.0.0.1:2379
ETCDCTL_CA_FILE=/etc/ssl/etcd/ssl/ca.pem
ETCDCTL_KEY_FILE=/etc/ssl/etcd/ssl/etcd-key.pem
ETCDCTL_CERT_FILE=/etc/ssl/etcd/ssl/etcd.pem

7、重启etcd服务

systemctl restart etcd
systemctl status etcd 

8、拷贝到k8s 使用的etcd文件

cp etcd-key.pem admin-kubesphere-key.pem
cp etcd-key.pem member-kubesphere-key.pem
cp etcd-key.pem node-kubesphere-key.pem
cp etcd.pem admin-kubesphere.pem
cp etcd.pem member-kubesphere.pem
cp etcd.pem node-kubesphere.pem

四、更新k8s配置信息

1、更改kubeadm-config.yaml

更改里面所有的IP地址

2、更改k8s conf配置

rm -f /etc/kubernetes/*.conf
kubeadm init phase kubeconfig all --config ~/kubeadm.yaml

必须将.conf删除，否则重新生成的命令将失效。下面涉及到删除的也类似

这时会生成以下文件:admin.conf、controller-manager.conf、kubelet.conf、scheduler.conf

3、重新生成K8S组件的POD配置yaml文件

rm -f /etc/kubernetes/manifests/*.yaml
kubeadm init phase control-plane all --config ~/kubeadm.yaml

这时会生成以下文件:kube-apiserver.yaml、
kube-controller-manager.yaml、kube-scheduler.yaml

4、重新生成证书

rm -rf /etc/kubernetes/pki/apiserver* /etc/kubernetes/pki/front-proxy* 
kubeadm init phase certs all --config ~/kubeadm.yaml

5、 替换.kube的配置信息

cp /etc/kubernetes/admin.conf ~/.kube/config

6、 重启k8s服务并检验

systemctl restart kubelet docker
kubectl get node -owide
kubectl get pods -A -owide

此时会发现,k8s节点已经起来，但是有很多pod还是无法启动。需要按照第7、第8步进行修改

7、重新安装k8s相关组件

kubeadm init phase addon all --config ~/kubeadm.yaml

8、修改集群configmap的IP

kubectl edit cm -n kube-system kubeadm-config

kubectl edit cm -n kube-system kube-proxy

kubectl edit cm -n kube-system coredns

kubectl edit cm -n kube-public cluster-info

9、修改k8s启动信息

vim  /etc/systemd/system/kubelet.service.d/10-kubeadm.conf

更换文件里的ip

10、重启k8s服务