本教程将演示使用kubeadm将3台master的kubernetes集群从v1.11.1版本升级至v1.12.1版本
下载最新kubernetes镜像（如有梯子可以跳过），若要升级后续版本则将版本号改为对应版本号，worker节点只需kube-proxy

export VERSION=v1.12.1
docker pull mirrorgooglecontainers/kube-apiserver:${VERSION}
docker pull mirrorgooglecontainers/kube-scheduler:${VERSION}
docker pull mirrorgooglecontainers/kube-proxy:${VERSION}
docker pull mirrorgooglecontainers/kube-controller-manager:${VERSION}
docker pull mirrorgooglecontainers/pause-amd64:3.1
docker pull mirrorgooglecontainers/etcd-amd64:3.2.24
docker pull coredns/coredns:1.2.2

docker tag mirrorgooglecontainers/kube-apiserver:${VERSION} k8s.gcr.io/kube-apiserver:${VERSION}
docker tag mirrorgooglecontainers/kube-scheduler:${VERSION} k8s.gcr.io/kube-scheduler:${VERSION}
docker tag mirrorgooglecontainers/kube-proxy:${VERSION} k8s.gcr.io/kube-proxy:${VERSION}
docker tag mirrorgooglecontainers/kube-controller-manager:${VERSION} k8s.gcr.io/kube-controller-manager:${VERSION}
docker tag mirrorgooglecontainers/pause-amd64:3.1 k8s.gcr.io/pause:3.1
docker tag mirrorgooglecontainers/etcd-amd64:3.2.24 k8s.gcr.io/etcd:3.2.24
docker tag coredns/coredns:1.2.2 k8s.gcr.io/coredns:1.2.2
docker images | grep mirrorgooglecontainers | awk '{print "docker rmi "$1":"$2}' | sh
docker rmi coredns/coredns:1.2.2

安装新版kubeadm

export VERSION=1.12.1
yum install -y kubeadm-${VERSION}

在第一个master节点上执行如下命令

kubeadm upgrade plan

会得到如下结果

[upgrade/versions] Latest version in the v1.11 series: v1.12.1

Components that must be upgraded manually after you have upgraded the control plane with 'kubeadm upgrade apply':
COMPONENT   CURRENT       AVAILABLE
Kubelet     6 x v1.11.1   v1.12.1

Upgrade to the latest stable version:

COMPONENT            CURRENT   AVAILABLE
API Server           v1.11.1   v1.12.1
Controller Manager   v1.11.1   v1.12.1
Scheduler            v1.11.1   v1.12.1
Kube Proxy           v1.11.1   v1.12.1
CoreDNS              1.1.3     1.2.2
Etcd                 3.2.18    3.2.24

You can now apply the upgrade by executing the following command:

	kubeadm upgrade apply v1.12.1

由于k8s高可用集群里的master节点名称使用的是浮动IP，在升级时需要先改为实际IP

kubectl get configmap -n kube-system kubeadm-config -o yaml >kubeadm-config-cm.yaml

修改kubeadm-config-cm.yaml文件，将以下字段信息改为当前节点的IP地址
api.advertiseAddress
etcd.local.extraArgs.advertise-client-urls
etcd.local.extraArgs.initial-advertise-peer-urls
etcd.local.extraArgs.listen-client-urls
etcd.local.extraArgs.listen-peer-urls
在etcd.local.extraArgs增加一个参数：initial-cluster-state: existing
将etcd.local.extraArgs.initial-cluster改为etcd集群地址信息，如

initial-cluster: k8s1=https://192.168.1.101:2380,k8s2=https://192.168.1.102:2380,k8s3=https://192.168.1.103:2380

修改完成后应用新配置文件

kubectl apply -f kubeadm-config-cm.yaml --force

执行以下命令开始升级

kubeadm upgrade apply v$VERSION

看到以下提示表示升级完成

[upgrade/successful] SUCCESS! Your cluster was upgraded to "v1.12.1". Enjoy!

主节点升级完成后继续升级剩余节点

kubectl get configmap -n kube-system kubeadm-config -o yaml >kubeadm-config-cm.yaml

修改kubeadm-config-cm.yaml文件，将以下字段信息改为当前节点的IP地址
etcd.local.extraArgs.advertise-client-urls
etcd.local.extraArgs.initial-advertise-peer-urls
etcd.local.extraArgs.listen-client-urls
etcd.local.extraArgs.listen-peer-urls
将etcd.local.extraArgs.name改为当前节点主机名
在ClusterStatus.apiEndpoints中加入当前节点信息，如：

  ClusterStatus: |
    apiEndpoints:
      k8s1.test.local:
        advertiseAddress: 192.168.1.101
        bindPort: 6443
      k8s2.test.local:
        advertiseAddress: 192.168.1.102
        bindPort: 6443

修改完成后应用新配置文件

kubectl apply -f kubeadm-config-cm.yaml --force

再为当前节点创建cri-socket注释

kubectl annotate node <nodename> kubeadm.alpha.kubernetes.io/cri-socket=/var/run/dockershim.sock

执行以下命令开始升级

kubeadm upgrade apply v$VERSION

Master节点全部升级完成后，手工安装新版kubelet、kubectl

export VERSION=1.12.1
yum install -y kubelet-${VERSION} kubectl-${VERSION}
systemctl daemon-reload
systemctl restart kubelet