Once you start to manage Kubernetes, you may encounter difficulties when upgrading Kubernetes clusters. Due to the fact that the Kubernetes project is very aggressive, there's a new release around every three months, such as version 1.11.0 (released on June 27th 2018), 1.12.0 (released on September 27th 2018), and 1.13.0 (released on December 3rd 2018).
GKE also keeps adding new version support in a timely manner. This allows us to upgrade both master and nodes via the gcloud command. You can run the following command to see which Kubernetes version is supported by GKE:
$ gcloud container get-server-config
Fetching server config for us-west1-b
defaultClusterVersion: 1.10.9-gke.5
defaultImageType: COS
validImageTypes:
- COS_CONTAINERD
- COS
- UBUNTU
validMasterVersions:
- 1.11.4-gke.8
- 1.11.3-gke.18
- 1.11.2-gke.20
- 1.11.2-gke.18
- 1.10.9-gke.7
- 1.10.9-gke.5
- 1.10.7-gke.13
- 1.10.7-gke.11
- 1.10.6-gke.13
- 1.10.6-gke.11
- 1.9.7-gke.11
validNodeVersions:
- 1.11.4-gke.8
- 1.11.3-gke.18
- 1.11.2-gke.20
- 1.11.2-gke.18
- 1.11.2-gke.15
- 1.11.2-gke.9
- 1.10.9-gke.7
- 1.10.9-gke.5
- 1.10.9-gke.3
- 1.10.9-gke.0
- 1.10.7-gke.13
- 1.10.7-gke.11
...
So, you can see that the latest supported Kubernetes version is 1.11.4-gke.8 on both master and node at the time of writing. Since the previous example installed is version 1.10.9-gke.5, let's update this to 1.11.4-gke.8. First of all, you need to upgrade master first:
//upgrade master using --master option
$ gcloud container clusters upgrade my-k8s-cluster --zone asia-northeast1-a --cluster-version 1.11.4-gke.8 --master
Master of cluster [my-k8s-cluster] will be upgraded from version
[1.10.9-gke.5] to version [1.11.4-gke.8]. This operation is
long-running and will block other operations on the cluster (including
delete) until it has run to completion.
Do you want to continue (Y/n)? y
Upgrading my-k8s-cluster...done.
Updated [https://container.googleapis.com/v1/projects/devops-with-kubernetes/zones/asia-northeast1-a/clusters/my-k8s-cluster].
This takes around 10 minutes depending on the environment. After that, you can verify MASTER_VERSION via the following command:
//master upgrade has been successfully to done
$ gcloud container clusters list --zone asia-northeast1-a
NAME LOCATION MASTER_VERSION MASTER_IP MACHINE_TYPE NODE_VERSION NUM_NODES STATUS
my-k8s-cluster asia-northeast1-a 1.11.4-gke.8 35.243.78.166 f1-micro 1.10.9-gke.5 * 6 RUNNING
Now you can start to upgrade all nodes to version 1.11.4-gke.8. Because GKE tries to perform a rolling upgrade, it'll perform the following steps on each node, one by one:
- De-register a target node from the cluster
- Delete old VM instances
- Provision a new VM instance
- Set up the node with the 1.11.4-gke.8 version
- Register to master
Therefore, a node upgrade takes much longer than a master upgrade:
//node upgrade (not specify --master)
$ gcloud container clusters upgrade my-k8s-cluster --zone asia-northeast1-a --cluster-version 1.11.4-gke.8
All nodes (6 nodes) of cluster [my-k8s-cluster] will be upgraded from
version [1.10.9-gke.5] to version [1.11.4-gke.8]. This operation is
long-running and will block other operations on the cluster (including delete) until it has run to completion.
Do you want to continue (Y/n)? y
During the rolling upgrade, you can see the node status as follows; this example shows the halfway point of a rolling update. Here, two nodes have upgraded to 1.11.4-gke.8, one node is about to upgrade, and the remaining three nodes are pending:
$ kubectl get nodes
NAME STATUS ROLES AGE VERSION
gke-my-k8s-cluster-default-pool-58e4e9a4-74hc Ready <none> 18m v1.11.4-gke.8
gke-my-k8s-cluster-default-pool-58e4e9a4-8jft Ready <none> 19m v1.11.4-gke.8
gke-my-k8s-cluster-default-pool-f7baf2e1-lk7j Ready <none> 19m v1.10.9-gke.5
gke-my-k8s-cluster-default-pool-f7baf2e1-zktg Ready <none> 19m v1.10.9-gke.5
gke-my-k8s-cluster-default-pool-fa5a04a5-bbj5 Ready,SchedulingDisabled <none> 19m v1.10.9-gke.5
gke-my-k8s-cluster-default-pool-fa5a04a5-d35z Ready <none> 19m v1.10.9-gke.5