Creating clusters in the cloud (GCP, AWS, Azure)

Creating clusters locally is fun, and important during development and when trying to troubleshoot problems locally. But, in the end, Kubernetes is designed for cloud-native applications (applications that run in the cloud). Kubernetes doesn't want to be aware of individual cloud environments because that doesn't scale. Instead, Kubernetes has the concept of a cloud-provider interface. Every cloud provider can implement this interface and then host Kubernetes. Note that, as of version 1.5, Kubernetes still maintains implementations for many cloud providers in its tree, but in the future, they will be refactored out.

The cloud-provider interface

The cloud-provider interface is a collection of Go data types and interfaces. It is defined in a file called cloud.go, available at http://bit.ly/2fq4NbW. Here is the main interface:

type Interface interface {
  LoadBalancer() (LoadBalancer, bool)
  Instances() (Instances, bool)
  Zones() (Zones, bool)
  Clusters() (Clusters, bool)
  Routes() (Routes, bool)
  ProviderName() string
  ScrubDNS(nameservers, searches []string) (nsOut, srchOut []string)
}

This is very clear. Kubernetes operates in terms of instances, Zones, Clusters, and Routes, and also requires access to a load balancer and provider name. The ScrubDNS() is the only low-level method. All the main methods return yet other interfaces.

For example, the Clusters interface is very simple:

type Clusters interface {
  ListClusters() ([]string, error)
  Master(clusterName string) (string, error)
}

The ListClusters() method returns cluster names. The Master() method returns the IP address or DNS name of the master node.

The other interfaces are not much more complicated. The entire file is 167 lines long including lots of comments. The take-home point is that it is not too complicated to implement a Kubernetes provider if your cloud utilizes those basic concepts.

GCP

The Google Cloud Platform (GCP) is the only cloud provider that supports Kubernetes out of the box. The so-called Google Kubernetes Engine (GKE) is a container management solution built on Kubernetes. You don't need to install Kubernetes on GCP, and you can use the Google cloud API to create Kubernetes clusters and provision them. The fact that Kubernetes is a built-in part of the GCP means it will always be well integrated and well tested, and you don't have to worry about changes in the underlying platform breaking the cloud-provider interface.

All in all, if you plan to base your system on Kubernetes and you don't have any existing code on other cloud platforms, then GCP is a solid choice.

AWS

AWS has its own container management service called ECS, but it is not based on Kubernetes. You can run Kubernetes on AWS very well. It is a supported provider and there is a lot of documentation on how to set it up. While you could provision some VMs yourself and use kubeadm, I recommend using the kops (Kubernetes Operations) project. Kops is a Kubernetes project available on GitHub (http://bit.ly/2ft5KA5). It is not part of Kubernetes itself, but it is developed and maintained by the Kubernetes developers.

It supports the following features:

  • Automated Kubernetes cluster CRUD for the cloud (AWS)
  • Highly Available (HA) Kubernetes clusters
  • Uses a state-sync model for dry-run and automatic idempotency
  • Custom support for kubectl add-ons
  • Kops can generate Terraform configuration
  • Based on a simple meta-model defined in a directory tree
  • Easy command-line syntax
  • Community support

To create a cluster, you need to do some minimal DNS configuration via route53, set up a S3 bucket to store the cluster configuration, and then run a single command:

kops create cluster --cloud=aws --zones=us-east-1c ${NAME}

The complete instructions are here: http://bit.ly/2f7r6EK.

Azure

Azure also has its own container management service. You can use the Mesos-based DC/OS or Docker Swarm to manage them. But you can also use Kubernetes, of course. You can provision the cluster yourself (for example, using Azure's desired state configuration) then create the Kubernetes cluster using kubeadm. But, the recommended approach is to use yet another non-core Kubernetes project, called kubernetes-anywhere (http://bit.ly/2eCS7Ps). The goal of kubernetes-anywhere is to provide a cross-platform way to create clusters in a cloud environment (at least GCP, AWS, and Azure).

The process is pretty painless. You need to have Docker, make, and kubectl installed, and of course, your Azure subscription ID. Then, you clone the kubernetes-anywhere repository, run a couple of make commands, and your cluster is good to go.

The complete instructions to create an Azure cluster are at http://bit.ly/2d56WdA.

In this section, we covered the cloud-provider interface and looked at the various recommended ways to create Kubernetes clusters on various cloud providers. The scene is still young and the tools evolve quickly. I believe convergence will happen soon. Tools and projects such as kubeadm, kops, Kargo, and kubernetes-anywhere will eventually merge and provide a uniform and easy way to bootstrap Kubernetes clusters.

..................Content has been hidden....................

You can't read the all page of ebook, please click here login for view all page.
Reset