In this chapter, we've covered the important topic of Kubernetes cluster federation. Cluster federation is still in the early stages, but it is already usable. There aren't a lot of deployments and the officially supported target platforms are currently AWS and GCE/GKE, but there is a lot of momentum behind cloud federation. It is a very important piece for building massively scalable systems on Kubernetes. We've discussed the motivation and use cases for Kubernetes cluster federation, the federation control plane components, and the federated Kubernetes objects. We also looked into the less supported aspects of federation such as custom scheduling, federated data access, and auto-scaling. We then looked at how to run multiple Kubernetes clusters, which includes setting up and Kubernetes cluster federation, adding and removing clusters to the federation along with load balancing, federated failover when something goes wrong, service discovery, and migration. Then, we dived into running federated workloads across multiple clusters with federated services and the various challenges associated with this scenario.

At this point, you should have a clear understanding of the current state of federation, what it takes to utilize the existing capabilities provided by Kubernetes, and what pieces you'll have to implement yourself to augment incomplete or immature features. Depending on your use case, you may decide that it's still too early or that you want to take the plunge. The developers working on Kubernetes federation are moving fast, so it's very likely that it will be much more mature and battle-tested by the time you need to make your decision.

In Chapter 12, Customizing Kubernetes - API and Plugins, we'll dig into Kubernetes internals and how to customize it. One of the best architectural principles of Kubernetes is that it is accessible through a full-fledged REST API. The Kubectl command-line tool is built on top the Kubernetes API and provides interactivity to the full spectrum of Kubernetes. However, programmatic API access you can leverage provides a lot of flexibility to enhance and extend Kubernetes. There are client libraries in many languages that allow you to leverage Kubernetes from the outside and integrate it into existing systems.

In addition to its REST API, Kubernetes is a very modular platform by design. Many aspects of its core operation can be customized and/or extended. In particular, you can add user-defined resources and integrate them with the Kubernetes object model and benefit from the management services of Kubernetes, storage in etcd, exposure through the API, and uniform access to built-in and custom objects.

We've already seen various aspects that are extremely extensible, such as networking and access control via CNI plugins and custom storage classes. However, Kubernetes goes even further and lets you customize the scheduler itself, which controls pod assignment to nodes.