We'll be using multiple Docker hosts in the following examples. We'll also be using a user-defined overlay network. It will be assumed that you know how to configure the Docker hosts for overlay networking. If you do not, please see the Creating a user-defined overlay network recipe in Chapter 3, User-Defined Networks.

Load balancing is not a new concept and is one that is well-understood in the physical and virtual machine space. However, load balancing with containers adds in an extra layer of complexity, which can make things drastically more complicated. To start with, let's look how load balancing typically works without containers:

In this case, we have a simple load balancer configuration where the load balancer is providing VIP for a single backend pool member (192.168.50.150). The flow works like this:

In most cases, the conversation involves two distinct sessions, one between the client and the load balancer and another between the load balancer and the server. Each of these is a distinct TCP session.

Now, let's show an example of how this might work in the container space. Examine the topology shown in the following figure:

In this example, we'll be using both container-based application servers as backend pool members as well as a container-based load balancer. Let's make the following assumptions:

Given this, we can assume that host network mode is out of the question for both the load balancer host (docker1) as well as the hosting hosts (docker2 and docker3) since it would require containers exposing services on a large number of ports. Before the introduction of user-defined networks, this would leave us with having to deal with port mapping on the docker0 bridge.

That would quickly become a problem both to manage as well as troubleshoot. For instance, the topology might really look like this:

In this case, the load balancer VIP would be a published port on the host docker1, that is, 32769. The web servers themselves are also publishing ports to expose their web servers. Let's walk through what a load balancing request might look like:

While doable, it's a lot to keep track of. In addition, the load balancer node needs to be aware of the published port and IP address of each backend container. If a container gets restarted, the published port can change effectively making it unreachable. Troubleshooting this with a large backend pool would be a headache as well.

So while this is certainly doable, the introduction of user-defined networks can make this much more manageable. For instance, we could leverage an overlay type network for the backend pool members and completely remove the need for much of the port publishing and outbound masquerading. That topology would look more like this:

Let's see what it would take to build this kind of configuration. The first thing we need to do is to define a user-defined overlay type network on one of the nodes. We'll define it on docker1 and call it presentation_backend:

user@docker1:~$ docker network create -d overlay 
--internal presentation_backend
bd9e9b5b5e064aee2ddaa58507fa6c15f49e4b0a28ea58ffb3da4cc63e6f8908
user@docker1:~$

The next thing we want to do is to create the two web containers which will serve as the backend pool members for the load balancer. We'll do that on hosts docker2 and docker3:

user@docker2:~$ docker run -dP --name=web1 --net 
presentation_backend jonlangemak/web_server_1
6cc8862f5288b14e84a0dd9ff5424a3988de52da5ef6a07ae593c9621baf2202
user@docker2:~$
user@docker3:~$ docker run -dP --name=web2 --net 
presentation_backend jonlangemak/web_server_2
e2504f08f234220dd6b14424d51bfc0cd4d065f75fcbaf46c7b6dece96676d46
user@docker3:~$

The remaining component left to deploy is the load balancer. As mentioned, haproxy has a container image of their load balancer, so we'll use that for this example. Before we run the container we need to come up with a configuration that we can pass into the container for haproxy to use. This is done through mounting a volume into the container as we'll see shortly. The configuration file is named haproxy.cfg and my example configuration looks like this:

global
    log 127.0.0.1   local0
defaults
    log     global
    mode    http
    option  httplog
    timeout connect 5000
    timeout client 50000
    timeout server 50000
    stats enable
    stats auth user:docker
    stats uri /lbstats
frontend all
    bind *:80
    use_backend pres_containers

backend pres_containers
    balance roundrobin
    server web1 web1:80 check
    server web2 web2:80 check
    option httpchk HEAD /index.html HTTP/1.0

A couple of items are worth pointing out in the preceding configuration:

One of the interesting items here is that we can define the pool members by name. This is a huge advantage that comes along with user-defined networks and means that we don't need to worry about tracking container IP addressing.

I put this config file in a folder in my home directory named haproxy:

user@docker1:~/haproxy$ pwd
/home/user/haproxy
user@docker1:~/haproxy$ ls
haproxy.cfg
user@docker1:~/haproxy$

Once the configuration file is in pace, we can run the container as follows:

user@docker1:~$ docker run -d --name haproxy --net 
presentation_backend -p 80:80 -v 
~/haproxy:/usr/local/etc/haproxy/ haproxy
d34667aa1118c70cd333810d9c8adf0986d58dab9d71630d68e6e15816741d2b
user@docker1:~$

You might be wondering why I'm specifying a port mapping when connecting the container to an internal type network. Recall from earlier chapters that port mappings are global across all network types. In other words, even though I'm not using it currently, it's still a characteristic of the container. So if I ever connect a network type to the container that can use the port mapping, it will. In this case, I first need to connect the container to the overlay network to ensure that it has reachability to the backend web servers. If the haproxy container is unable to resolve the pool member names when it starts, it will fail to load.

At this point, the haproxy container has reachability to its pool members, but we have no way to access the haproxy container externally. To do that, we'll connect another interface to the container that can use the port mapping. In this case, that will be the docker0 bridge:

user@docker1:~$ docker network connect bridge haproxy
user@docker1:~

At this point, the haproxy container should be available externally at the following URLs:

If we check the stats page, we should see that the haproxy container can reach each backend web server across the overlay. We can see that the health check for each is coming back with a 200 OK status:

Now if we check VIP itself and hit refresh a couple of times, we should see the web page presented from each container:

This type of topology provides us several notable advantages over the first concept we had around container load balancing. The use of the overlay-based network not only provided name-based resolution of containers but also significantly reduced the complexity of the traffic path. Granted, the traffic took the same physical path in either case, but we didn't need to rely on so many different NATs for the traffic to work. It also made the entire solution far more dynamic. This type of design can be easily replicated to provide load balancing for many different backend overlay networks.