So far we have learned how to launch individual Docker hosts locally using Docker for Mac, Docker for Windows, and Docker Machine for remote hosts, as well as using Docker locally on Linux. Individual Docker hosts are great for local development, or launching a few test instances however as you start moving towards production you need fewer single points of failure.
In this chapter, we are going to get a little more adventurous and create a cluster of Docker hosts. Docker ships a tool called Swarm, when deployed it acts as a scheduler between your Docker client and the Docker host, deciding where to launch containers based on scheduling rules.
We are going to look at the following topics:
- Manually launching a Docker Swarm cluster
- Launching Docker for Amazon Web Services
- Launching Docker for Azure
And also how to launch containers within our cluster.
At the start of Chapter 3, Docker in the Cloud we looked at using a Docker Machine to launch a Docker host in Digital Ocean. We are going to start with Digital Ocean again, but this time we are going to launch three hosts and then create a Docker Swarm cluster on them.
To start off with we need to launch the hosts and to do this, run the following commands, remembering to replace the Digital Ocean API access token with your own:
docker-machine create --driver digitalocean --digitalocean-access-token 57e4aeaff8d7d1a8a8e46132969c2149117081536d50741191c79d8bc083ae73 swarm01 docker-machine create --driver digitalocean --digitalocean-access-token 57e4aeaff8d7d1a8a8e46132969c2149117081536d50741191c79d8bc083ae73 swarm02 docker-machine create --driver digitalocean --digitalocean-access-token 57e4aeaff8d7d1a8a8e46132969c2149117081536d50741191c79d8bc083ae73 swarm03
Once launched, running docker-machine ls should show you a list of your images. Also, this should be reflected in your Digital Ocean control panel:
Now we have our Docker hosts and we need to assign a role to each of the nodes within the cluster. Docker Swarm has two node roles:
- Manager: A manager is a node which dispatches tasks to the workers, all your interaction with the Swarm cluster will be targeted against a manager node. You can have more than one Manger node, however in this example we will be using just one.
- Worker: Worker nodes accept the tasks dispatched by the Manager node(s), these are where all your services are launched. We will go in to services in more detail once we have our cluster configured.
In our cluster, swarm01 will be the manager node with swarm02 and swarm03 being our two worker nodes. We are going to use the docker-machine ssh command to execute commands directly on our three nodes, starting with configuring our manager node.
Before we initialize the manager node, we need to capture the IP address of swarm01 as a command-line variable:
managerIP=$(docker-machine ip swarm01)
Now that we have the IP address, run the following command to check if it is correct:
echo $managerIP
And then to configure the manager node, run the following command:
docker-machine ssh swarm01 docker swarm init --advertise-addr $managerIP
You will then receive confirmation that swarm01 is now a manager along with instructions on what to run to add a worker to the cluster:
You don't have to a make a note of the instructions as we will be running the command in a slightly different way.
To add our two workers, we need to capture the join token in a similar way we captured the IP address of our manager node using the $managerIP variable; to do this, run:
joinToken=$(docker-machine ssh swarm01 docker swarm join-token -q worker)
Again, you echo the variable out to check that it is valid:
echo $joinToken
Now it's time to add our two worker nodes into the cluster by running:
docker-machine ssh swarm02 docker swarm join --token $joinToken $managerIP:2377 docker-machine ssh swarm03 docker swarm join --token $joinToken $managerIP:2377
You should see something like the following terminal output:
Connect your local Docker client to the manager node using the following:
eval $(docker-machine env swarm01)
And then running a docker-machine ls again shows. As you can see from the list of hosts, swarm01 is now active but there is nothing in the SWARM column; why is that?
Confusingly, there are two different types of Docker Swarm cluster, there is the Legacy Docker Swarm which was managed by Docker Machine, and then there is the new Docker Swarm mode which is managed by the Docker Engine itself.
We have a launched a Docker Swarm Mode cluster. This is now the preferred way of launching Swarm, the legacy Docker Swarm is slowly being retired.
To get a list of the nodes within our Swarm cluster, we need to run the following command:
docker node ls
For information on each node you can run the following command (the --pretty flag renders the JSON output from the Docker API):
docker node inspect swarm01 --pretty
You are given a wealth of information about the host, including the fact that it is a manager and it has been launched in Digital Ocean. Running the same command; but for a worker node shows similar information:
docker node inspect swarm02 --pretty
However, as the node is not a manager that section is missing.
Before we look at launching services into our cluster, we should look at how to launch our cluster using Docker Machine on Windows. We will be using PowerShell for this rather than the more traditional Windows CMD prompt, however, even using PowerShell there are a few differences in the commands used due differences between PowerShell and bash.
First, we need to launch the three hosts:
docker-machine.exe create --driver digitalocean --digitalocean-access-token 57e4aeaff8d7d1a8a8e46132969c2149117081536d50741191c79d8bc083ae73 swarm01 docker-machine.exe create --driver digitalocean --digitalocean-access-token 57e4aeaff8d7d1a8a8e46132969c2149117081536d50741191c79d8bc083ae73 swarm02 docker-machine.exe create --driver digitalocean --digitalocean-access-token 57e4aeaff8d7d1a8a8e46132969c2149117081536d50741191c79d8bc083ae73 swarm03
Once the three hosts are up and running:
You can create the manager node by running:
$managerIP = $(docker-machine.exe ip swarm01) echo $managerIP docker-machine.exe ssh swarm01 docker swarm init --advertise-addr $managerIP
Once you have your manager you can add the two worker nodes:
$joinIP = "$(docker-machine.exe ip swarm01):2377" echo $joinIP $joinToken = $(docker-machine.exe ssh swarm01 docker swarm join-token -q worker) echo $joinToken docker-machine.exe ssh swarm02 docker swarm join --token $joinToken $joinIP docker-machine.exe ssh swarm03 docker swarm join --token $joinToken $joinIP
And then configure your local Docker client to use your manager node and check the cluster status:
docker-machine.exe env swarm01 | Invoke-Expression docker-machine.exe ls docker node ls
At this stage, no matter which operating system you are using, you should have a three node Docker Swarm cluster in Digital Ocean, we can now look at a launching service into our cluster.