As mentioned earlier, Flannel supports multiple different backend configurations. A backend is considered to be the means by which Flannel passes traffic between Flannel-enabled hosts. By default, this is done through UDP as we saw in the previous recipe. However, Flannel also supports VXLAN. The advantage to using VXLAN over UDP is that newer hosts support VXLAN in the kernel. In this recipe, we'll demonstrate how to change the Flannel backend type to VXLAN.
It is assumed that you're building off the lab described in the previous recipes in this chapter. You'll need Flannel-enabled hosts that are integrated with Docker as described in the first two recipes of this chapter. In some cases, the changes we make may require you to have root-level access to the system.
The type of backend you wish to use is defined when you first instantiate your network within etcd. Since we didn't specify a type when we defined the network 10.100.0.0/16, Flannel defaulted to using the UDP backend. This can be changed by updating the configuration we initially set in etcd. Recall that our Flannel network was first defined with this command:
etcdctl mk /coreos.com/network/config '{"Network":"10.10.0.0/16"}'Note how we used the mk command of etcdctl to make the key. If we wanted to change the backend type to VXLAN, we could run this command:
etcdctl set /coreos.com/network/config '{"Network":"10.100.0.0/16", "Backend": {"Type": "vxlan"}}'Note that, since we're updating the object, we now use the set command in place of mk. While sometimes hard to see when in plain text form, the properly formatted JSON that we're passing to etcd looks like this:
{
"Network": "10.100.0.0/16",
"Backend": {
"Type": "vxlan",
}
}This defines the type of this backend to be VXLAN. While the preceding configuration by itself would be sufficient to change the backend type, there are sometimes additional parameters that we can specify as part of the backend. For instance, when defining the type as VXLAN, we can also specify a VXLAN Identifier (VNI) and a UDP port. If not specified, the VNI defaults to 1 and the port defaults to the 8472. For the sake of demonstration, we'll apply the defaults as part of our configuration:
user@docker1:~$ etcdctl set /coreos.com/network/config
'{"Network":"10.100.0.0/16", "Backend": {"Type": "vxlan","VNI": 1, "Port": 8472}}'This in properly formatted JSON looks like this:
{
"Network": "10.100.0.0/16",
"Backend": {
"Type": "vxlan",
"VNI": 1,
"Port": 8472
}
}If we run the command the local etcd instances configuration will be updated. We can verify that etcd has the proper configuration by querying etcd through the etcdctl command-line tool. To read the configuration, we can use the etcdctl get subcommand:
user@docker1:~$ etcdctl get /coreos.com/network/config
{"Network":"10.100.0.0/16", "Backend": {"Type": "vxlan", "VNI": 1, "Port": 8472}}
user@docker1:~$Although we've successfully updated etcd, the Flannel services on each node will not act on this new configuration. This is because the Flannel service on each host only reads these variables when the service starts. In order for this change to take effect, we need to restart the Flannel service on each node:
user@docker4:~$ sudo systemctl restart flanneld
Make sure that you restart the Flannel service on each host. Hosts will not be able to communicate if some are using the VXLAN backend and others are using the UDP backend. Once restarted, we can check our Docker host's interfaces once again:
user@docker4:~$ ip addr show …<Additional output removed for brevity>… 11: flannel.1: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1450 qdisc noqueue state UNKNOWN group default link/ether 2e:28:e7:34:1a:ff brd ff:ff:ff:ff:ff:ff inet 10.100.15.0/16 scope global flannel.1 valid_lft forever preferred_lft forever inet6 fe80::2c28:e7ff:fe34:1aff/64 scope link valid_lft forever preferred_lft forever
Here, we can see that the host now has a new interface named flannel.1. If we check the interface with ethtool, we can see that it is using the VXLAN driver:
user@docker4:~$ ethtool -i flannel.1 driver: vxlan version: 0.1 firmware-version: bus-info: supports-statistics: no supports-test: no supports-eeprom-access: no supports-register-dump: no supports-priv-flags: no user@docker4:~$
And we should still be able to access the services using the Flannel IP addresses:
user@docker4:~$ docker exec -it web2 curl http://10.100.93.2 <body> <html> <h1><span style="color:#FF0000;font-size:72px;">Web Server #1 - Running on port 80</span> </h1> </body> </html> user@docker4:~$
It is important to know that Flannel does not take care of cleaning up artifacts from older configurations. For instance, if you change the VXLAN ID in etcd and restart the Flannel service, you will end up with two interfaces on the same network. You'll want to manually delete the old interface that was named using the old VNI. In addition, if you change the subnet allocated to Flannel, you'll want to restart the Docker service after you restart the Flannel service. Recall that Docker reads configuration variables from Flannel when the Docker service loads. If those change you'll need to reload the configuration for them to take effect.