Docker uses the Linux bridge docker0 by default. However, there are cases where Open vSwitch (OVS) might be required instead of a Linux bridge. A single Linux bridge can only handle 1024 ports – this limits the scalability of Docker as we can only create 1024 containers, each with a single network interface.
We will now install OVS on a single host, create two containers, and connect them to an OVS bridge.
Use this command to install OVS:
# sudo apt-get install openvswitch-switch
Install the ovs-docker utility with the following:
# cd /usr/bin # wget https://raw.githubusercontent.com/openvswitch/ovs/master/utilities/ovs-docker # chmod a+rwx ovs-docker
The following diagram shows the single-host OVS:
Here, we will be adding a new OVS bridge and configuring it so that we can get the containers connected on a different network, as follows:
# ovs-vsctl add-br ovs-br1 # ifconfig ovs-br1 173.16.1.1 netmask 255.255.255.0 up
Add a port from the OVS bridge to the Docker container using the following steps:
- Create two Ubuntu Docker containers:
# docker run -I -t --name container1 ubuntu /bin/bash # docekr run -I -t --name container2 ubuntu /bin/bash
- Connect the container to the OVS bridge:
# ovs-docker add-port ovs-br1 eth1 container1 --ipaddress=173.16.1.2/24 # ovs-docker add-port ovs-br1 eth1 container2 --ipaddress=173.16.1.3/24
- Test the connection between the two containers connected via an OVS bridge using the
pingcommand. First, find out their IP addresses:# docker exec container1 ifconfig eth0 Link encap:Ethernet HWaddr 02:42:ac:10:11:02 inet addr:172.16.17.2 Bcast:0.0.0.0 Mask:255.255.255.0 inet6 addr: fe80::42:acff:fe10:1102/64 Scope:Link UP BROADCAST RUNNING MULTICAST MTU:1472 Metric:1 RX packets:36 errors:0 dropped:0 overruns:0 frame:0 TX packets:8 errors:0 dropped:0 overruns:0 carrier:0 collisions:0 txqueuelen:0 RX bytes:4956 (4.9 KB) TX bytes:648 (648.0 B) lo Link encap:Local Loopback inet addr:127.0.0.1 Mask:255.0.0.0 inet6 addr: ::1/128 Scope:Host UP LOOPBACK RUNNING MTU:65536 Metric:1 RX packets:0 errors:0 dropped:0 overruns:0 frame:0 TX packets:0 errors:0 dropped:0 overruns:0 carrier:0 collisions:0 txqueuelen:0 RX bytes:0 (0.0 B) TX bytes:0 (0.0 B) # docker exec container2 ifconfig eth0 Link encap:Ethernet HWaddr 02:42:ac:10:11:03 inet addr:172.16.17.3 Bcast:0.0.0.0 Mask:255.255.255.0 inet6 addr: fe80::42:acff:fe10:1103/64 Scope:Link UP BROADCAST RUNNING MULTICAST MTU:1472 Metric:1 RX packets:27 errors:0 dropped:0 overruns:0 frame:0 TX packets:8 errors:0 dropped:0 overruns:0 carrier:0 collisions:0 txqueuelen:0 RX bytes:4201 (4.2 KB) TX bytes:648 (648.0 B) lo Link encap:Local Loopback inet addr:127.0.0.1 Mask:255.0.0.0 inet6 addr: ::1/128 Scope:Host UP LOOPBACK RUNNING MTU:65536 Metric:1 RX packets:0 errors:0 dropped:0 overruns:0 frame:0 TX packets:0 errors:0 dropped:0 overruns:0 carrier:0 collisions:0 txqueuelen:0 RX bytes:0 (0.0 B) TX bytes:0 (0.0 B)
Now that we know the IP addresses of
container1andcontainer2, we can ping them:# docker exec container2 ping 172.16.17.2 PING 172.16.17.2 (172.16.17.2) 56(84) bytes of data. 64 bytes from 172.16.17.2: icmp_seq=1 ttl=64 time=0.257 ms 64 bytes from 172.16.17.2: icmp_seq=2 ttl=64 time=0.048 ms 64 bytes from 172.16.17.2: icmp_seq=3 ttl=64 time=0.052 ms # docker exec container1 ping 172.16.17.2 PING 172.16.17.2 (172.16.17.2) 56(84) bytes of data. 64 bytes from 172.16.17.2: icmp_seq=1 ttl=64 time=0.060 ms 64 bytes from 172.16.17.2: icmp_seq=2 ttl=64 time=0.035 ms 64 bytes from 172.16.17.2: icmp_seq=3 ttl=64 time=0.031 ms
Let's see how to connect Docker containers on multiple hosts using OVS.
Let's consider our setup as shown in the following diagram, which contains two hosts, Host 1 and Host 2, running Ubuntu 14.04:
Install Docker and Open vSwitch on both the hosts:
# wget -qO- https://get.docker.com/ | sh # sudo apt-get install openvswitch-switch
Install the ovs-docker utility:
# cd /usr/bin # wget https://raw.githubusercontent.com/openvswitch/ovs/master/utilities/ovs-docker # chmod a+rwx ovs-docker
By default, Docker chooses a random network to run its containers in. It creates a bridge, docker0, and assigns an IP address (172.17.42.1) to it. So, both Host 1 and Host 2 docker0 bridge IP addresses are the same, due to which it is difficult for containers in both the hosts to communicate. To overcome this, let's assign static IP addresses to the network, that is, 192.168.10.0/24.
Let's see how to change the default Docker subnet.
Execute the following commands on Host 1:
# service docker stop # ip link set dev docker0 down # ip addr del 172.17.42.1/16 dev docker0 # ip addr add 192.168.10.1/24 dev docker0 # ip link set dev docker0 up # ip addr show docker0 # service docker start
Add the br0 OVS bridge:
# ovs-vsctl add-br br0
Create the tunnel to the other host and attach it to the:
# add-port br0 gre0 -- set interface gre0 type=gre options:remote_ip=30.30.30.8
Add the br0 bridge to the docker0 bridge:
# brctl addif docker0 br0
Execute the following commands on Host 2:
# service docker stop # iptables -t nat -F POSTROUTING # ip link set dev docker0 down # ip addr del 172.17.42.1/16 dev docker0 # ip addr add 192.168.10.2/24 dev docker0 # ip link set dev docker0 up # ip addr show docker0 # service docker start
Add the br0 OVS bridge:
# ip link set br0 up # ovs-vsctl add-br br0
Create the tunnel to the other host and attach it to the:
# br0 bridge ovs-vsctl add-port br0 gre0 -- set interface gre0 type=gre options:remote_ip=30.30.30.7
Add the br0 bridge to the docker0 bridge:
# brctl addif docker0 br0
The docker0 bridge is attached to another bridge, br0. This time, it's an OVS bridge. This means that all traffic between the containers is routed through br0 too.
Additionally, we need to connect together the networks from both the hosts in which the containers are running. A GRE tunnel is used for this purpose. This tunnel is attached to the br0 OVS bridge and, as a result, to docker0 too.
After executing the preceding commands on both hosts, you should be able to ping the docker0 bridge addresses from both hosts.
On Host 1, the following output is generated on using the ping command:
# ping 192.168.10.2 PING 192.168.10.2 (192.168.10.2) 56(84) bytes of data. 64 bytes from 192.168.10.2: icmp_seq=1 ttl=64 time=0.088 ms 64 bytes from 192.168.10.2: icmp_seq=2 ttl=64 time=0.032 ms ^C --- 192.168.10.2 ping statistics --- 2 packets transmitted, 2 received, 0% packet loss, time 999ms rtt min/avg/max/mdev = 0.032/0.060/0.088/0.028 ms
On Host 2, the following output is generated on using the ping command:
# ping 192.168.10.1 PING 192.168.10.1 (192.168.10.1) 56(84) bytes of data. 64 bytes from 192.168.10.1: icmp_seq=1 ttl=64 time=0.088 ms 64 bytes from 192.168.10.1: icmp_seq=2 ttl=64 time=0.032 ms ^C --- 192.168.10.1 ping statistics --- 2 packets transmitted, 2 received, 0% packet loss, time 999ms rtt min/avg/max/mdev = 0.032/0.060/0.088/0.028 ms
Let's see how to create containers on both the hosts.
On Host 1, use the following code:
# docker run -t -i --name container1 ubuntu:latest /bin/bash
On Host 2, use the following code:
# docker run -t -i --name container2 ubuntu:latest /bin/bash
Now we can ping container2 from container1. In this way, we connect Docker containers on multiple hosts using Open vSwitch.