One of the more difficult pieces involved in Docker networking is iptables
. The iptables
/netfilter integration plays a key role in providing functionality like port publication and outbound masquerading. However, iptables
can be difficult to understand and troubleshoot if you're not already familiar with it. In this recipe, we'll review how to examine the iptables
configuration in detail and verify that connectivity is working as expected.
In this recipe, we'll be using a single Docker host. It is assumed that Docker is installed and in its default configuration. You'll also need root-level access in order to inspect the iptables
rule set.
As we've seen in earlier chapters, Docker does an outstanding job of managing host firewall rules on your behalf. There will likely be very few instances in which you need to view or modify the iptables
rules as they relate to Docker. However, it's always a good idea to be able to validate the configuration to rule out iptables
as a possible issue when you're troubleshooting container networking.
To demonstrate walking through the iptables
rule set, we'll examine an example container that's publishing a port. The steps we perform to do this are easily transferable to examining rules for any other Docker-integrated iptables
use cases. To do this, we'll run a simple container that exposes port 80
for publishing:
user@docker1:~$ docker run -dP --name web1 jonlangemak/web_server_1
Since we told Docker to publish any exposed ports, we know that this container should have its exposed port of 80
published to the host. To verify that the port is actually being published, we can check the iptables
rule set. The first thing we'd want to do is to make sure that the destination NAT required for port publication is in place. To examine an iptables
table, we can use the iptables
command and pass the following parameters:
n
: Tells iptables
to use numeric information in the output for things such as addresses and portsL
: Tells iptables
that you want to output a list of rulesv
: Tells iptables
to provide verbose output, so we can see all of the rule information as well as rule counterst
: Tells iptables
to only show information from a specific tablePutting that all together, we can use the command sudo iptables –nL –t nat
to view the rules in the NAT table of the host:
If you're not comfortable with iptables
, interpreting this output can be a bit daunting. Even though we're looking at the NAT table, we need to know what chain is being processed for inbound communication to the host. In our case, since the traffic is coming into the host, the chain we're interested with is the PREROUTING
chain. Let's walk through how the table is processed:
PREROUTING
chain looks for traffic destined for LOCAL
or the host itself. Since the traffic is destined to an IP address on one of the host's interfaces, we match on this rule and perform the action that references jumping to a new chain named DOCKER
.DOCKER
chain, we hit the first rule that is looking for traffic coming into the docker0
bridge. Since this traffic isn't coming into the docker0
bridge, the rule is passed over and we move to the next rule in the chain.DOCKER
chain is looking for traffic that's not coming into the docker0
bridge and has a destination port of TCP 32768
. We match this rule and perform the action to perform a destination NAT to 172.17.0.2
port 80
.The processing in the table looks like this:
The arrows in the preceding image indicate the traffic flow as the traffic traverses the NAT table. In this example, we only have one container running on the host, so it's pretty easy to see which rules are being processed.
Now that we've traversed the NAT table, the next thing we need to worry about is the filter table. We can view the filter table in much the same way that we viewed the NAT table:
At first glance, we can see that this table is laid out slightly different than the NAT table was. For instance, we have different chains in this table than we did with the NAT table. In our case, the chain we're interested in for inbound published port communication would be the forward chain. This is because the host is forwarding, or routing, the traffic to the container. The traffic will traverse this table as follows:
DOCKER-ISOLATION
chain.DOCKER-ISOLATION
chain is a rule to send the traffic back, so we resume reviewing rules in the FORWARD
table.docker0
bridge to send the traffic to the DOCKER
chain. Since our destination (172.17.0.20
) lives out the docker0
bridge, we match on this rule and jump to the DOCKER
chain.DOCKER
chain, we inspect the first rule and determine that it's looking for traffic that is destined to the container IP address on port TCP 80
and is going out, but not in, the docker0
bridge. We match on this rule and the flow is accepted.The processing in the table looks like this:
Passing the filter table is the last step published port traffic has to take in order to reach the container. However, we've now only reached the container. We still need to account for the return traffic from the container back to the host talking to the published port. So now, we need to talk about how traffic originated from the container is handled by iptables
.
The first table we'll encounter with outbound traffic is the filter table. Traffic originating from the container will once again use the forward chain of the filter table. The flow would look something like this:
DOCKER-ISOLATION
chain.DOCKER-ISOLATION
chain is a rule to send the traffic back, so we resume reviewing rules in the FORWARD table.docker0
bridge, send the traffic to the DOCKER
chain. Since our traffic is going into the docker0
bridge rather than out, this rule is passed over and we move to the next rule in the chain.docker0
bridge and its connection state is RELATED
or ESTABLISHED
that the flow should be accepted. This traffic is going into the docker0
bridge, so we won't match this rule either. However, it is worth pointing out that this rule is used to allow return traffic for flows initiated from the container. It's just not hit as part of the initial outbound connection since that represents a new flow.docker0
bridge, but not out the docker0
bridge, to accept it. Because our traffic is going into the docker0
bridge, we match on this rule and the traffic is accepted.The processing in the table looks like this:
The next table we'd hit for outbound traffic is the NAT table. This time, we want to look at the POSTROUTING
chain. In this case, we match the first rule of the chain which is looking for traffic that is not going out the docker0
bridge and is sourced from the docker0
bridge subnet (172.17.0.0/16
):
The action for this rule is to MASQUERADE
, which will hide the traffic behind one of the hosts interfaces based on the hosts routing table.
Taking this same approach, you can easily validate other iptables
flows related to Docker. Granted, as the number of containers scale, this becomes a harder task. However, since the majority of rules are written on a per container basis, the hit counters will be unique to each container, making it easier to narrow the scope.
For more information on the order in which iptables
tables and chains are processed, take a look at this iptables
web page and the associated flow charts at http://www.iptables.info/en/structure-of-iptables.html.