In the network world, north-south traffic is traditionally defined as client-to-server traffic. In Neutron, as it relates to distributed virtual routers, north-south traffic is the traffic that originates from an external network to virtual machine instances using floating IPs, or vice-versa.
In the legacy model, all traffic to or from external clients traverses a centralized network node hosting a router with floating IPs. With DVR, the same traffic bypasses the network node and is routed directly to the compute nodes that host the virtual machine instance. This functionality requires compute nodes to be connected directly to external networks through an external bridge; a configuration that, up until now, has only been seen on nodes hosting legacy or HA routers.
Unlike SNAT traffic, the traffic through a floating IP is handled on individual compute nodes rather than a centralized node. When a floating IP is attached to a virtual machine instance, the L3 agent on the compute node creates a new fip namespace, which corresponds to the external network that the floating IP belongs to if one doesn't already exist:
Figure 9.34
Router namespaces on a compute node connected to the same external network share a single fip namespace and are connected to the namespace using a veth pair. Veth pairs are treated as point-to-point links between the fip namespace and individual qrouter namespaces and are addressed as /31 networks using a common 169.254/16 link-local address space. As the network connections between namespaces exist only within the nodes themselves and are used as point-to-point links, a Neutron tenant network allocation is not required.
In the qrouter namespace, one end of the veth pair has the prefix rfp, which means router to FIP:
Figure 9.35
Inside the fip namespace, the other end of the veth pair has the prefix fpr, meaning FIP to router:
Figure 9.36
In addition to the fpr interface, a new interface with the prefix fg can be found inside the FIP namespace. The rfp, fpr, and fg interfaces will be discussed in the following sections.
When a floating IP is assigned to an instance using the floatingip-associate command, a couple of things occur:
- A
fipnamespace for the external network is created on the compute node if one doesn't exist - The route table within the
qrouternamespace on the compute node is modified
The following sections demonstrate how traffic to and from floating IPs is processed.
Imagine a scenario where a floating IP, 10.50.0.101, has been assigned to the green VM, as represented in the following diagram:
Figure 9.37
When the green virtual machine instance at 10.30.0.3 sends traffic to an external resource, it arrives at the local qrouter namespace, as follows:
Figure 9.38
|
Source MAC |
Destination MAC |
Source IP |
Destination IP |
|---|---|---|---|
|
Green VM |
Green |
Green VM Fixed IP ( |
(Google DNS) |
When traffic arrives at the local qrouter namespace, the routing policy database is consulted so that traffic may be routed accordingly. Upon association of the floating IP with a port using the Neutron floatingip-associate command, a source routing rule is added to the route table within the qrouter namespace:
Figure 9.39
The main routing table inside the qrouter namespace with a higher priority does not have a default route, so the 32768: from 10.30.0.3 lookup 16 rule is matched instead.
A look at the referenced routing table, table 16, shows that the fip namespace's fpr interface is the default route for traffic sourced from the fixed IP of the instance 10.30.0.3:
Figure 9.40
The qrouter namespace performs the NAT translation of the fixed IP to the floating IP and sends the traffic to the fip namespace, as demonstrated in the following diagram:
Figure 9.41
|
Source MAC |
Destination MAC |
Source IP |
Destination IP |
|---|---|---|---|
|
|
|
Green VM Floating IP ( |
(Google DNS) |
Once traffic arrives at the fip namespace, it is forwarded out of the fg interface to its default gateway:
Figure 9.42
|
Source MAC |
Destination MAC |
Source IP |
Destination IP |
|---|---|---|---|
|
|
Physical default gateway |
Green VM floating IP ( |
(Google DNS) |
If you recall from earlier in this chapter, a single fip namespace on a compute node is shared by every qrouter namespace on this node connected to the external network. Much as a standalone or HA router has an IP address from the external network on its qg interface, each fip namespace has a single IP address from the external network configured on its fg interface.
On compute01, the IP address is 10.50.0.102:
Figure 9.43
If a floating IP were assigned to an instance on compute02, an fip namespace would be created on this node along with a corresponding fg interface with its own address from the external network. On compute02, the IP address is 10.50.0.104:
Figure 9.44
Unlike a legacy router, the qrouter namespaces of distributed routers do not have direct connectivity to the external network. However, the qrouter namespace is still responsible for performing the source NAT from the fixed IP to the floating IP. Traffic is then routed to the fip namespace and from there out to the external network.
Floating IPs are configured on the rfp interface within the qrouter namespace, but they are not directly reachable from the gateway of the external network as the fip namespace sits between the qrouter namespace and the external network. To allow the routing of traffic through the fip namespace back to the qrouter namespace, Neutron relies on the use of proxy ARP. By automatically enabling proxy ARP on the fg interface, the fip namespace can respond to ARP requests for the floating IP, and on behalf of it, from the upstream gateway device.
When traffic is routed from the gateway device to the fip namespace, the routing table is consulted and traffic is routed to the respective qrouter namespace:
Figure 9.45
The following diagram demonstrates how proxy ARP works in this scenario:
Figure 9.46
The fg interface within the fip namespace responds on behalf of the rfp interface within the qrouter namespace as this interface is not directly connected to the external network. The use of a single fip namespace and proxy ARP eliminates the need to provide each qrouter namespace with its own IP address from the external network, which reduces unnecessary IP address consumption and makes more floating IPs available for use by virtual machine instances and other network resources.