An alternative to MacVLAN is IPVLAN. IPVLAN comes in two flavors. The first is L2 mode, which operates very similarly to MacVLAN with the exception of how MAC addresses are assigned. With IPVLAN mode, all logical IP interfaces use the same MAC address. This allows you to keep the parent NIC out of promiscuous mode and also prevents you from running into any possible NIC or switch port MAC limitations. The second mode is IPVLAN layer 3. In layer 3 mode, IPVLAN acts like a router and forwards unicast packets in and out of IPVLAN connected networks. In this recipe, we'll cover the basic IPVLAN networking construct to get an idea of how it works and how it can be implemented.
In this recipe, we'll be using the same Linux hosts (net1 and net2) from the Understanding MacVLAN interfaces recipe in this chapter. Please refer to Understanding MacVLAN recipe's Getting ready section for more information about the topology.
Note
Older versions of the iproute2 toolset did not include full support for IPVLAN. If the commands are not working for the IPVLAN configuration there's a good chance you're on an older version without support. You likely need to update in order to get a newer version that has full support. The older versions have some support for IPVLAN but lack the ability to define a mode (L2 or L3).
As mentioned, IPVLAN in L2 mode is almost identical to MacVLAN in terms of functionality. The major difference is in the fact that IPVLAN leverages the same MAC address for all IPVLAN interfaces attached to the same master. You'll recall that MacVLAN interfaces leveraged a different MAC address for each MacVLAN interface attached to the same parent.
We can create the same interfaces we did in MacVLAN recipe to show that the interface addresses are created with an identical MAC address:
user@net1:~$ sudo ip link add ipvlan1 link eth0 type ipvlan mode l2 user@net1:~$ sudo ip address add 172.16.10.5/24 dev ipvlan1 user@net1:~$ sudo ip link set dev ipvlan1 up user@net1:~$ sudo ip link add ipvlan2 link eth0 type ipvlan mode l2 user@net1:~$ sudo ip address add 172.16.10.6/24 dev ipvlan2 user@net1:~$ sudo ip link set dev ipvlan2 up
Note that the only difference in the configuration is that we're specifying the type as IPVLAN and the mode as L2. In the case of IPVLAN, the default mode is L3, so we need to specify L2 in order to get the interfaces to operate in that fashion. Since IPVLAN interfaces inherit the MAC address of the parent, our topology should look like this:
We can prove this just by checking the interface themselves:
user@net1:~$ ip -d link …<loopback interface removed for brevity>… 2: eth0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast state UP mode DEFAULT group default qlen 1000 link/ether 00:0c:29:2d:dd:79 brd ff:ff:ff:ff:ff:ff promiscuity 1 addrgenmode eui64 3: eth1: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast state UP mode DEFAULT group default qlen 1000 link/ether 00:0c:29:2d:dd:83 brd ff:ff:ff:ff:ff:ff promiscuity 0 addrgenmode eui64 28: ipvlan1@eth0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue state UNKNOWN mode DEFAULT group default link/ether 00:0c:29:2d:dd:79 brd ff:ff:ff:ff:ff:ff promiscuity 0 ipvlan mode l2 addrgenmode eui64 29: ipvlan2@eth0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue state UNKNOWN mode DEFAULT group default link/ether 00:0c:29:2d:dd:79 brd ff:ff:ff:ff:ff:ff promiscuity 0 ipvlan mode l2 addrgenmode eui64 user@net1:~$
If we were to initiate traffic toward these IPs from off of the local subnet we could validate that each IP is reporting the same MAC address by checking the ARP table on the upstream gateway:
switch#show ip arp vlan 10 Protocol Address Age (min) Hardware Addr Type Interface Internet 172.16.10.6 0 000c.292d.dd79 ARPA Vlan30 Internet 172.16.10.5 0 000c.292d.dd79 ARPA Vlan30 Internet 172.16.10.2 111 000c.292d.dd79 ARPA Vlan30 Internet 172.16.10.3 110 000c.2959.caca ARPA Vlan30 Internet 172.16.10.1 - 0021.d7c5.f245 ARPA Vlan30
And while we won't show an example here, IPVLAN interfaces in L2 mode are also namespace aware just like we saw in the last couple of recipes with the MacVLAN interface type. The only difference would be in the interface MAC addresses as we saw in the preceding code. The same restrictions apply in regard to the parent interface being unable to talk to the child interfaces and vice versa.
Now that we know how IPVLAN in L2 mode works, let's discuss IPVLAN L3 mode. L3 mode is significantly different than what we've seen up to this point. As the name L3 mode suggests, this interface type routes traffic between all attached subinterfaces. This is easiest to comprehend in a namespace configuration. For instance, let's look at this quick lab topology:
In the preceding image, you can see that I've created four unique namespaces across our two lab hosts. I also created four unique IPVLAN interfaces, mapped them into the different namespaces, and gave them each a unique IP address. Since these are IPVLAN interfaces, you'll note that all of the IPVLAN interfaces share the MAC address of the parent interface. To build this topology, I used the following configuration on each respective host:
user@net1:~$ sudo ip link del dev ipvlan1 user@net1:~$ sudo ip link del dev ipvlan2 user@net1:~$ sudo ip netns add namespace1 user@net1:~$ sudo ip netns add namespace2 user@net1:~$ sudo ip link add ipvlan1 link eth0 type ipvlan mode l3 user@net1:~$ sudo ip link add ipvlan2 link eth0 type ipvlan mode l3 user@net1:~$ sudo ip link set ipvlan1 netns namespace1 user@net1:~$ sudo ip link set ipvlan2 netns namespace2 user@net1:~$ sudo ip netns exec namespace1 ip address add 10.10.20.10/24 dev ipvlan1 user@net1:~$ sudo ip netns exec namespace1 ip link set dev ipvlan1 up user@net1:~$ sudo ip netns exec namespace2 sudo ip address add 10.10.30.10/24 dev ipvlan2 user@net1:~$ sudo ip netns exec namespace2 ip link set dev ipvlan2 up user@net2:~$ sudo ip netns add namespace3 user@net2:~$ sudo ip netns add namespace4 user@net2:~$ sudo ip link add ipvlan3 link eth0 type ipvlan mode l3 user@net2:~$ sudo ip link add ipvlan4 link eth0 type ipvlan mode l3 user@net2:~$ sudo ip link set ipvlan3 netns namespace3 user@net2:~$ sudo ip link set ipvlan4 netns namespace4 user@net2:~$ sudo ip netns exec namespace3 ip address add 10.10.40.10/24 dev ipvlan3 user@net2:~$ sudo ip netns exec namespace3 ip link set dev ipvlan3 up user@net2:~$ sudo ip netns exec namespace4 sudo ip address add 10.10.40.11/24 dev ipvlan4 user@net2:~$ sudo ip netns exec namespace4 ip link set dev ipvlan4 up
Once this is configured, you'll note that the only interfaces that can communicate with one another are those on the host net2 (10.10.40.10 and 10.10.40.11). Let's look at this topology logically to understand why:
Looking at this logically, it starts to look like a routed network. You'll notice that all the IP addresses assigned are unique without any overlap. As I mentioned earlier, IPVLAN L3 mode acts like a router. From a conceptual perspective, you can think of the parent interface as that router. If we look at this from a layer 3 perspective, it makes sense that only the interfaces in namespaces 3 and 4 can talk because they are in the same broadcast domain. The other namespaces would need to route through a gateway to talk to each other. Let's check the routing table on all of the namespaces to see where things stand:
user@net1:~$ sudo ip netns exec namespace1 ip route 10.10.20.0/24 dev ipvlan1 proto kernel scope link src 10.10.20.10 user@net1:~$ sudo ip netns exec namespace2 ip route 10.10.30.0/24 dev ipvlan2 proto kernel scope link src 10.10.30.10 user@net2:~$ sudo ip netns exec namespace3 ip route 10.10.40.0/24 dev ipvlan3 proto kernel scope link src 10.10.40.10 user@net2:~$ sudo ip netns exec namespace4 ip route 10.10.40.0/24 dev ipvlan4 proto kernel scope link src 10.10.40.11
As expected, each namespace only knows about the local network. So in order for these interfaces to communicate, they need to have at the very least a default route. This is where things get a little interesting. IPVLAN interfaces do not allow broadcast or multicast traffic. This means that if we defined the gateway of the interface to be the upstream switch, it would never be able to reach it because it wouldn't be able to ARP for it. However, since the parent is acting like a sort of router, we can have the namespaces use the IPVLAN interface itself as a gateway. We can do that by adding default routes in this fashion:
user@net1:~$ sudo ip netns exec namespace1 ip route add default dev ipvlan1 user@net1:~$ sudo ip netns exec namespace2 ip route add default dev ipvlan2 user@net2:~$ sudo ip netns exec namespace3 ip route add default dev ipvlan3 user@net2:~$ sudo ip netns exec namespace4 ip route add default dev ipvlan4
After adding these routes, you'll also need to add routes on each Linux host to tell them where to go to reach these remote subnets. Since the two hosts in this example are layer 2 adjacent, the best place to do this on the host itself. While you could also rely on the default route and configure these routes on the upstream network device that would not be ideal. You would effectively be routing in and out of the same L3 interface on the gateway, which isn't great network design practice. If the hosts had not been layer 2 adjacent adding the routes on the multilayer switch would have been required:
user@net1:~$ sudo ip route add 10.10.40.0/24 via 172.16.10.3 user@net2:~$ sudo ip route add 10.10.20.0/24 via 172.16.10.2 user@net2:~$ sudo ip route add 10.10.30.0/24 via 172.16.10.2
After you have all of the routes installed, you should be able to reach all of the namespaces from any of the other namespaces:
user@net1:~$ sudo ip netns exec namespace1 ping 10.10.30.10 -c 2 PING 10.10.30.10 (10.10.30.10) 56(84) bytes of data. 64 bytes from 10.10.30.10: icmp_seq=1 ttl=64 time=0.047 ms 64 bytes from 10.10.30.10: icmp_seq=2 ttl=64 time=0.033 ms --- 10.10.30.10 ping statistics --- 2 packets transmitted, 2 received, 0% packet loss, time 999ms rtt min/avg/max/mdev = 0.033/0.040/0.047/0.007 ms user@net1:~$ sudo ip netns exec namespace1 ping 10.10.40.10 -c 2 PING 10.10.40.10 (10.10.40.10) 56(84) bytes of data. 64 bytes from 10.10.40.10: icmp_seq=1 ttl=64 time=0.258 ms 64 bytes from 10.10.40.10: icmp_seq=2 ttl=64 time=0.366 ms --- 10.10.40.10 ping statistics --- 2 packets transmitted, 2 received, +3 duplicates, 0% packet loss, time 1001ms rtt min/avg/max/mdev = 0.258/0.307/0.366/0.042 ms user@net1:~$ sudo ip netns exec namespace1 ping 10.10.40.11 -c 2 PING 10.10.40.11 (10.10.40.11) 56(84) bytes of data. 64 bytes from 10.10.40.11: icmp_seq=1 ttl=64 time=0.246 ms 64 bytes from 10.10.40.11: icmp_seq=2 ttl=64 time=0.366 ms --- 10.10.40.11 ping statistics --- 2 packets transmitted, 2 received, +3 duplicates, 0% packet loss, time 1001ms rtt min/avg/max/mdev = 0.246/0.293/0.366/0.046 ms user@net1:~$ s
As you can see, IPVLAN L3 mode is a different animal than what we've seen up until this point. Unlike MacVLAN or IPVLAN L2, you'll need to tell the network how to reach these new interfaces.