Multi-WAN configuration

Gateway load balancing is accomplished by setting up gateway groups, which consist of two or more WAN interfaces. Configuring gateway load balancing involves several steps:

  1. Adding and configuring additional WAN interfaces.
  2. Configuring DNS servers for each of the new WAN interfaces.
  3. Adding gateway groups which include the new interfaces.
  4. Adding firewall rules for each of the new gateway groups.

The first step, adding and configuring additional WAN interfaces, is a fairly simple process. When you initially set up pfSense, the WAN interface was automatically configured, but configuring additional WAN interfaces isn't much different from configuring any other interfaces. Refer to the following steps:

  1. Navigate to Interfaces | (assign), and on the Interface Assignments tab, there should be a tab showing all the existing interface assignments (which should be, at a minimum, the WAN and LAN interfaces).
  2. To add the second WAN interface, select an unused network interface in the Available network ports drop-down box and click on the Add button on the right side of the row. This will add the new interface (which will initially have a generic name like OPT1). Click on the new interface's name in the leftmost column in the table.
  3. Alternately, you can select it from the Interfaces drop-down menu to begin configuration.
  4. On the interface configuration page, check the Enable checkbox, and enter an appropriate description in the Description field (for example, WAN2).

 

  1. In the IPv4 Configuration Type and IPv6 Configuration Type drop-down boxes, you must select the appropriate configuration type for the interface IP. If the interface will be receiving an IP address from your ISP, then the correct selection is DHCP (or DHCP6 or SLAAC for IPv6). If you choose DHCP and/or DHCP6 or SLAAC, then there isn't much more you have to do for interface configuration. pfSense will automatically set up the interface as a gateway, so you don't have to do this.
  2. If you chose Static IPv4 and/or Static IPv6 as Configuration Type, however, you will have to manually configure this interface as a gateway.
  3. Fortunately, all we have to do is scroll down to the Static IPv4/IPv6 Configuration section of the page and click on the Add new gateway button.
  4. This will launch a dialog box which will allow you to configure the most basic options for the gateway.
  5. You should leave the Default gateway checkbox unchecked, as the first WAN interface is already the default gateway. Give the gateway an appropriate name in the Gateway name edit box (for example, GW2), and enter a gateway IP address in the Gateway IPv4 (or Gateway IPv6) edit box. This should be an address different to the interface's IP address, but on the same subnet.
  6. Finally, you may enter a non-parsed description in the Description edit box. Click on Add when done. The screen for adding a gateway from the interface configuration page is shown here:

There isn't much more you have to do for interface configuration, unless you have some other options you need to enter for your connection (for example, if you need to configure advanced DHCP options, or if you have a PPP or PPPoE connection and must enter a username or password).

  1. You probably should check the Block bogon networks checkbox to block non-IANA-assigned networks. Otherwise, you can click on the Save button at the bottom of the page and, when the page reloads, click on the Apply Changes button at the top of the page.
  2. Repeat this process for as many WAN-type interfaces as you have.

The next step is DNS configuration for each of the new gateways. Perform the following steps:

  1. Navigate to System | General Setup and enter a DNS server for each of the new gateways.

There should be at least one unique DNS server per gateway in a multi-WAN step, as we are looking to eliminate single points of failure in our setup.

  1. You enter the DNS server information by entering a DNS server IP address in one of the edit boxes (for DNS Server 1–4), and then selecting one of the gateways in the adjacent drop-down box, and then clicking on the Add DNS Server button. This process should be repeated for each of the gateways. When you have finished, click on the Save button at the bottom of the page.
  2. Now that we have completed DNS configuration, we can navigate to System | Routing and begin gateway configuration.

 

  1. On the Gateway tab, the newly created gateways should be listed in the table. If you configured the gateways manually in the previous step, they will have whatever names you assigned to them; otherwise, they will have names such as WAN2_DHCP or WAN2_DHCP6, and so on. The screen for editing a gateway from the Gateways tab is shown here:
  1. One thing you will notice if you click on the Edit icon (the pencil) for any of the gateways is that you have many more options than were presented in the dialog box that appears when you click on Add new gateway on the interface configuration page.
  2. There is a Disable this gateway checkbox that allows you to save the gateway configuration while forcing the gateway offline, which can be useful in troubleshooting. 

 

  1. The Interface drop-down box allows you to change the interface being configured; the options for Address Family are IPv4 or IPv6. Gateway allows you to specify the gateway IP address; if the interface is configured to use DHCP or DHCPv6, this field will be read-only.
  2. As in the Add new gateway dialog box,  there is a Default Gateway checkbox which allows you to set this gateway as the default. There is a Disable Gateway Monitoring checkbox, which, if checked, will cause pfSense to consider the gateway to always be up.
  3. There is also a Monitor IP edit box, in which you can enter an alternative IP address to monitor the gateway. To determine whether a gateway is up or not, pfSense will first ping the gateway. Sometimes, however, having the gateway ping a remote address is a better measure of whether the gateway is actually up, so if the gateway fails to respond to a ping and a monitor IP is specified, pfSense will have the gateway ping the monitor IP. You should probably configure this option. To configure Monitor IP, enter a non-local IP address to ping (you can enter the DNS server IP address for the gateway if you can't think of another reliable site to ping).
  4. The Mark Gateway as Down checkbox, if checked allows you to force pfSense to consider the gateway to be down.
  5. There is also a Description field. Also of interest are the options available in the Advanced section (you need to click the Display Advanced button for these options to appear).
  6. The Weight drop-down box allows you to select the weight for the gateway within a gateway group. Higher numbers mean the gateway has more weight. For example, if one gateway has a weight of 2 and another gateway has a weight of 1, the gateway with a weight of 2 will have twice as many connections going through it as the other gateway.
  7. The Data Payload edit box allows us to set the -s parameter of the ping command, which in turn allows us to set the number of data bytes to send. The default is 1.
  8. The Latency thresholds edit boxes determine the low and high thresholds for latency (in milliseconds) in cases where latency is one of the criteria for determining if a gateway is up or down.
  9. The low threshold value sends an alarm, while the high threshold value signifies the gateway is down. The defaults of 200/500 should be fine, although you may want to adjust this if you have a high latency connection (for example, a satellite connection).

The Packet Loss thresholds specify the low and high thresholds for how much packet loss is acceptable before the gateway is considered down in cases where packet loss is one of the criteria for determining if a gateway is functional. Again, the low threshold sends an alarm and the high threshold signifies the gateway is down. The Probe Interval edit box allows you to specify how often (in milliseconds) an ICMP (ping) probe is sent. The default is 500 ms. The Loss Interval is the allowed latency of replies; in other words, the time interval before packets are considered lost. The default is 2000 ms. The Time Period is the period over which results are averaged. The default is 60000 ms. The Alert interval edit box allows you to specify the time period between checking for an alert condition; the default is one second. There is also a Use non-local gateway checkbox which, if checked, allows use of a gateway outside of the interface's subnet.

  1. Most of the settings in Advanced can be kept at their default values, although it is good to know that you can change them if needed. pfSense gateway pings were once hardcoded in such a way that a reply received 5 seconds after a ping request was considered successful, and a ping request was considered successful if at least 1 reply was received from every 5 requests. That means that pfSense tolerated up to 80% packet loss and a very high level of latency before it declared a gateway to be down. These settings undoubtedly eliminated a lot of false positives and flapping (when an interface is alternately advertised as up and down in rapid succession), but it also meant that a gateway could still be up when in practice the packet loss level would be so high that the connection would be unusable. Making these settings configurable allows us to make our own trade-offs. The value you will most likely have to adjust is Latency thresholds, and even then only for certain types of connections. Click on the Save button when you have finished making changes, and from the main Routing page, click on the Apply Changes button.

Now we can set up the actual gateway group for our multi-WAN connection, for which we can refer to the following steps:

  1. Click on the Gateway Groups tab. This tab should display a table with all configured gateways.
  2. Click on the Add button at the bottom of the table on the right to add a new gateway. The screen for adding a gateway group is shown here:

There are only a few options when configuring a gateway group:

  1. In the Group Name edit box, enter the name of the group.
  2. Under Gateway Priority, there are two configurable options. Tier allows you to select the tier on which a gateway exists. With tiers, lower numbers have priority over higher numbers.
  3. Gateways on the same tier level are load balanced with each other, while gateways on a higher tier level only become invoked when all gateways on lower tiers are down.
  4. Thus, if we set WAN_DHCP to Tier 1 and WAN2_DHCP to Tier 2, then WAN_DHCP will get all the traffic for this gateway group, and WAN2_DHCP will not be used at all unless WAN_DHCP is down. But if both are set to Tier 1, then both gateways will be online at the same time and handle a share of the traffic.
This means that we can convert a gateway group from failover to load balancing very easily and vice-versa. 

However, they will be load balanced. Since this is what we want, you should set all the gateways in the group to Tier 1.

  1. The Virtual IP drop-down box allows you to select which virtual IP to use for each gateway. As this only applies to cases where the gateway group is used as an endpoint for a local Dynamic DNS (DDNS), IPsec or OpenVPN connection, you can leave it set to Interface Address.
  2. The Trigger Level drop-down box allows you to specify when to trigger exclusion of a gateway member. The choices are as follows:
    • Member down: Member is excluded when it fails to respond to a ping attempt, or when it fails to ping the monitor IP.
    • Packet Loss: Member is excluded when packet loss is unacceptably high (the actual percentage value is set in Advanced in the gateway configuration).
    • High Latency: Member is excluded when latency is unacceptably high (the actual threshold is set in Advanced in the gateway configuration).
    • Packet Loss or High Latency: Member is excluded when either packet loss or high latency becomes unacceptably high.
  3. Finally, in the Description field, you can enter a brief non-parsed description. Click on Save when you are finished making changes, and on the main Routing page, click on Apply Changes.

You may also want to configure failover groups for each of the gateways, the following are the steps to be performed:

  1. Again, click on the Add button on the Gateway Groups tab.
  2. Type an appropriate name in the Group Name field (for example, FAILOVER1). In the Gateway priority Tier drop-down box, set the first WAN connection to Tier 1 and the second WAN connection to Tier 2.
  3. Set the desired trigger level in the Trigger Level drop-down box, type an appropriate Description (for example, WAN1 failover), and then click on the Save button.
  4. On the Gateway Groups tab, click on Add once again (or, to make life even easier, click on the Copy icon in the table entry for the first failover group, which will make a new gateway group with all values defaulting to the first failover group options), and configure another gateway group with an appropriate Group Name, only this time, set the second WAN connection to Tier 1 and the first WAN connection to Tier 2.

 

  1. When you are finished making changes, click on the Save button. On the Gateway Groups page, click on Apply Changes.

The gateway group is now configured, but without corresponding firewall rules, no network traffic will be directed through the group. You could create a rule for each interface that will be using the group, but if the group is going to be used on more than one network, it will be easier to create a floating rule. Refer to the following steps:

  1. Navigate to Firewall | Rules and click on the Floating tab. Scroll down to the bottom of the table and click on one of the Add buttons to add a new rule.
  2. The objective of this rule is to pass traffic, so the Action drop-down box should be left at its default value of Pass.
  3. In the Interface listbox, you should select every interface that will be using the gateway group.
  4. In the Direction drop-down box, select out (rules involving gateways can only be one-way rules).
  5. In the Protocol drop-down box, select any. The Source field should be an alias which refers to all the interfaces selected in the Interface listbox (or any if you want to keep it simple); the Destination field can be left as any.
  6. Then scroll down to Advanced Options (in the Extra Options section) and click on the Show Advanced button.
  7. Scroll down further, and the third last option will be Gateway. In the Gateway drop-down box, select the newly created gateway group.
  8. Then click on Save button at the bottom of the page. 

Although it is easier to set up a floating rule to direct traffic to the gateway group, if you want to do per-interface policy-based routing, do not let this hinder you. The following are the steps:

  1. Click on the tab for the interface for which you want to set the policy, then click on one of the Add buttons.
  2. If you just want to make a general rule directing all interface traffic to a specific gateway, the only default values you have to change are Protocol (change this from TCP to any), and Gateway (found by clicking on the Advanced Options button and then scrolling down).
  3. Change Gateway to the interface to which you want to direct traffic. If your policies are more granular, make the necessary changes so the rule only matches the traffic you want to direct to the gateway.

 

  1. Enter a brief Description for each rule and then click on Save and click on Apply Changes.
  2. Even though you are making specific rules for each interface, you may want to also have a floating rule as a fallback, so that all traffic that doesn't match the policy-based routing rules goes to the gateway group. If so, make sure the Quick option on the floating rule is disabled; otherwise the floating rule will take precedence over the policy rules. As always, make sure the ordering of the rules is correct.
  3. You still need to create rules for the two failover gateway groups.
  4. Click on the Copy icon in the table entry for the new rule, and create a new rule for each of the failover groups.
  5. All you need to change is the Description field and the gateway in the Gateway drop-down box.
  6. Click on Save when you have finished creating each rule, and when you have created all the necessary rules, click on Apply Changes on the main floating rules page.

Under most circumstances, you will not have to configure static routes, because when a route needs to change due to a gateway going offline, pfSense automatically creates a temporary static route to re-route traffic. There may be some cases, however, where you need to create static routes. One such case is with traffic that comes directly from pfSense, such as services, ping requests, and so on. Policy-based routing will work with traffic that enters a pfSense network interface from the outside, but it cannot be applied to traffic that originates from pfSense because it cannot be tagged for alternate routing. The solution is to set up a static route for such traffic. To do so, refer to the following steps:

  1. Navigate to System | Routing and click on the Static Routes tab.
  2. Click on the Add button to add a new route. On the Edit page for the route, choose the Destination network for the route (IP address and CIDR).
  3. In the Gateway drop-down box, select the correct gateway (there is a link provided that enables you to add a new gateway, if necessary).
  4. There is a Disable this static route checkbox, in case we need to do this, and in the last field, you may enter a brief Description.
  5. Click on the Save button when you are finished making changes and then click on the Apply Changes button.  The screen for adding a static route is shown here: 
  1. There are two options available from System | Advanced settings that are of interest in configuring gateway groups.
  2. If you click on the Miscellaneous tab, there are two options in the Gateway Monitoring section.
  3. The State Killing on Gateway Failure checkbox, if checked, will cause all states to be flushed when a gateway goes down. Otherwise, active states from the gateway that is now down will be transferred to other gateways in the gateway group. This may be undesirable if you don't want persistent connections to be transferred in such a way.
  4. The Skip rules when gateway is down checkbox, when checked, will change the default behavior regarding what happens to a rule specifying a gateway when the gateway is down.

By default, the rule will be created without the gateway which is down. This option changes that behavior so that if a rule specifies a gateway which is down, the rule is not created.

If you completed these steps, your gateway group should be up and running, but you will probably want to check to make sure it is functioning correctly. To do so the following steps will be helpful:

  1. Navigate to Status | Gateways. There are two tabs on the Gateways page: Gateways and Gateway Groups. The Gateways tab has more useful information about configured gateways.

 

  1. On this tab, there is a table showing all configured gateways. The meaning of the Name, Gateway, Monitor and Description fields are obvious, but there are also the following fields which convey crucial information about the gateways:
    • RTT (Round Trip Time): The ping round trip time in milliseconds, averaged over the calculation interval
    • RTTsd (Round Trip Time standard deviation): New to pfSense 2.3, this is the standard deviation of the round trip time over the calculation interval
    • Loss: Packet loss over the calculation interval
    • Status: Either online or offline
  1. You can test the gateway group monitoring by unplugging each of the WAN interfaces in turn, and seeing how long it takes for this page to report the gateway as offline.
  2. If the amount of time it takes is unacceptably high, you may have to adjust the Trigger Level setting in Gateway Groups or adjust the Latency threshold or Packet Loss threshold in the gateway settings.
..................Content has been hidden....................

You can't read the all page of ebook, please click here login for view all page.
Reset