Before making a final decision on what ISPs to use, you probably want to see each provider's Service Level Agreement (SLA). The SLA is the official commitment between the ISP and the client which defines particular aspects of service that should be provided to the client per the service contract. An ISP is not required to have an SLA, but many do, and many of them make their SLA available on their website.

The SLA may be part of the contract between the ISP and the client, or might not be. Aspects of service defined by the SLA can include such things as throughput, mean time between failures, mean time between recovery, mean time between repair, and uptime, as well as which party is responsible for reporting faults and paying fees. A good SLA will also delineate the process for supervising and monitoring performance levels, who to contact when there are issues, the time frame in which problems will be responded to and resolved, and the consequences for a service provider not fulfilling its SLA obligations. In some cases, an ISP failing to abide by the SLA can lead to the client having the right to terminate the contract or receive a refund for the time during which connectivity is unavailable.

You may be configuring multi-WAN for a business environment; in such cases, T1 service has been held in high regard for many years. The SLA for a T1 connection is generally more favorable than SLAs for other types of connections, and T1 service is seen as more reliable. With pfSense, however, you can achieve a higher level of reliability at a lesser cost. Two low-cost broadband connections in a multi-WAN setup can give you greater bandwidth and the same or greater level of reliability. It is less likely, for example, that two broadband connections will go down at the same time than two T1 connections.

One of the advantages of a multi-WAN implementation is that it makes possible policy-based routing, which is a form of routing in which routing decisions are based on administrative policy rather than some other criteria. In a multi-WAN context, it means that we can divert traffic that matches a firewall rule to a specific gateway. A simple example of this would be directing outbound traffic from one subnet to a certain gateway. We can also direct traffic from a specific host or traffic that uses a certain protocol. You might also want to use policy-based routing to segregate certain services based on their priority. For example, you may have a high-quality connection that has little bandwidth (for example, a T1 connection with 1.544 Mbps), and a low-quality connection with a higher level of bandwidth (a broadband connection). Using policy-based routing, you could route high-priority traffic (for example, VoIP traffic) to the T1 connection, leaving other permitted but low-priority traffic (for example, file downloads) to the broadband connection.

As with server load balancing, multi-WAN provides for two forms of redundancy and high availability: failover and load balancing. With failover, some gateways have priority over other gateways, and only when all higher priority gateways fail do the lower priority gateways come online. Load balancing entails having multiple gateways online at the same time, handling some share of the total WAN traffic. This means that if we want to utilize multiple connections in order to increase our bandwidth, we can, but if we want to use additional connections only when our primary connection goes down, we can do that as well.

One caveat that should be mentioned in connection with bandwidth aggregation is that while we can use multiple connections in a load balancing gateway group to increase overall bandwidth, a single connection will still be routed through a single WAN interface. As a result, while we might have two connections, each providing x Mbps of bandwidth, our total bandwidth may be 2x, an individual connection will still only be able to use x Mbps of bandwidth. If you have applications that use multiple connections (for example, a download client), such an application should be able to take advantage of multiple connections. Of course, if there are multiple users on your network, a load balancing gateway configuration will enable you to take advantage of the additional bandwidth.

There are many different algorithms that could define the behavior of a gateway group; pfSense load balancing uses a round-robin algorithm (requests distributed sequentially). Different weights can be assigned to each gateway; by default, they each handle an equal share of traffic, but if there is a need to change the share of traffic handled by a gateway, we can alter it by changing the weight assigned to it.

A multi-WAN setup requires a means of determining when a gateway is down. The way it is done in pfSense is that each WAN has its own monitor IP. pfSense will ping the monitor IP, and if it stops responding, the gateway is assumed to be down. If the monitor IP is for an OPT WAN interface, then pfSense will automatically add a static route to divert traffic to the correct gateway within the gateway group. Thus, each WAN interface within a gateway group must have a unique monitor IP; WAN interfaces in different gateway groups, however, can have the same monitor IP.

You may wonder what constitutes a ping failure for purposes of gateway monitoring. pfSense uses the following default parameters:

• If packet loss reaches 20% and packet loss is one of the criteria used for determining when a gateway is down, the gateway will go down
• If a packet is sent and does not receive a reply after 2 seconds, the packet is considered lost
• An ICMP probe will be sent every 0.5 seconds
• If latency (the time between when a packet is sent and a reply is received) averages 0.5 seconds and high latency is one of the criteria used for determining when a gateway is down, the gateway will go down

In addition, a gateway will reach alert status (the gateway will remain up but the background of its listing in Status | Gateways changes to yellow) when packet loss reaches 10% or latency reaches 0.2 seconds. If these default settings are unacceptable, you can adjust them by navigating to System | Routing, clicking on the Gateways tab and editing the gateway whose values you wish to change (click on the Advanced button on the Edit page to reveal these settings). We will cover this in greater detail in the next section.