Squid

Squid is a caching and forwarding web proxy which was originally designed to run as a daemon on Unix-like systems (Linux, FreeBSD, and so on). Version 1.0.0 was released in July 1996, and as of today, Squid is capable of running on over a dozen different Unix variants. It is also capable of being used as a client-side cache (allowing the client to cache web pages or other content), and as a reverse proxy. If it is being used as a reverse proxy, it is being used server side to cache pages from one or more web servers.

Proxy servers were quite commonplace in the days when dialup internet access was common. The reason for this is obvious. If you have at best a 56 Kbps internet connection, then it is faster to retrieve a local copy of a web page than it would be to acquire the remote copy of the page. As broadband internet connections have become more commonplace, proxy servers have become less commonplace. If you have a fast enough internet connection, using a proxy server might actually take longer than it would take to retrieve the page directly. The reason for this is obvious if you consider what happens when you request a web page through a proxy. First, you make a request to the proxy. Next, the proxy must check to see if there is a newer version of the web page available. If there is a newer version, it must retrieve the page, cache it, and send it to the client. If not, it will send a copy of the cached page.

If the page has been updated, then it is pretty obvious that the time required to retrieve the page is longer, since we end up eventually requesting the page from a remote web server, which we would have done if we didn't have a proxy, but now we have added another step to the process. But even if the page has not been updated, there is some overhead associated with checking to see if the page has been updated. Thus, in many cases, using a web proxy is slower than directly accessing websites.

However, there are some situations where using a web proxy can be advantageous. Let's assume that 25 users on your network simultaneously request a web page (imagine a classroom-type situation, or perhaps one in which users are simultaneously reading online documentation for a product). If there is no proxy, then 25 separate requests for the same web page are consuming your bandwidth. If, however, you have a web proxy and a cached local copy, the page can be sent to your network users at LAN speed instead of internet speed, and without consuming any of your internet bandwidth.

To begin Squid configuration, once it is installed, follow these steps:

  1. Navigate to Services | Squid Proxy Server.
  2. There are ten separate tabs on this page, but the default tab is General. From this tab, you should be able to get Squid up and running.
  3. The Enable Squid Proxy checkbox controls whether the Squid proxy is active; checking it activates Squid.
  4. The Keep Settings/Data checkbox, if checked, preserves settings and data across package reinstalls.
  5. The Proxy Interfaces list box allows you to select to which interfaces the proxy server will bind.
  1. The Proxy Port edit box allows you to specify on which port the proxy server will listen; the default is 3128.
  2. The ICP Port edit box allows you to specify the port on which the proxy server will send and receive ICP queries to and from neighbor caches.
  3. By default, such queries are not allowed. The Allow Users on Interface checkbox, if checked, will allow users connected to interfaces selected in the Proxy Interfaces list box to use the proxy.
  4. The Resolve DNS IPv4 First checkbox enables forcing DNS IPv4 lookup first:
Configuring general settings in Squid

Starting with version 1.1.9, Squid supports using ICMP round trip time (RTT) measurements to select the best location to forward a cache miss.

Checking the Disable ICMP checkbox, however, disables the Squid ICMP pinger helper, forcing Squid to rely on ICP reply times in determining where to forward cache misses.

Finally, the Use Alternate DNS Servers for the Proxy Server edit box allows you to specify DNS servers other than the servers configured in the pfSense DNS forwarder/resolver.

If you check the Transparent HTTP Proxy checkbox, pfSense will forward all requests that have port 80 as their destination to the Squid proxy server without any additional configuration being necessary.

In the Transparent Proxy Interface(s) list box, you can choose the interfaces on which Squid will transparently intercept requests.

The Bypass Proxy for Private Address Destination checkbox, if checked, will cause Squid to not forward traffic to private address space (10.x.x.x, 172.16.x.x to 172.31.x.x and 192.168.x.x) addresses.

The Bypass Proxy for These Source Addresses edit box allows you to specify IPs, networks, hostnames, or aliases for which the proxy server will not be invoked if they are the source; instead, these hosts will be able to pass directly through the firewall.

The Bypass Proxy for These Destination IPs edit box allows you to specify IPs, networks hostnames or aliases to which the proxy server will not be invoked if they are the destination.

The next section is SSL Man In the Middle Filtering. The HTTPS/SSL Interception checkbox, if checked, will enable SSL filtering.

If this option is not enabled, then port 443 traffic will not be filtered by Squid even if Squid is being run in transparent mode. The SSL Intercept Interface list box allows you to select the interfaces on which Squid will intercept SSL requests. The SSL Proxy port edit box allows you to specify the port on which the proxy will listen to intercept SSL. The CA drop-down box allows you to select a Certificate Authority (CA) to use when SSL interception is enabled. You should install the CA certificate as a trusted root CA on each computer on which you want to filter SSL to avoid an SSL error on each connection. The SSL Certificate Daemon Children edit box allows you to specify the number of SSL certificate daemon children to start. You may need to increase this if Squid is going to be used in busy environments. The Remote Cert Checks list box allows you to select which remote SSL certificate checks to perform on SSL traffic. Finally, the Certificate Adapt list box allows you to pass SSL certificate information on to users in order to allow the end user to make an informed decision on whether to trust a server certificate.

The Enable Access Logging checkbox, if checked, will enable the access log. This, of course, will use up even more disk space, so you should not enable this unless you have enough disk space. The Log Store Directory edit box allows you to specify where the logs will be stored. The Rotate Logs edit box allows you to choose how many days of log files will be kept. By default, rotation is disabled. The Log Pages Denied By SquidGuard checkbox, if checked, will make it possible for pages denied by Squid to be included in the logs.

The next section of the page is called Headers Handling, Language and Other Customizations. This section includes such options as setting the hostname and email address to display on error pages, and the ability to suppress the Squid version string in HTTP headers and HTML error pages (this can be useful if you don't want end users to know which Squid version is being used). By clicking the Show Advanced button, we can see a number of advanced options. The Integrations list box allows you to add Squid options added from packages such as SquidGuard. The Custom ACLS (Before Auth) list box allows you to put custom options that will be added to the configuration before Squid processes the authentication access control list (ACL) lines. Any options places in the Custom ACLS (After Auth) line will be executed after the ACL lines.

There are several other tabs with options. The Remote Cache tab allows you to specify remote Squid proxy servers from which web pages can be forwarded, which can help reduce network latency and incorporate redundancy into the network. Clicking on the Add button allows you to add configuration information about a remote server. Remote caches can be configured hierarchically. The three options for cache hierarchy are as follows:

  • Parent: This is typically a more distant cache, such as your ISP's cache.
  • Sibling: Closer caches are usually configured as siblings. You can have more than one sibling, and Squid will query them simultaneously.
  • Multicast: You can cut down on network traffic by setting up a multicast address for a cache.

Note that the difference between a parent and a sibling is, to some extent, academic. When Squid queries more than one cache, it does not query each cache in sequence, but instead sends all ICP queries at the same time. Squid will get the page from the fastest-responding cache. The designation of a cache as a parent is significant because Squid will go to a parent cache when there is no response from a sibling cache; when there is no response from parent caches, by default Squid will attempt to go directly to the origin server.

Multicast can be used to increase the efficiency of cache requests. Unlike unicast (one-to-one communication) and broadcast (one-to-everyone on the subnet), multicast packets are one-to-many, and unlike broadcast packets, they can traverse network segments. This can be helpful in a scenario where multiple caches are being used. For example, if your cache hierarchy has four caches, to find out if a web page is cached, a host would normally have to query each of the four caches, which consumes bandwidth. If you configure a multicast address, however, the host can just send one packet to the multicast address. Once the packet reaches the local subnet, each cache can pick up the packet and reply. This cuts down considerably on traffic between networks.

There are also several options for determining which cache is selected:

  • Default: The first peer to respond to an ICP query is used as the source.
  • Round-robin: This is a simple load balancing method. Squid maintains a counter for each cache. The cache with the lowest counter is used (and its counter is incremented).
  • Weighted-round-robin: Parent caches are used in round-robin fashion, with each cache having its own counter, but caches with lower RTTs are given greater weight.
  • CARP: Not to be confused with Common Address Redundancy Protocol, this CARP refers to Cache Array Routing Protocol. This method entails taking the URL requested and feeding it into a hash function that generates large numbers. These numbers all fit in a certain range, and by dividing the range by however many caches we have, we can send the request to one of the caches based on this hash value.
  • Userhash: Similar to CARP, but the hashing is done based on the client proxy_auth or ident username.
  • Sourcehash: The hash function takes as input the client source IP.
  • Multicast-siblings: If the peer is of type multicast, you can use this option.

Also of interest on this page is the ICP Settings section. In the ICP Port edit box, you can specify a port to connect to the upstream proxy with the ICP protocol. The default value is 7, which disables ICP communication. In the ICP Options drop-down box, you can select an ICP mode for the cache being configured:

  • no-query: Do not allow ICP queries from this cache to the remote cache
  • multicast-responder: The peer being configured is a member of a multicast group
  • closest-only: For ICP_IP_MISS replies (the cache did not have the page requested), we'll only forward CLOSEST_PARENT_MISSes (parent with lowest RTT) and never FIRST_PARENT_MISSes (fastest weighted RTT)
  • background-ping: Only send ICP queries to this neighbor infrequently

The Local Cache tab, as the name implies, controls settings for the Squid cache on the local firewall. The Cache Replacement Policy drop-down box allows you to select from amongst several cache replacement policy options:

  • Least Recently Used (LRU): This algorithm discards the least recently used items first.
  • Greedy Dual Size Frequency (Heap GDSF): This algorithm keeps smaller popular objects in the cache at the expense of larger popular objects.
  • Least Frequently Used with Dynamic Aging (Heap LFUDA): This algorithm keeps popular objects in the cache regardless of their size. Thus, a large popular object may keep smaller objects out of the cache.
  • Least Recently Used (Heap LRU): Algorithm implemented with a heap.

There are also several options on this page for controlling the hard disk cache size, as well as the cache location and the threshold at which cache replacement occurs (the point at which the cache replacement policy is invoked to evict certain items from the cache). You can also select the hard disk cache system, and you can force a wiping of the cache on this page if necessary.

The Antivirus tab allows you to use Squid in conjunction with Clam Antivirus (ClamAV). Clam AV is a free and open source antivirus program licensed under the GNU GPL. The Enable checkbox, if checked, enables ClamAV. The Client Forward Options drop-down box allows you to choose what client information to forward to ClamAV. The Enable Manual Configuration drop-down box allows you to select manual configuration mode, which causes ClamAV to ignore any options set on this tab; instead, it uses the configuration settings from the configuration files (squidclamav.conf, c-icap.conf, c-icap.magic, freshclam.conf, and clamd. conf). These configuration files can be edited from this page by scrolling down and clicking on the Show Advanced button.

The Redirect URL edit box allows you to specify a URL to send users to when a virus is found. If no URL is specified here, the default Squid/pfSense web GUI error URL is used. Checking the Google Safe Browsing checkbox enables Google Safe Browsing. The Google Safe Browsing database includes information about harmful websites, such as websites that may be phishing sites or sources of malware. It should be noted that this option consumes a significant amount of RAM. The Exclude Audio/Video Streams checkbox, if checked, will disable antivirus scanning of streamed video and audio.

The ClamAV Database Update drop-down box allows you to select the database update interval. If you are using Google Safe Browsing, the interval should be set to one hour. You can also click on the Update AV button to update the database now. You can also schedule updates with the cron daemon. The Regional ClamAV Database Update Mirror drop-down box allows you to select a regional database mirror. Finally, the Optional ClamAV Database Update Servers edit box allows you to specify additional ClamAV databases (separated by semicolons).

The ACLs tab is where you can configure the access control lists (ACLs) for Squid. The Allowed Subnets list box is where you can enter subnets that are allowed to use the proxy. The interfaces specified on the General tab in the Proxy Interfaces list box do not have to have their subnets added. However, if you want to add subnets  other than the subnets of the interfaces selected in Proxy Interfaces, you can add them here.

The Unrestricted IPs list box allows you to add unrestricted IP addresses and/or networks. These entries will not be subject to the access control directives specified on the ACLs tab. The Banned Hosts Addresses list box allows you to enter IP addresses and/or networks that will not be allowed to use the proxy. The Whitelist list box allows you to enter domains that will be accessible to users, while the Blacklist list box allows you to enter domains that will be blocked for proxy users. The Block User Agents edit box allows you to enter user agents that will be blocked for proxy users. This field is useful if you want to prevent users on your network from using certain types of software (for example, torrent clients). The Block MIME Types (Reply Only) list box allows you to enter MIME types that will be blocked for users that use the proxy. This is useful for blocking JavaScript, among other types.

The next section on the page is Squid Allowed Ports. The ACL Safe Ports edit box allows you to enter ports on which traffic will be allowed to pass, in addition to the default list of ports (the default ports are 21, 70, 80, 210, 280, 443, 488, 563, 591, 631, 777, 901, and 1025-65535). The ACL SSL Ports edit box allows you to specify ports on which SSL connections will be allowed, in addition to the default list (443 and 563).

The Traffic Mgmt tab allows you to have some degree of control over your users' bandwidth consumption. The Maximum Download Size and Maximum Upload Size edit boxes allow you to set the maximum total download and upload sizes (in kilobytes) respectively. Of particular interest is the last section of the page: Squid Transfer Quick Abort Settings. By default, Squid continues downloading aborted requests which are almost done downloading. This may be undesirable in some cases (for example, slow links).

Users may repeatedly request and abort downloads, thus tying up file descriptors and bandwidth. The parameters in this section allow you to control under what circumstances a transfer continues or is aborted. Thus, we get the Finish transfer if less than x KB remaining, Abort transfer if more than x KB remaining, and Finish transfer if more than x % finished edit boxes.

By default, proxy users are not required to provide authentication, but if you want to require authentication, you can do so on the Authentication tab. The Authentication Method drop-down box allows you to select what form of authentication takes place. The default is None, but Local is supported (authentication through Squid), as well as LDAP, RADIUS, Captive Portal and NT Domain. You can also specify subnets that will not be asked for authentication to access the proxy in the Subnets That Don't Need Authentication list box.

If you selected Local as the Authentication Method on the Authentication tab, you will have to enter users via the Users tab. Click on the Add button to add a user. There are fields for Username, Password, and a Description field where you can enter a brief non-parsed description.

The Real Time tab allows you to view information about Squid as it is running. You can view the access logs here, the cache logs, SquidGuard logs, and ClamAV logs. You can also filter the logs using the options in the Filtering section. You can specify the number of lines that will be displayed, and you can also filter the results by typing a regular expression into the String filter edit box.

The Sync tab allows you to do an XMLRPC sync of the Squid proxy to another firewall. There are three options in the Enable Sync drop-down box:

  • Do not sync this package configuration: This is the default; no sync is performed
  • Sync to configured system backup server: Sync to the CARP backup firewall (or firewalls)
  • Sync to host(s) defined below: Allows you to specify the hosts to which Squid will sync

The Sync Timeout drop-down allows you to choose the XMLRPC timeout (the default is 250 seconds). The Replication Targets subsection allows you to specify the IP address and/or hostname and port of hosts to which Squid will sync. You can also specify a Replication Protocol (either HTTP or HTTPS), as well as an Admin Password. Click on the Add button to add a host. You also need to check the Enable checkbox to enable replication.

..................Content has been hidden....................

You can't read the all page of ebook, please click here login for view all page.
Reset