ApacheBench is a utility that ships with Apache. You will find it in the bin directory of where you built Apache. The file is called ab, and you can get the command-line syntax by typing ./ab -h while in that directory. ApacheBench enables you to benchmark how fast your Apache server runs. It is actually a client application, not closely tied to Apache itself, so you can run it against any Web server, not just Apache.

To run a test with ApacheBench, you can type a command such as the following:

./ab -n 50 http://www.apacheadmin.com/testing/url.html

This will fetch that URL 50 times and display a report of the process; giving times required to connect and to transfer the content, and the transfer rate.

The -c flag can be added to make several requests concurrently, to simulate a busy site where a number of users are requesting content at the same time.

./ab -n 500 -c 10 http://www.apacheadmin.com/testing/url.html

For CGI programs, you may want to simulate user behavior by posting form contents to the program. This behavior is invoked with the -p argument:

./ab -n 100 -p /tmp/formcontents http://www.apacheadmin.com/cgi-bin/example.cgi

In the above example, the specified file contains the data that you want posted to the given URL.

Profiling, on the other hand, is much more difficult, and is somewhat beyond the scope of this book. Profiling means getting into your code—such as CGI programs, or mod_perl handlers, for example—and determining which part of the code is taking the longest. Or, more generally, how long each part of the code is taking. This permits you to determine which parts of the code are being the least efficient, and therefore could stand a little optimization.

Profiling techniques are very dependent on the particular programming language in use, but usually involve using a debugger of some variety. Most modern programming languages have a profiler of some description available for this sort of analysis.

You need enough memory (RAM) so that you never have to swap. If you run out of memory, and data has to be written to swap space, things will slow down substantially because disk access is orders of magnitude slower than memory.

Note that the more Apache children you have running at one time, the more memory you will be using. The MaxClients directive limits the total number of Apache child processes you can have running at any one time, and you should make sure that this is not set so high that you run out of memory.

Because most of your data will load off of a disk, you should try to get the fastest drive that you can. SCSI drives will, in general, be faster than the IDE alternative. They are also, of course, more expensive.

Placing drives in a RAID configuration, in addition to giving you data redundancy in the event of a hard-drive failure, will also (usually) give you improved data-access speed because the data can be loaded off of multiple drives simultaneously, improving the data throughout.

A faster processor will improve the performance of your Web server. This seems fairly obvious and will be even more the case if more of your content is dynamically generated in some fashion.

Each time a request is made to your Apache server the access is logged in the server access log, (see Chapter 24). Part of the information that is stored is the address of the client accessing your site. This is, by default, just the IP address of that client. However the HostnameLookups directive, if set to On, causes Apache to log the full hostname of the client, rather than just the IP address.

HostnameLookups On

With this setting a DNS lookup will be performed each time a client requests a file from your site, which will cause Apache to pause until the hostname has been looked up. In the event that the client is connecting from a host that does not have a reverse DNS entry, which is increasingly common, you might have to wait through the entire DNS timeout before the file is served.

If you need to have host information in your logs, you should consider processing your logs in a separate process to convert the IP addresses into full hostnames. Apache provides a program that performs this process. It is called logrotate, and is located in the bin directory when you have finished building Apache. To use logrotate, copy your log file to a location other than where it was created, and run the command

/usr/local/apache/bin/logresolve < access_log > output_file

Note that this can take a very long time. It's a good idea to do this on some machine other than your production Web server machine, so that the DNS lookups don't interfere with server performance.

Symbolic links, or symlinks, are a feature of most modern file systems that permit a file (or directory) to be located in one location, but appear to also be in another location. Because Windows file systems do not permit symbolic links, this section really only applies to the *nix readers.

Most file operations make no distinction between symlinks and “real” files (or directories), so Apache actually has to do more work when you tell it not to follow symlinks. That is, using Options +FollowSymLinks causes a performance improvement because Apache does not need to check to see if the files are really symbolic links to files located elsewhere.

The exception to this rule is if you are using Options SymLinksIfOwnerMatch. In which case, not only does Apache have to check if they are symlinks, but it must also verify that the owner of the link is the same as the owner of the file. That requires a total of three file stats—one to see if it is a symlink, then, if it is one, it must check the owner of the file, and of the link. Note that Apache will have to check every file, and all the directories leading up to that file, to see if they are symlinks. And because this lookup is not cached, it has to do this for every resource that is requested.

For the best performance, always use Options FollowSymLinks, and never use Options SymLinksIfOwnerMatch.

Refer to Chapter 4, “Configuration Directives,” for more information about the FollowSymLinks option and for information about the SymLinksIfOwnerMatch option.

As was discussed in Chapter 6, “.htaccess files—Per-Directory Configuration,” permitting per-directory configuration files, or .htaccess files, can cause a substantial performance degradation because every file access requires that the file .htaccess be looked for in the directory where the content is located, and all parent directories thereof.

Setting AllowOverride off disables the parsing of .htaccess files entirely, and will give you a performance improvement as a result.

Content negotiation results in a performance hit because Apache has to consider more than one option of files to serve, compare the available files to the preferences of the client, and choose the right file. However, the benefits of content negotiation outweigh the performance loss.

There are two ways in which you can improve performance while using content negotiation.

First, rather than using a wildcard DirectoryIndex, such as index, which will then get expanded to the possible variants of this file, explicitly list all the variants. For example, rather than using the directive

DirectoryIndex index

use a full listing instead, as in:

DirectoryIndex index.html.en index.html.es index.html.fr

Second, use a type-map file, rather than allowing MultiViews to figure things out for you. By taking the time to create the type-map file, and list the possible variants with their attributes, you can save Apache the time of figuring this all out for you. And because it has to do this each time a resource is requested the time investment to create the type-map is well worth your time.

Apache handles increased server load by creating new child processes to handle the additional incoming connections. And, when the number of connections dies down, Apache reaps those unused child processes. Starting up new processes can potentially be expensive, so you should set your configuration so that you usually have as many processes as you need and, therefore, don't need to start and stop processes frequently.

These settings are managed by the MinSpareServers, MaxSpareServers, and StartServers directives, which determine, respectively, the minimum and maximum number of child processes that should be kept waiting in the wings, and how many child processes are to be started when the server comes up in the first place.

The MaxRequestsPerChild directive is also related to this process. It determines how many requests a particular child process should be allowed to serve before it is reaped. The default value of this directive is 0, which means that there is no limit—a particular child should be allowed to run for the entirety of the life of the Apache process. Setting it to anything else causes Apache to kill a child process after it has served that many requests. If you have this set to some non-zero value, make sure that it is sufficiently high that you are not continually starting new processes.

On Microsoft Windows, MaxRequestsPerlChild needs to be set to 0. Because Apache under Windows is a single multithreaded process, and the directives mentioned previously have their analogies in MinSpareThreads, MaxSpareThreads, and StartThreads. Setting MaxRequestsPerChild to something non-zero in this threaded model causes that single process to get shut down, and restarted, taking all the threads with it, and causing momentary server outage while the process comes back up.

The MaxClients directive (or, in a threaded environment, the MaxThreadsPerChild directive) determines the maximum number of connections that will be possible to your server. This should be set high enough that clients will not ever have to wait, but not so high that this many child processes will consume all available memory and cause your server to swap to disk. A lot of experimentation is required to determine the appropriate numbers here. Examining the output of mod_status is very useful in determining these numbers.

Prior to version 2.0 Apache shipped with mod_proxy, which you could build into Apache, and configure as a caching proxy server to run with your Apache server. This has been removed from Apache 2.0 so that it could be developed independently of Apache. mod_proxy is being developed as a separate project under the umbrella of the Apache Software Foundation. Versions are available for Apache 1.3 and for Apache 2.0.

You can read more about mod_proxy in the Apache 1.3 documentation, but you should be aware that that documentation has not been updated for a long time. The latest version of the documentation can be seen in the CVS repository for the mod_proxy project, at http://cvs.apache.org/viewcvs.cgi/httpd-proxy/docs/.

mod_proxy performs two main roles—proxy and cache server. These functions are turned on with the following requests, respectively:

ProxyRequests On
CacheRoot /var/httpd/cache

Turning ProxyRequests On enables the proxying functionality, but does not enable caching because Apache will not know where to put the cached documents. The CacheRoot directive tells Apache where to put the cached content and, therefore, enables the caching part of mod_proxy.

A variety of further configuration tweaks can be made to mod_proxy to get it to do a variety of interesting things. What particular things you do with it will depend largely on your particular network configuration and user needs.

There are a number of alternate ways to do caching. The most popular of these is Squid. Squid, available from http://www.squid-cache.org/, is a caching proxy server. What this means is that it sits at the edge of your network and proxys either incoming, or outgoing requests, making those requests on behalf of the client.

Proxying outgoing connections from your network has two main effects—one immediate, and one longer term.

The first effect is that all outgoing requests from your network, through the proxy, appear to come from a single IP address. This hides the size and layout of your network from the outside world. Many other technologies perform this same task, NAT (Network Address Translation) and SOCKS, being two of the most popular, and provide services at the physical network layer. Squid provides caching at the protocol layer, proxying a variety of protocols including HTTP, FTP, SSL, and DNS.

The second way that Squid benefits your outgoing network connections is that it caches content. This becomes more useful the larger your organization is. The first time someone visits a Web site, Squid retrieves that content for the user, and stores it. The next user that gets that same content will get it from Squid, rather than from the actual Web site, resulting in a much faster retrieval of the content. Squid checks with the live Web site to see if the content has been modified because the last time someone looked at it, and if it has been, it will get the new content, rather than serving the now-stale cached content.

However, more to the point of this chapter is what happens when you use Squid to proxy incoming requests to your Web server. This looks similar to proxying outgoing requests, except that now it directly results in improved performance (or at least perceived improved performance, which amounts to the same thing) of your Web server.

Because Squid caches requests in memory, retiring them to disk after they have reached a certain age, the most frequently requested resources are always immediately available to be served and don't have to wait for a disk read. This frees up the Web server to serve only that content that is being modified—the dynamically generated content—and Squid serves the static HTML and image files from memory cache.

On the user end, this is directly seen as a faster load time. On the server side it is seen as a greatly reduced server load, and capability to focus processor time on generating the dynamic content, rather than spending time reading the same files from disk again and again.

You can read more about Squid, or download the program, at http://www.squid-cache.org.