Chapter 23

Network Performance Anxiety

In This Chapter

Understanding performance problems

Looking at bottlenecks

Developing a procedure for solving performance problems

Monitoring performance

Implementing other tips for speeding up your network

The term network performance refers to how efficiently the network responds to users’ needs. Obviously, any access to resources that involves the network is slower than similar access that doesn’t involve the network. For example, opening a Word document that resides on a network file server takes longer than opening a similar document that resides on the user’s local hard drive. However, it shouldn’t take much longer. If it does, you have a network performance problem.

This chapter is a general introduction to the practice of tuning your network so that it performs as well as possible. Keep in mind that many specific bits of network tuning advice are scattered throughout this book. In this chapter, you can find some specific techniques for analyzing your network’s performance, taking corrective action when a performance problem develops, and charting your progress.

Why Administrators Hate Performance Problems

Network performance problems are among the most difficult network problems to track down and solve. If a user simply can’t access the network, it usually doesn’t take long to figure out why: The cable is unplugged, a network card is malfunctioning, or the user doesn’t have permission to access the resource, for example. After you do a little investigating, the problem usually reveals itself, and you fix it and move on to the next problem.

Unfortunately, performance problems are messier. Here are just a few reasons that network administrators hate performance problems:

Performance problems are difficult to quantify. Exactly how much slower is the network now than it was a week ago, a month ago, or even a year ago? Sometimes the network just feels slow, but you can’t quite define exactly how slow it really is.

Performance problems usually develop gradually. Sometimes a network slows down suddenly and drastically. More often, though, the network gradually gets slower, a little bit at a time, until one day its users notice that the network is slow.

Performance problems often go unreported. Users gripe about the problem to each other around the water cooler, but they don’t formally contact you to let you know that the network seems 10 percent slower than usual. As long as they can still access the network, they just assume that the problem is temporary or that they’re imagining a problem.

Many performance problems are intermittent. Sometimes a user calls you and complains that a certain network operation has become slower than molasses, and by the time you get to that person’s desk, the operation performs in a snap. Sometimes you can find a pattern to the intermittent behavior, such as it’s slower in the morning than in the afternoon or it’s slow only while backups are running or while the printer is working. At other times, you can’t find a pattern: Sometimes the operation is slow, and sometimes it isn’t.

Performance tuning isn’t an exact science. Improving performance sometimes involves educated guesswork. Will upgrading all users from 100 Mbps to 1 Gbps improve performance? Probably. Will segmenting the network improve performance? Maybe. Will adding another 8GB of RAM to the server improve performance? Hopefully.

The solution to a performance problem is sometimes a hard sell. If a user can’t access the network because of a malfunctioning component, the purchase of a replacement is usually undeniably justified. However, if the network is slow and you think that you can fix it by upgrading the entire network to Gigabit Ethernet, you may have trouble selling management on the upgrade.

What Exactly Is a Bottleneck?

The term bottleneck doesn’t refer in any way to the physique of the typical computer geek. Rather, computer geeks coined the phrase when they discovered that the tapered shape of a bottle of Jolt cola limited the rate at which they could consume the beverage. “Hey,” a computer geek said one day, “the gently tapered narrowness of this bottle’s neck imposes a distinct limiting effect upon the rate at which I can consume the tasty caffeine-laden beverage contained within. This observation draws to mind a hitherto undiscovered yet obvious analogy to the limiting effect that a single slow component of a computer system can have upon the performance of the system as a whole.”

“Fascinating,” replied all the other computer geeks, who were fortunate enough to be present at that historic moment.

The term stuck and is used to this day to draw attention to the simple fact that a computer system is only as fast as its slowest component. It’s the computer equivalent of the old truism that a chain is only as strong as its weakest link.

For a simple demonstration of this concept, consider what happens when you print a word-processing document on a slow printer. Your word-processing program reads the data from disk and sends it to the printer. Then you sit and wait while the printer prints the document.

Would buying a faster CPU or adding more memory make the document print faster? No. The CPU is already much faster than the printer, and your computer already has more than enough memory to print the document. The printer itself is the bottleneck, so the only way to print the document faster is to replace the slow printer with a faster one.

Here are some other, random thoughts about bottlenecks:

A computer system always has a bottleneck. Suppose that you decide that the bottleneck on your file server is a slow 10,000 RPM SCSI disk drive, so you replace it with a fast 15,000 RPM drive. Now the hard drive is no longer the bottleneck: The drive can process information faster than the controller card to which the disk is connected. You didn’t really eliminate the bottleneck — you just moved it from the hard drive to the disk controller. No matter what you do, the computer will always have a component that limits the overall performance of the system.

One way to limit the effect of a bottleneck is to avoid waiting for the bottleneck. For example, print spooling lets you avoid waiting for a slow printer. Although spooling doesn’t speed up the printer, it frees you to do other work while the printer chugs along. Similarly, disk caching lets you avoid waiting for a slow hard drive.

The Five Most Common Network Bottlenecks

Direct from the home office in sunny Fresno, California, here are the top five most common network bottlenecks, in no particular order.

The hardware inside your servers

Your servers should be powerful computers capable of handling all the work your network will throw at them. Don’t cut corners by using a bottom-of-the-line computer that you bought at a discount computer store.

The following list describes the four most important components of your server hardware:

Processor: Your server should have a powerful processor. Any processor that’s available in a $500 computer from a low-cost general appliance store is generally not a processor that you want to see in your file server. In other words, avoid processors designed for consumer-grade home computers.

Memory: You can’t have too much memory. Memory is cheap, so don’t skimp. Don’t even think about running a server with fewer than 4GB of RAM.

Disk: Don’t mess around with inexpensive IDE hard drives. To have a respectable system, you should have nothing but SCSI or SAS (Serially Attached SCSI) drives.

Network interface: A $9.95 network card may be fine for your home network, but don’t use one in a file server that supports 100 users and expect to be happy with the server’s performance. Remember that the server computer uses the network more often than any clients do. Make sure your servers are equipped with good high-speed network interfaces.

The server’s configuration options

All network operating systems (NOSes) have options that you can configure. Some of these options can make the difference between a pokey network and a zippy network. Unfortunately, no hard-and-fast rules exist for setting these options. Otherwise, you wouldn’t have options.

The following important tuning options are available for most servers:

Virtual memory options: Virtual memory refers to disk paging files that the server uses when it doesn’t have enough real memory to do its work. Few servers ever have enough real memory, so virtual memory is always an important server feature. You can specify the size and location of the virtual memory paging files.

For the best performance, provide at least as much virtual memory as your computer has real memory. For example, if your server has 16GB of real memory, allocate at least 16GB of virtual memory. If necessary, you can increase this size later.

Disk striping: Use the disk defragmenter to optimize the data storage on your server’s disks.

If the server has more than one hard drive, you can increase performance by creating striped volumes, which allow disk I/O operations to run concurrently on each of the drives in the stripe set.

Network protocols: Make sure that your network protocols are configured correctly and remove any protocols that aren’t necessary.

Free disk space on the server: Servers like to have plenty of breathing room on their disks.

If the amount of free disk space on your server drops precipitously low, the server chokes up and slows to a crawl. Make sure that your server has plenty of space — a few gigabytes of unused disk space provides a healthy buffer.

Servers that do too much

One common source of network performance problems is a server overloaded with too many duties. Just because a modern network operating system comes equipped with dozens of different types of services doesn’t mean that you should enable and use them all on a single server. If a single server is bogged down because of too much work, add a second server to relieve the first server of some of its chores. Remember the old saying: “Many hands make light work.”

For example, if your network needs more disk space, consider adding a second file server rather than adding another drive to the server that already has four nearly full drives. Better yet, purchase a file server appliance dedicated to the task of serving files.

As a side benefit, your network will be easier to administer and more reliable if you place separate functions on separate servers. For example, if a single server doubles as a file server and a mail server, you lose both services if you have to take down the server to perform an upgrade or repair a failed component. However, if you have separate file and mail server computers, only one of the services is interrupted if you have to take down one of the servers.

The network infrastructure

The infrastructure consists of the cables and any switches, hubs, routers, and other components that sit between your clients and your servers.

The following network infrastructure items can slow down your network:

Hubs: If you have old network hubs rather than switches, replace them immediately. Do not pass go, do not collect $200.

Segment sizes: Keep the number of computers and other devices on each network segment to a reasonable number. About 20 devices is usually the right number. (Note that if you replace your hubs with switches, you instantly cut the size of each segment because each port on a switch constitutes a separate segment.)

The network’s speed: If you have a really old network, you may discover that many — if not all — of your users are still working at 10 Mbps. Upgrading to 100 Mbps speeds up the network dramatically. Upgrading to 1 Gbps speeds it up even more.

The backbone speed: If your network uses a backbone to connect segments, consider upgrading the backbone to 1 Gbps.

The hardest part about improving the performance of a network is determining where the bottlenecks are. With sophisticated test equipment and years of experience, network gurus can make good educated guesses. Without the equipment and experience, you can still make good uneducated guesses.

Malfunctioning components

Sometimes a malfunctioning network card or other component slows down the network. For example, a switch may malfunction intermittently, occasionally letting packets through but dropping enough of them to slow down the network. After you identify the faulty component, replacing it restores the network to its original speed.

Tune Your Network the Compulsive Way

You can tune your network in one of two ways. The first is to think about it a bit, take a guess at an approach that may improve performance, try that approach, and see whether the network seems to run faster. This strategy is the way most people go about tuning the network.

You can also try the compulsive way, which is suitable for people who organize their sock drawers by color and their food cupboards alphabetically by food group. The compulsive approach to tuning a network goes something like this:

1. Establish a method for objectively testing the performance of some aspect of the network.

In this method, you create a benchmark. The result of your benchmark is a baseline.

2. Change one variable of your network configuration and rerun the test.

For example, you may think that increasing the size of the disk cache can improve performance. Change the cache size, restart the server, and run the benchmark test. Note whether performance improves, stays the same, or becomes worse.

3. Repeat Step 2 for each variable that you want to test.

Here are some salient points to keep in mind if you decide to tune your network the compulsive way:

If possible, test each variable separately. In other words, before proceeding, reverse the changes you made to other network variables.

Write down the results of each test so that you have an accurate record of the effect that each change makes on your network’s performance.

Be sure to change only one aspect of the network each time you run the benchmark. If you make several changes, you don’t know which one caused the change. One change may improve performance, but the other change may worsen performance so that the changes cancel each other out — kind of like offsetting penalties in a football game.

If possible, conduct the baseline test during normal working hours, when the network is undergoing its normal workload.

To establish the network’s baseline performance, run the benchmark test two or three times to make sure that the results are repeatable.

Monitoring Network Performance

One way to monitor network performance is to use a stopwatch to see how long it actually takes to complete common network tasks, such as opening documents or printing reports. If you choose to monitor your network by using the stopwatch technique, you’ll want to get a clipboard, baseball cap, and gray sweat suit to complete the ensemble.

A more high-tech approach to monitoring network performance is to use a monitor program that automatically gathers network statistics for you. After you set up the monitor, it plugs away, silently spying on your network and recording what it sees in performance logs. You can then review the performance logs to see how your network is doing.

For large networks, you can purchase sophisticated monitoring programs that run on their own dedicated servers. For small- and medium-sized networks, you can probably get by with the built-in monitoring facilities that come with the network operating system. For example, Figure 23-1 shows the Performance Monitor tool that comes with Windows Server 2012. Other operating systems come with similar tools.

Figure 23-1: Monitoring performance.

Windows Performance Monitor lets you keep track of several different aspects of system performance at once. You track each performance aspect by setting up a counter. You can choose from dozens of different counters. Table 23-1 describes some of the most commonly used counters. Note that each counter refers to a server object, such as physical disk, memory, or the processor.

Table 23-1 Commonly Used Performance Counters

Object	Counter	Description
Physical Disk	% Free Space	Percentage of free space on the server’s physical disks. Should be at least 15%.
Physical Disk Length	Average Queue	Indicates how many disk operations are waiting while the disk is busy servicing other disk operations. Should be two or fewer.
Memory	Pages/second	Number of pages retrieved from the virtual memory page files per second (pps). A typical threshold is about 2,500 pps.
Processor	% Processor Time	Indicates the percentage of the processor’s time that it’s busy doing work rather than sitting idle. Should be 85% or less.

Here are a few more things to consider about performance monitoring:

Performance Monitor enables you to view real-time data or to view data that you can save in a log file. Real-time data gives you an idea about what’s happening with the network at a particular moment, but the more useful information comes from the logs.

You can schedule logging to occur at certain times of the day and for certain intervals. For example, you may schedule the log to gather data every 15 seconds from 9:00 to 9:30 every morning and then again from 3:00 to 3:30 every afternoon.

Even if you don’t have a performance problem now, you should set up performance logging and let it run for a few weeks to gather baseline data. If you develop a problem, this baseline data will prove invaluable while you research the problem.

  Don’t leave performance logging turned on all the time. Gathering performance data slows down your server. Use it only occasionally to gather baseline data or when you’re experiencing a performance problem.

More Performance Tips

Here are a few last-minute performance tips that barely made it in:

You can often find the source of a slow network by staring at the network hubs or switches for a few minutes. These devices have colorful arrays of green and red lights. The green lights flash whenever data is transmitted; the red lights flash when a collision occurs. An occasional red flash is normal, but if one or more of the red lights is flashing repeatedly, the network interface card (NIC) connected to that port may be faulty.

Check for scheduled tasks, such as backups, batched database updates, or report jobs. If at all possible, schedule these tasks to run after normal business hours, such as at night when no one is in the office. These jobs tend to slow down the network by hogging the server’s hard drives.

Sometimes, faulty application programs can degrade performance. For example, some programs develop a memory leak. They use memory but then forget to release the memory after they finish. Programs with memory leaks can slowly eat up all the memory on a server, until the server runs out and grinds to a halt. If you think a program has a memory leak, contact the manufacturer of the program to see whether a fix is available.

Spyware can slow a system to a crawl. A common source of performance problems on client computers is spyware, those annoying programs that you almost can’t help but pick up when you surf the Internet. Fortunately, you can remove spyware with a variety of free or inexpensive spyware removal tools. For more information, use Google or another search engine to search for spyware removal.