Threads don’t correspond directly to any physical feature of your computer—a program with four threads running on a quad-core computer might end up running one thread on each core, but it doesn’t usually happen that way. For one thing, your program shares the computer with other processes, so it can’t have all the cores to itself. Moreover, one of the main ideas behind threads is to provide an abstraction that’s mostly independent of the real number of processor cores. You are free to have far more threads than cores. It’s the job of the operating system scheduler to decide which thread gets to run on any particular processor core at any one time. (Or, more accurately, which thread gets to run on any particular logical processor—see the sidebar on the next page.)

A machine will usually have lots of threads—a quick glance at the Windows Task Manager’s Performance pane indicates that this machine currently has 1,340 threads. Who’d have thought that writing a book would be such a complex activity? The extent to which this outnumbers the machine’s four CPU cores highlights the fact that threads are an abstraction. They offer the illusion that the computer has an almost endless capacity for executing concurrent tasks.

Threads are able to outnumber logical processors by this margin because most threads spend the majority of their time waiting for something to happen. Most of those 1,340 threads have called operating system APIs that have blocked—they won’t return until they have some information to provide. For example, desktop applications spend most of their time inside a Windows API call that returns messages describing mouse and keyboard input and the occasional system message (such as color scheme change notifications). If the user doesn’t click on or type into an application, and if there are no system messages, these applications sit idle—their main thread remains blocked inside the API call until there’s a message to return. This explains how a quad-core machine can support 1,340 threads while the CPU usage registers as just 1 percent.

When a blocking API is finally ready to return, the thread becomes runnable. The operating system’s scheduler is responsible for deciding which runnable threads get to use which logical processors. In an ideal world, you’ll have exactly enough runnable threads to make full and perfect use of the CPU cycles you’ve paid for. In practice, there’s usually a mismatch, so either one or more logical processors will be idle, or there will be contention for processing resources.

In the latter case—where there are more runnable threads than logical processors—the scheduler has to decide which threads currently most deserve to run. If a thread runs without blocking for a while (typically a few milliseconds) and there are other runnable threads, the OS scheduler may preempt that thread—it interrupts its execution, stores information about what it was doing at the point at which it was preempted, and gives a different thread some CPU time. If a logical processor becomes available later (either because enough threads block or because some other thread was preempted) the OS will put things back to how they were before preemption, and allow it to carry on. The time for which a thread will be allowed to run before preemption is known as a quantum.

The upshot of this is that even if you have more threads than logical processors, and all of the threads are trying to execute code simultaneously, the OS scheduler arranges for all of them to make progress, despite outnumbering the logical processors. This illusion has a price: preempting a thread and scheduling a different thread to use the CPU slows things down, and you’ll often want to use the techniques we’ll see later that try to avoid forcing the scheduler to do this.

Each thread has its own call stack, which means that items that live on the stack—function arguments and local variables—are local to the thread. We can exploit this to simplify Example 16-1, which contains three almost identical loops. Example 16-2 has just one copy of the loop which is shared by all three threads.

using System;
using System.Threading;

class Program
{
    static void Main(string[] args)
    {
        Thread t1 = new Thread(Go);
        Thread t2 = new Thread(Go);

        t1.Start("One");
        t2.Start("Two");

        Go("Main");
    }

    static void Go(object name)
    {
        for (int i = 0; i < 100; ++i)
        {
            Console.WriteLine("{0}: {1}", name, i);
        }
    }
}

The Go method here contains the common loop—it has been modified slightly to take an argument so that each thread can print out either One, Two, or Main as before. Running this produces similar output to the previous example. (It’s not identical, of course, because these examples produce slightly different output every time they run.)

This example illustrates an important point: multiple threads can be inside the same function at any time. All three threads in Example 16-2 spend most of their time inside the Go method. But since each thread gets its own stack, the values of the name argument and the loop variable (i) can be different for each thread.

Information that lives elsewhere is not intrinsically private to one thread. Example 16-3 shows another variation on our example. As with Example 16-2, it uses a common Go method to run a loop on all three threads, but the loop variable (i) is now a static field of the Program class—all three threads share the same variable.

using System;
using System.Threading;

class Program
{
    // Visible to all threads. (Bad, in this example.)
    static int i;

    static void Main(string[] args)
    {
        i = 0;
        Thread t1 = new Thread(Go);
        Thread t2 = new Thread(Go);

        t1.Start("One");
        t2.Start("Two");

        Go("Main");
    }

    static void Go(object name)
    {
        // Modifying shared state without suitable protection - bad!
        for ( ; i < 100; ++i)
        {
            Console.WriteLine("{0}: {1}", name, i);
        }
    }
}

This example has a problem: all three threads will try to read and write the shared field, and things often go wrong when you do this. You might think that with three threads all sharing a single common counter, with each thread incrementing that counter every time they loop and each thread running until the counter hits 100, we’d just see all the numbers from 0 to 99 once. But it’s not quite that simple. For one thing, you might see all three threads print out 0, because they may all get to the point where they’re trying to print out the first value before any of them has gotten as far as trying to increment the counter. (Remember, a for loop executes its iterator clause—the ++i in this example—at the end of each iteration.) Then again you might not see that—it all really depends on when the OS scheduler lets the threads run. But there’s a subtler problem: if two threads both attempt to execute the ++i at the same time, we may see anomalous results—the value of i may end up being lower than the number of times it has been incremented, for example. If you want to share state between threads, you’ll need to use some of the synchronization mechanisms discussed later in this chapter.

Be aware that using local variables is not necessarily a guarantee that the state you’re working with lives on the stack. For example, when using reference types (and most types are reference types) you need to keep in mind the distinction between the variable that contains the reference and the object to which that reference refers. Example 16-4 uses nothing but local variables, but ends up using the same StringBuilder object from all three threads—each thread might have its own local variable to refer to that object, but all three variables refer to the same object.

using System;
using System.Threading;
using System.Text;

class Program
{
    static void Main(string[] args)
    {
        StringBuilder result = new StringBuilder();

        // Sharing the StringBuilder between threads. BAD!
        Thread t1 = new Thread(() => Go(result, "One"));
        Thread t2 = new Thread(() => Go(result, "Two"));

        t1.Start();
        t2.Start();

        Go(result, "Main");

        t1.Join();
        t2.Join();

        Console.WriteLine(result);
    }

    static void Go(StringBuilder sb, string name)
    {
        for (int i = 0; i < 100; ++i)
        {
            // All threads using the same StringBuilder - BAD!
            sb.AppendFormat("{0}: {1}", name, i);
            sb.AppendLine();
        }
    }
}

By the way, you’ll have noticed that this code calls Join on both Thread objects. The Join method blocks until the thread has finished—this code needs to do that because it prints the output only once the threads are done. This is a simple example of coordinating operations across multiple threads. However, it’s not sufficient to avoid problems here. Looking at the output, it’s clear that all is not well. Here are the first few lines, running on a quad-core system:

Main: One: Two: 00
Main:
1

2
MainTwo: 3
Main: 1
2
2
Two: 3One: Two: 4
Two: One: 6Two: One: 7
One: 8
: One: 9

That’s a whole lot more chaotic than the previous examples, which merely scrambled the order of the lines, and lost the odd increment. The reason this has gone more obviously wrong is that with the earlier examples, our attempt to observe the system profoundly changed its behavior. (That happens a lot with multithreaded code.) The calls to Console.WriteLine were imposing some order on the system, because the .NET Framework was forcing the threads to take it in turns when printing their output—that’s why we don’t get lines mixed up with one another. But Example 16-4 does all of its work in memory using a StringBuilder, and since it calls Console.WriteLine just once when it’s done, to print the results, nothing is forcing things to happen in any particular order, and so we can see the chaos in full effect.

Just to be clear, the reason for the chaos is that even though each thread has its own local sb variable held privately on that thread’s stack, they all refer to the same StringBuilder object—we have three references to the same object. All three threads are trying to add output to the same StringBuilder at the same time, and the result is a mess.

You need to be absolutely clear in your head on where the information you’re working with lives, where it came from, and whether other threads might be able to see it. Objects created by a particular thread are OK as long as you never make them visible to other threads—Example 16-4 hit problems because the main thread created a StringBuilder and then arranged for it to be accessible from the other two threads.

This means you need to be especially careful when using nested methods—either anonymous delegates or lambdas—because these provide a way to share local variables between threads. Example 16-5 shows how the problem in Example 16-3 can happen even with value type local variables. It has just one loop count variable (i) shared between all the threads, but it does this without making it a field—it’s a local variable.

using System;
using System.Threading;

class Program
{
    static void Main(string[] args)
    {
        // Visible to all threads, thanks to use of
        // anonymous method. (Bad, in this example.)
        int i = 0;

        ParameterizedThreadStart go = delegate(object name)
        {
            // Modifying shared state without suitable protection - bad!
            for (; i < 100; ++i)
            {
                Console.WriteLine("{0}: {1}", name, i);
            }
        };


        Thread t1 = new Thread(go);
        Thread t2 = new Thread(go);

        t1.Start("One");
        t2.Start("Two");

        go("Main");
    }
}

This example demonstrates that while it may be convenient to think of value type local variables as living on the stack, it’s not always true. Example 16-5 contains an anonymous method that makes use of the local i variable declared by the containing method (Main), so the C# compiler has been obliged to convert that variable into a field inside a generated class, in order to make it possible for that one variable to be used from multiple methods.

To summarize: information that really does live on the stack is private to a particular thread. Unfortunately, using local variables doesn’t necessarily guarantee that the state you’re working with is on the stack. Be wary of reference types—no matter where the reference lives, the thing it refers to will not be on the stack, so you need to understand what other code might have a reference to the object you’re using. Be wary of value types whose implementation you do not control—value types are allowed to contain fields of reference types, so you’re not guaranteed to be safe just because you’re using a value type. And be wary of lambdas and anonymous methods—they can move information off the stack and into a place where it’s accessible to multiple threads at once. We’ll see later what to do if you really have to share information across threads.

The examples we’ve seen so far create threads explicitly in order to illustrate the operation of multiple threads. But .NET often creates threads automatically without you having created Thread objects. The most obvious example is the thread that the .NET Framework calls your Main method on, but there are others—some of the asynchronous communication mechanisms we saw in Chapter 13 call back into your code on different threads than the one you started work on. We’ll be seeing more examples of this later in the chapter when we examine .NET’s Asynchronous Programming Model.

In fact, it’s relatively unusual to create new threads explicitly. If you need concurrent execution and you’re not using some part of the .NET Framework that supplies you with threads when you need them, it’s often better to use the thread pool or the Task Parallel Library, both of which we’ll see later.

One problem with explicit thread creation is in knowing how many to create. Threads are relatively expensive—each one consumes system resources, and there are also factors that can limit the number of threads in a single process. There’s also a cost in switching between threads—the context switch that occurs when the OS scheduler moves a thread from one logical processor to another. If you have many more runnable threads than logical processors, you’ll pay this cost on a very regular basis, and it can start to have a significant effect on throughput. In an ideal world you would have no more threads than logical processors, avoiding any context switch overhead. However, most threads block from time to time, so in reality you tend to need more threads than logical processors if you want to fully use your CPU cycles. In general, you should try to keep the thread count as low as is practical—a single program that creates more than a handful per logical processor is likely to have problems.

Creating just enough threads is often hard, because getting the balance right depends on things such as your application’s current workload, other work in progress on the machine, and characteristics of the machine itself. Fortunately, .NET provides the thread pool to make this sort of thing easier.

The .NET Framework provides a thread pool, which is a collection of worker threads available to perform short pieces of work. The thread pool continuously adjusts the number of threads that are allowed to process work items simultaneously in an attempt to optimize throughput.

The exact algorithm used to adjust the thread count is not documented, but as a general rule, if the system is not busy, work will typically be serviced very quickly after you queue it up. But as the computer becomes busier, items will sit in the queue for longer—the thread pool tries to avoid the overheads of preemption, thread switching, and resource contention by not running too much concurrent work. When a system is already busy, trying to process more work items would probably slow it down further, and so keeping items queued up and processing fewer at a time is likely to result in better overall performance.

The simplest way to use the thread pool is through its QueueUserWorkItem method. Example 16-6 shows a modification to the previous examples—rather than creating threads, it uses the thread pool. QueueUserWorkItem takes a delegate to any method that accepts a single object as its argument, so it’s happy with the same Go method as Example 16-2. (Unlike the Thread constructor, there’s no overload that accepts a method without an argument—the thread pool insists on there being an argument whether you have a use for one or not.)

static void Main(string[] args)
{
    ThreadPool.QueueUserWorkItem(Go, "One");
    ThreadPool.QueueUserWorkItem(Go, "Two");

    Go("Main");
    // Problem: not waiting for work items to complete!
}

This example has a problem: if the main thread finishes first, the program may exit before the thread pool work items complete. So this only illustrates how to start work on the thread pool. This might not be a problem in practice—it depends on your application’s typical life cycle, but you may need to add additional code to coordinate completion. Running on a quad-core machine, this particular example behaves in much the same way as the previous ones, because the thread pool ends up creating a thread for both work items. On a single-core machine, you might see a difference—it could decide to let the first item run to completion and then run the second afterward.

The thread pool is designed for fairly short pieces of work. One of the most important jobs it was originally introduced for was to handle web requests in ASP.NET web applications, so if you’re wondering how much work constitutes “fairly short,” a reasonable answer is “about as much work as it takes to generate a web page.”

.NET 4 introduces a new way to use the thread pool, the Task Parallel Library, which offers a couple of advantages. First, it handles certain common scenarios more efficiently than QueueUserWorkItem. Second, it offers more functionality. For example, tasks have much more comprehensive support for handling errors and completion, issues Example 16-6 utterly fails to address. If the main thread finishes before either of the work items is complete, that example will simply exit without waiting for them! And if you want the main thread to discover exceptions that occurred on the thread pool threads, there’s no easy way to do that. If any of these things is important to you, the Task Parallel Library is a better way to use the thread pool. There’s a whole section on that later in this chapter. For now, we’ll continue looking at some aspects of threading that you need to know, no matter what multithreading mechanisms you may be using.

Not all threads are equal. Some work can be done on only certain threads. For example, WPF and Windows Forms both impose a similar requirement: an object representing something in the user interface must be used only on the thread that created that object in the first place. These objects have thread affinity, meaning that they belong to one particular thread.

If you never write multithreaded code, you never have to worry about thread affinity—if you do everything on one thread, it will always be the right one. But as soon as multiple threads get involved—either explicitly or implicitly^[41]—you may need to add code to get things back on the right thread.

ASP.NET has a similar problem. It makes contextual information about the current request available to the thread handling the request, so if you use multiple threads to handle a single request, those other threads will not have access to that contextual information. Strictly speaking, this isn’t a thread affinity issue—ASP.NET can use different threads at different stages of handling a single request—but it presents the same challenge to the developer: if you start trying to use ASP.NET objects from some random thread, you will have problems.

The .NET Framework defines a solution that’s common to WPF, Windows Forms, and ASP.NET. The SynchronizationContext class can help you out if you find yourself on the wrong thread when using any of these frameworks. Example 16-7 shows how you can use this in an event handler for a GUI application—the click handler for a button, perhaps.

SynchronizationContext originalContext = SynchronizationContext.Current;

ThreadPool.QueueUserWorkItem(delegate
    {
        string text = File.ReadAllText(@"c:	emplog.txt");

        originalContext.Post(delegate
        {
            myTextBox.Text = text;
        }, null);
    });

The code reads all the text in from a file, and that’s something that might take awhile. Event handlers in WPF and Windows Forms are called on the thread that the event source belongs to—a UI thread. (Or the UI thread if, like most desktop applications, you have only one UI thread.) You should never do slow work on a UI thread—thread affinity means that if your code is busy using that thread, none of the UI elements belonging to that thread will be able to do anything until you’re finished. The user interface will be unresponsive for as long as you keep the thread busy. So Example 16-7 uses the thread pool to do the work, keeping the UI thread free.

But the code wants to update the UI when it has finished—it’s going to put the text it has retrieved from the file into a text box. Since a text box is a UI element, it has thread affinity—we can update it only if we’re on the UI thread. This is where SynchronizationContext comes to the rescue.

Before starting the slow work, Example 16-7 reads the SynchronizationContext class’s Current property. This static property returns an object that represents the context you’re in when you read it—precisely what that means will depend on what UI framework you’re using. (The object you get back works differently depending on whether your application uses WPF, Windows Forms, or ASP.NET.) But the exact implementation doesn’t matter—you just need to hold on to it until you need to get back to that context. Having grabbed the context while we were in the click handler, we then kick off the work in the thread pool. And once that work is complete, it calls the stored SynchronizationContext object’s Post method. Post takes a delegate, and it’ll invoke that delegate back in whatever context you were in when you grabbed the context. So in this case, it’ll invoke our delegate on the UI thread that the button belongs to. Since we’re back on our application’s UI thread, we’re now able to update the text box.

There are some persistent myths surrounding threads, which sometimes lead to their overuse. As the current trend seems to be for the number of logical processors in typical machines to edge ever upward, developers sometimes feel practically obliged to write multithreaded code. Since using threads correctly is difficult and error-prone, it’s worth tackling some of these myths, in case you’re in a situation where a single-threaded solution might actually be better.

Myth: Maxing the CPU must mean we’re going really fast

It’s possible to construct parallel solutions to problems that manage to use all the available CPU time on all logical processors, and yet which proceed more slowly than single-threaded code that does the same job on one logical processor. Figure 16-1 shows the CPU load reported by the Windows Task Manager for two different solutions to the same task.

The image on the left might make you feel that you’re making better use of your multicore system than the one on right. The righthand side is using far less than half the available CPU capacity. But measuring the elapsed time to complete the task, the code that produced the lefthand image took about 15 times longer to complete than the code that produced the righthand one!

The job in hand was trivial—both examples just increment a field 400 million times. Example 16-8 shows both main loops. The Go function is the one that gets invoked concurrently on four threads. GoSingle just runs multiple times in succession to perform the iterations sequentially.

Figure 16-1. Using all logical processors (left) versus just one (right)

Example 16-8. Multithreaded versus single-threaded

class Program
{
    static int Count;
    const int Iterations = 100000000;

    static void Go()
    {
        for (int i = 0; i < Iterations; ++i)
        {
            Interlocked.Increment(ref Count);
        }
    }

    static void GoSingle(int repeat)
    {
        for (int outer = 0; outer < repeat; ++outer)
        {
            for (int i = 0; i < Iterations; ++i)
            {
                Count += 1;
            }
        }
    }

    ...

Here’s the code that launches Go concurrently:

Count = 0;
List<Thread> threads = (from i in Enumerable.Range(0, 4)
                        select new Thread(Go)).ToList();

threads.ForEach(t => t.Start());
threads.ForEach(t => t.Join());

This creates four threads, all of which call Go. Next, it calls Start on each of the threads. Having started them all, it calls Join on each thread to wait for them all to complete. (We could have written three loops here, but our use of LINQ and lambdas makes the code much more compact. In particular, if you have a loop that invokes just one operation on every item in a list the List<T> class’s ForEach method is a less cluttered way of expressing this than a foreach loop.) The code to launch the single-threaded version is a lot simpler:

Count = 0;
GoSingle(4);

Both produce the same result: the Count field contains 400,000,000 at the end. But the multithreaded version was much slower. One reason is the difference in how the two versions increment the Count. Here’s the line in question from the single-threaded code:

Count += 1;

But if we try that in the multithreaded version, it doesn’t work. It certainly makes it run nice and quickly—about three times faster than the single-threaded version—but trying it a few times, Count comes to 110,460,151, then 133,533,503, then 133,888,803... The majority of increments are getting lost—that’s the sort of thing that happens when you don’t use suitable protection for accessing shared state. That’s why the multithreaded version needs to do this:

Interlocked.Increment(ref Count);

The Interlocked class in the System.Threading namespace provides methods that perform certain simple operations in ways that work even if multiple threads try to use them on the same data at the same time. As the name suggests, Increment increments the count, and it does this in a way that locks out any other logical processors attempting to do the same thing at the same time. It forces the logical processors to take it in turns.

It works—the Count total is correct with this code in place—but at some cost. On a quad-core system, with all four cores burning away at 100 percent, this takes 15 times longer than the simple single-threaded solution.

In fact, the cost of Interlocked.Increment does not fully explain the difference. Modifying the single-threaded version to work the same way makes it run about five times slower, but that’s still three times faster than the multithreaded code. So a considerable amount of the slowdown is down to the communication costs between the processors.

Don’t take these numbers too seriously. This is clearly a contrived example. (If we wanted it to run really fast we could just have initialized Count to 400,000,000 to start with, and removed the loop.) But while the details are spurious the basic principle applies broadly: the cost of contention between logical processors that are supposed to be cooperating can work against you. Sometimes they merely erode the benefit—you might see a 2.5x speedup on a quad-core system, for example. But sometimes they really do negate the benefit—contrived though this example may be, much worse examples have cropped up in real systems.

Note

Some implementations may come out worse on some systems and better on others. For example, some parallel algorithms take a considerable hit relative to their sequential counterparts in order to be able to scale well on more processors. Such an algorithm might make sense only on systems where you have a large number of processors—it might be slower than the single-threaded version on a dual-core system, but very worthwhile on a system with 16 logical processors, for example.

The bottom line is that if you want to understand whether a parallel solution is effective, you need to compare it against a single-threaded solution on the same hardware. Just because your CPU loads indicate that you’re being extremely parallel, that’s not necessarily the same as being really fast. And unless your code will only ever run on a single hardware configuration, you need to perform the same comparison on lots of different machines to get a good idea of how often a parallel solution might be the best choice.

Even when multithreaded code provides a demonstrable performance benefit, it’s very hard to get right. We’ve already seen a few bizarre behaviors in some extremely simple examples. Achieving correct behavior in a real concurrent system can be very challenging. So we’ll look at two of the most common classes of pitfalls before examining some strategies for avoiding them.

There are several approaches for mitigating the difficulties of multithreading, each with a different trade-off between difficulty and flexibility.

The most widely used synchronization primitive in .NET is the monitor. It is supported directly by the .NET Framework—any object can be used with this facility—and also by C#, which provides a special keyword for working with monitors. Monitors offer both mutual exclusion and notification.

The simplest use of a monitor is to ensure that threads take it in turns to access shared state. Example 16-9 shows some code that would need the kind of protection a monitor can provide before we could use it from multiple threads. It is designed for handling lists of recently used strings—you might use this sort of code to provide a recently used file list on an application’s File menu. This code makes no attempt to protect itself in the face of multithreading.

class MostRecentlyUsed
{
    private List<string> items = new List<string>();
    private int maxItems;

    public MostRecentlyUsed(int maximumItemCount)
    {
        maxItems = maximumItemCount;
    }

    public void UseItem(string item)
    {
        // If the item was already in the list, and isn't the first
        // item, remove it from its current position, since we're
        // about to make it this first item.
        int itemIndex = items.IndexOf(item);
        if (itemIndex > 0)
        {
            items.RemoveAt(itemIndex);
        }

        // If the item's already the first, we don't need to do anything.
        if (itemIndex != 0)
        {
            items.Insert(0, item);

            // Ensure we have no more than the maximum specified
            // number of items.
            if (items.Count > maxItems)
            {
                items.RemoveAt(items.Count - 1);
            }
        }
    }

    public IEnumerable<string> GetItems()
    {
        return items.ToArray();
    }
}

Example 16-10 is some test code to exercise the class.

const int Iterations = 10000;

static void TestMru(MostRecentlyUsed mru)
{
    // Initializing random number generator with thread ID ensures
    // each thread provides different data. (Although it also makes
    // each test run different, which may not be ideal.)
    Random r = new Random(Thread.CurrentThread.ManagedThreadId);
    string[] items = { "One", "Two", "Three", "Four", "Five",
                       "Six", "Seven", "Eight" };
    for (int i = 0; i < Iterations; ++i)
    {
        mru.UseItem(items[r.Next(items.Length)]);
    }
}

Example 16-10 just feeds in strings from a fixed set of items at random. Calling this test function from just one thread produces the expected results: at the end, the MostRecentlyUsed object just returns the most recent items put into it by this test. However, the multithreaded test in Example 16-11 causes something quite different to happen.

MostRecentlyUsed mru = new MostRecentlyUsed(4);

const int TestThreadCount = 2;
List<Thread> threads = (from i in Enumerable.Range(0, TestThreadCount)
                        select new Thread(() => TestMru(mru))).ToList();

threads.ForEach(t => t.Start());
threads.ForEach(t => t.Join());

foreach (string item in mru.GetItems())
{
    Console.WriteLine(item);
}

This example crashes on a multicore machine—after awhile, it throws an ArgumentOutOfRangeException. It doesn’t crash at the same place on every run; it crashes inside either of the two calls to the List<T> class’s RemoveAt method.

The exceptions occur due to races. For instance, consider this line of code from Example 16-9:

items.RemoveAt(items.Count - 1);

This reads the value of the Count property, then subtracts 1 to get the index of the last item in the list, and then removes that last item. The race here is that some other thread may manage to remove an item from the list in between this thread reading the Count property and calling RemoveAt. This causes the method to throw an ArgumentOutOfRangeException, because we end up asking it to remove an item at an index that’s after the final item.

In fact, we’re lucky we got an exception at all. The List<T> class makes no guarantees when we use it from multiple threads. Here’s what the documentation for the class says in the Thread Safety section:

This means that it’s our problem to make sure we never try to use a List<T> instance from more than one thread at a time. It could fail in subtler ways than crashing—it could corrupt data, for example.

We could add similar documentation to our MostRecentlyUsed class, declaring that it does not make any guarantees either. In fact, that might well be the best option—it’s very difficult for an individual class to guarantee to work correctly in all possible multithreading scenarios. Only the application that uses the class really knows what constitutes correct behavior. For example, it might be necessary for a MostRecentlyUsed object to be kept in sync with some other object, in which case the application is going to have to manage all synchronization itself, and there’s nothing useful that our class could do on its own. This is one reason why the lack of thread safety guarantees is so widespread in the class libraries—there isn’t a good general-purpose definition of thread-safe for individual types.

If we decide to make it the application’s problem, how would that look? We don’t have a real application here, only a test, so our test code would need to synchronize its calls into our object. Example 16-12 shows a suitably modified version of the test method from Example 16-10. (Note that Example 16-11 adds code that also uses the same object, so you might think we need to make a similar modification there. However, it waits until all the test threads have finished before touching the object, so its reads won’t overlap with their writes, making locking superfluous. Therefore, Example 16-12 is sufficient in this case.)

static void TestMru(MostRecentlyUsed mru)
{
    Random r = new Random(Thread.CurrentThread.ManagedThreadId);
    string[] items = { "One", "Two", "Three", "Four", "Five",
                       "Six", "Seven", "Eight" };
    for (int i = 0; i < Iterations; ++i)
    {
        lock (mru)
        {
            mru.UseItem(items[r.Next(items.Length)]);
        }
    }
}

The only modification here is to wrap the call to the MostRecentlyUsed type’s UseItem method with a lock block. The C# lock syntax generates some code that uses the Monitor class, along with some exception handling. Here’s what the lock block in Example 16-12 is equivalent to:

MostRecentlyUsed referenceToLock = mru);
bool lockAcquired = false;
try
{
    Monitor.Enter(referenceToLock, ref lockAcquired);
    mru.UseItem(items[r.Next(items.Length)]);
}
finally
{
    if (lockAcquired)
    {
        Monitor.Exit(referenceToLock);
    }
}

(This is what C# 4.0 generates. Older versions do something slightly simpler that mishandles an obscure and unusual failure mode. But the basic idea is the same in either case. The generated code copies the mru reference into a separate variable to ensure correct operation even if the code inside the lock block were to change mru.)

The documentation says that Monitor.Enter acquires an exclusive lock on the object passed as the first argument, but what exactly does that mean? Well, the first thread to do this will find that Monitor.Enter returns immediately. Any other threads that try to make the same call on the same object will be made to wait—Monitor.Enter on those other threads will not return until the thread that currently owns the lock releases it by calling Monitor.Exit. Only one thread can hold the lock at any time, so if multiple other threads are waiting for the same object’s lock in Monitor.Enter, the .NET Framework will pick just one as the next owner of the lock, and the other threads will continue to be blocked.

Use of monitors is entirely a matter of convention—it is up to you to decide which objects’ locks you use to protect which information. Example 16-12 happens to acquire a lock on the very object whose state is being protected, but in fact, it’s quite common—preferable, even—to create separate objects whose only job is to be the thing on which you acquire a lock. There are a couple of reasons for this. First, you’ll often want multiple pieces of data to fall under the protection of a single lock—perhaps updates to our MostRecentlyUsed object need to be done in conjunction with changes to other state within the application, such as a history-tracking service. When multiple objects are involved, arbitrarily choosing one of those objects to act as the lock target is likely to make your code harder to follow, because it may not be clear to someone reading your code that you’re using that object’s lock to protect multiple objects rather than just the one whose lock you’re acquiring. If you create a special object whose only purpose is locking, this makes it clear to anyone reading your code that she needs to think about what state that lock protects.

The other reason to avoid acquiring a lock on the object you want to synchronize access to is that you can’t always be sure that the object doesn’t acquire a lock on itself—some developers write lock(this) inside instance methods when trying to make thread-safe objects (for whatever definition of thread-safe they have chosen). It is a bad practice to acquire a lock on your this reference, of course, because you can’t necessarily know whether someone using your object will decide to try to acquire a lock on it for his own purposes—internal locking is an implementation detail, but your this reference is public, and you don’t usually want implementation details to be public.

So in short, you shouldn’t try to acquire a lock on any object you are trying to synchronize access to.

Bearing all that in mind, what should we do if we want to try to make our MostRecentlyUsed class more robust in multithreaded environments? First, we need to decide what multithreading scenarios we want to support. Simply declaring that we want the type to be thread-safe is meaningless.

So let’s say that we want to allow multiple threads to call UseItem and GetItems simultaneously without causing exceptions. That’s a pretty weak guarantee—notice we’ve said nothing about what state the object will be in afterward, merely that it won’t actually blow up. Surely it would be better to guarantee to handle the calls in the order in which they were made. Unfortunately, we can’t do this if the locking logic lives entirely inside the class. The OS scheduler might decide to preempt a thread moments after it called UseItem, and before it has had a chance to get to any of our synchronization code.

For example, consider what could happen if Thread A calls UseItem, and then before that call returns, Thread B also calls UseItem, and before either returns, Thread C calls GetItems. It’s fairly easy to think of at least five reasonable outcomes. GetItems might return neither of the items passed in by A and B. It might return both, and there are two ways it could do this—GetItems returns an ordered list and either A or B might come first. Or it could return just one—either the one passed by A or just the one passed by B. If you need coordination across multiple calls like this it’s not going to be possible to do that inside MostRecentlyUsed, because you only have the opportunity to start synchronization work once calls are already underway. This is another reason why synchronization code usually belongs at the application level and not in the individual objects. So about the best we can hope to achieve within this class is to prevent the exceptions from occurring when it’s used from multiple threads. Example 16-13 does this.

class MostRecentlyUsed
{
    private List<string> items = new List<string>();
    private int maxItems;
    private object lockObject = new object();

    public MostRecentlyUsed(int maximumItemCount)
    {
        maxItems = maximumItemCount;
    }

    public void UseItem(string item)
    {
        lock (lockObject)
        {
            // If the item was already in the list, and isn't the first item,
            // remove it from its current position, since we're about to make
            // it this first item.
            int itemIndex = items.IndexOf(item);
            if (itemIndex > 0)
            {
                items.RemoveAt(itemIndex);
            }

            // If the item's already the first, we don't need to do anything.
            if (itemIndex != 0)
            {
                items.Insert(0, item);

                // Ensure we have no more than the maximum specified
                // number of items.
                if (items.Count > maxItems)
                {
                    items.RemoveAt(items.Count - 1);
                }
            }
        }
    }

    public IEnumerable<string> GetItems()
    {
        lock (lockObject)
        {
            return items.ToArray();
        }
    }
}

Notice that we added a new field, lockObject, which holds a reference to an object whose only job is to be the thing on which we acquire a lock. And we simply acquire this lock inside the methods that work with the list of items. We have to hold the lock for the whole of the UseItem method, because the code looks at the state of the items list right at the start, and then the rest of its operation is guided by what it found. The code simply won’t work if the items list changes halfway through, and so we hold on to the lock for the duration.

In this particular case, holding the lock for the whole method is unlikely to cause problems because this method won’t take long to run. But as a general rule, you want to avoid holding locks for any longer than necessary. The longer you hold a lock, the greater the chances of some other thread wanting to acquire the same lock, and being forced to wait. It’s a particularly bad idea to call code that might make a request over a network and wait for a response while you’re holding a lock (e.g., holding a lock while making a request to a database).

As we mentioned several pages ago, the Monitor class isn’t just about locking. It also provides a form of notification.

class WaitForIt
{
    private bool canGo;
    private object lockObject = new object();

    public void WaitUntilReady()
    {
        lock (lockObject)
        {
            while (!canGo)
            {
                Monitor.Wait(lockObject);
            }
        }
    }

    public void GoNow()
    {
        lock (lockObject)
        {
            canGo = true;
            // Wake me up, before you go go.
            Monitor.PulseAll(lockObject);
        }
    }
}

class Program
{
    static void Main(string[] args)
    {
        WaitForIt waiter = new WaitForIt();

        ThreadStart twork = delegate
        {
            Console.WriteLine("Thread running...");
            Thread.Sleep(1000);
            Console.WriteLine("Notifying");
            waiter.GoNow();
            Console.WriteLine("Notified");
            Thread.Sleep(1000);
            Console.WriteLine("Thread exiting...");
        };

        Thread t = new Thread(twork);
        Console.WriteLine("Starting new thread");
        t.Start();
        Console.WriteLine("Waiting for thread to get going");
        waiter.WaitUntilReady();
        Console.WriteLine("Wait over");

    }
}

Starting new thread
Waiting for thread to get going
Thread running...
Notifying
Notified
Wait over
Thread exiting...

Monitors are very useful and are typically a good first choice for locking, but there are some scenarios in which more specialized alternatives might offer slightly better performance or greater flexibility. Since it’s relatively unusual to use these alternatives, we won’t illustrate them in detail—this section will just describe what’s there and when it might be useful to you.

To understand why the alternatives exist, it’s useful to know something more about the capabilities and limitations of monitors. Monitors are designed for use within a single appdomain—you cannot use them to synchronize operations between processes, nor between appdomains sharing a process. Cross-process coordination is possible in Windows, but you need to use other mechanisms to do that. One reason for this is that monitors try to avoid getting the OS scheduler involved in locking where possible.

If your code hits a lock statement (or calls Monitor.Enter directly) for an object on which no other thread currently has a lock, the .NET Framework is able to handle this situation efficiently. It does not need to make calls into the operating system. Monitors can do this because they’re appdomain-local; to ensure cross-appdomain synchronization typically means getting help from the OS, and once you need to call into the OS, lock acquisition becomes many times more expensive. So when there’s no contention, monitors work really well. But once blocking occurs—either due to contention or because of an explicit call to Wait—the OS scheduler has to get involved because it’s the only thing that can move a thread between runnable and blocked states. This is usually a good thing, because it means the thread can wait efficiently; once a thread becomes blocked it doesn’t consume CPU cycles, and the CPU is free to get on with other useful work, or it may be able to go into its power-efficient idle state when no threads need to run, which is particularly important for laptops running on battery power. However, there are some situations where the cost of getting the OS involved outweighs the benefits. This brings us to the first of the specialized lock types.

Monitors are not just for locking, of course—they offer coordination facilities through pulsing and waiting. And the .NET Framework offers some slightly more specialized types for coordination too.

Events provide a very similar service to the WaitForIt class we built in Example 16-14—an event is effectively a Boolean variable you can wait on. And rather than being a simple one-shot mechanism as in Example 16-14, an event can go back and forth between its two states.

.NET offers ManualResetEvent and AutoResetEvent classes. The latter automatically reverts to its default state when letting waiting threads go, whereas the manual one remains in its so-called signaled state until you explicitly reset it.

These types are wrappers around the underlying Windows event synchronization primitive, so as with Mutex, you can use events for cross-process coordination. And of course, this also means that you incur the cost of getting the OS scheduler involved.

.NET 4 introduces an alternative called ManualResetEventSlim. This will use busy-waiting techniques similar to a spinlock for short waits. So just like the Monitor, it gets the scheduler involved only when a wait is necessary. Therefore, unless you really need the extra features available from the nonslim version (e.g., you need cross-process synchronization) the ManualResetEventSlim class is a better choice than ManualResetEvent if you’re using .NET 4 or later.

.NET 4 introduces the CountdownEvent class, which provides a handy solution to a fairly common problem: knowing when you’re done. Remember back in Example 16-6, we ran into an issue with the thread pool. We queued up a couple of pieces of work, but we had no way of knowing when they were done. One solution to that would be to use the Task Parallel Library, which we’ll get to shortly, but an alternative would have been to use the CountdownEvent class.

CountdownEvent is very simple. For each piece of work you start, you call AddCount (or if you know how many pieces of work there will be up front, you can pass that number into the constructor). For each piece of work that completes you call Signal. And if you need to wait for all outstanding work to complete (e.g., before your program exits), just call Wait.

The System.Collections.Concurrent namespace provides various collection classes that are designed to be used in multithreaded environments. They look a little different from the normal collection classes because they are designed to be used without needing any locking, which means they can’t offer features that rely on things staying consistent from one moment to the next. Numerical indexing is out, for example, because the number of items in the collection may change, as we saw when trying to use List<T> in a multithreaded fashion in Example 16-11. So these are not thread-safe versions of normal collection classes—they are collections whose APIs are designed to support multithreaded use without the caller needing to use locks.

BlockingCollection is not just a multithreaded collection; it also offers associated coordination. It provides a way for threads to sit and wait for items to become available in the collection. Its Take method will block if the collection is empty. Once data becomes available, Take will return one item. Any number of threads may be waiting inside Take at any time, and other threads are free to call Add. If you Add enough items that all the threads waiting to Take are satisfied, and then you keep calling Add, that’s when items start to get added to the collection. And if the collection is nonempty, calls to Take will return immediately.

This allows you to have one or more threads dedicated to processing work items generated by other threads. The BlockingCollection acts as a kind of buffer—if you generate items faster than you process them, they will queue up in the BlockingCollection, and if the processing threads catch up, they will block efficiently until more items come along.

You could use this in a WPF application that needs to do slow work in the background—the UI thread could add work into a blocking collection, and then one or more worker threads could take items from the collection and process them. This is not hugely different from using the thread pool, but it gives you the opportunity to limit the number of worker threads—if you had just a single thread that performs background work, you might be able to get away with much simpler synchronization code, because all your background work is always done by the same thread.

We’ve looked at how to create threads explicitly and at the tools available for ensuring that our programs function correctly in a multithreaded world. Next we’re going to look at asynchronous programming models, where we don’t explicitly create new threads. We will need the locking and synchronization techniques we just explored because we are still working in a concurrent world; it’s just a slightly different programming style.

The Asynchronous Programming Model (APM) is a pattern that many asynchronous APIs in the .NET Framework conform to. It defines common mechanisms for discovering when work is complete, for collecting the results of completed work, and for reporting errors that occurred during the asynchronous operation.

APIs that use the APM offer pairs of methods, starting with Begin and End. For example, the Socket class in the System.Net.Sockets namespace offers numerous instances of this pattern: BeginAccept and EndAccept, BeginSend and EndSend, BeginConnect and EndConnect, and so on.

The exact signature of the Begin method depends on what it does. For example, a socket’s BeginConnect needs the address to which you’d like to connect, whereas BeginReceive needs to know where you’d like to put the data and how much you’re ready to receive. But the APM requires all Begin methods to have the same final two parameters: the method must take an AsyncCallback delegate and an object. And it also requires the method to return an implementation of the IAsyncResult interface. Here’s an example from the Dns class in System.Net:

public static IAsyncResult BeginGetHostEntry(
    string hostNameOrAddress,
    AsyncCallback requestCallback,
    object stateObject
)

Callers may pass a null AsyncCallback. But if they pass a non-null reference, the type implementing the APM is required to invoke the callback once the operation is complete. The AsyncCallback delegate signature requires the callback method to accept an IAsyncResult argument—the APM implementation will pass in the same IAsyncResult to this completion callback as it returns from the Begin method. This object represents an asynchronous operation in progress—many classes can have multiple operations in progress simultaneously, and the IAsyncResult distinguishes between them.

Example 16-16 shows one way to use this pattern. It calls the asynchronous BeginGetHostEntry method provided by the Dns class. This looks up the IP address for a computer, so it takes a string—the name of the computer to find. And then it takes the two standard final APM arguments—a delegate and an object. We can pass anything we like as the object—the function we call doesn’t actually use it, it just hands it back to us later. We could pass null because our example doesn’t need the argument, but we’re passing a number just to demonstrate where it comes out. The reason the APM offers this argument is so that if you have multiple simultaneous asynchronous operations in progress at once, you have a convenient way to associate information with each operation. (This mattered much more in older versions of C#, which didn’t offer anonymous methods or lambdas—back then this argument was the easiest way to pass data into the callback.)

class Program
{
    static void Main(string[] args)
    {
        Dns.BeginGetHostEntry("oreilly.com", OnGetHostEntryComplete, 42);

        Console.ReadKey();
    }

    static void OnGetHostEntryComplete(IAsyncResult iar)
    {
        IPHostEntry result = Dns.EndGetHostEntry(iar);
        Console.WriteLine(result.AddressList[0]);
        Console.WriteLine(iar.AsyncState);
    }
}

The Main method waits until a key is pressed—much like with work items in the thread pool, having active asynchronous requests will not keep the process alive, so the program would exit before finishing its work without that ReadKey. (A more robust approach for a real program that needed to wait for work to complete would be to use the CountdownEvent described earlier.)

The Dns class will call the OnGetHostEntryComplete method once it has finished its lookup. Notice that the first thing we do is call the EndGetHostEntry method—the other half of the APM. The End method always takes the IAsyncResult object corresponding to the call—recall that this identifies the call in progress, so this is how EndGetHostEntry knows which particular lookup operation we want to get the results for.

The End method in the APM returns any data that comes out of the operation. In this case, there’s a single return value of IPHostEntry, but some implementations may return more by having out or ref arguments. Example 16-16 then prints the results, and finally prints the AsyncState property of the IAsyncResult, which will be 42—this is where the value we passed as the final argument to BeginGetHostEntry pops out.

This is not the only way to use the Asynchronous Programming Model—you are allowed to pass null as the delegate argument. You have three other options, all revolving around the IAsyncResult object returned by the Begin call. You can poll the IsCompleted property to test for completion. You can call the End method at any time—if the work is not finished this will block until it completes.^[46] Or you can use the AsyncWaitHandle property—this returns an object that is a wrapper around a Win32 synchronization handle that will become signaled when the work is complete. (That last one is rarely used, and has some complications regarding ownership and lifetime of the handle, which are described in the MSDN documentation. We mention this technique only out of a pedantic sense of duty to completeness.)

Asynchronous operations can throw exceptions. If the exception is the result of bad input, such as a null reference where an object is required, the Begin method will throw an exception. But it’s possible that something failed while the operation was in progress—perhaps we lost network connectivity partway through some work. In this case, the End method will throw an exception.

The Asynchronous Programming Model is widely used in the .NET Framework class library, and while it is an efficient and flexible way to support asynchronous operations, it’s slightly awkward to use in user interfaces. The completion callback typically happens on some random thread, so you can’t update the UI in that callback. And the support for multiple simultaneous operations, possible because each operation is represented by a distinct IAsyncResult object, may be useful in server environments, but it’s often just an unnecessary complication for client-side code. So there’s an alternative pattern better suited to the UI.

Some classes offer an alternative pattern for asynchronous programming. You start an operation by calling a method whose name typically ends in Async; for example, the WebClient class’s DownloadDataAsync method. And unlike the APM, you do not pass a delegate to the method. Completion is indicated through an event, such as the DownloadDataCompleted event. Classes that implement this pattern are required to use the SynchronizationContext class (or the related AsyncOperationManager) to ensure that the event is raised in the same context in which the operation was started. So in a user interface, this means that completion events are raised on the UI thread.

This is, in effect, a single-threaded asynchronous model. You have the responsiveness benefits of asynchronous handling of slow operations, with fewer complications than multithreaded code. So in scenarios where this pattern is an option, it’s usually the best choice, as it is far simpler than the alternatives. It’s not always available, because some classes offer only the APM. (And some don’t offer any kind of asynchronous API, in which case you’d need to use one of the other multithreading mechanisms in this chapter to maintain a responsive UI.)

There are two optional features of the event-based asynchronous model. Some classes also offer progress change notification events, such as the WebClient class’s DownloadProgressChanged event. (Such events are also raised on the original thread.) And there may be cancellation support. For example, WebClient offers a CancelAsync method.

There’s no fundamental need for code to use either the APM or the event-based asynchronous pattern. These are just conventions. You will occasionally come across code that uses its own unusual solution for asynchronous operation. This can happen when the design of the code in question is constrained by external influences—for example, the System.Threading namespace defines an Overlapped class that provides a managed representation of a Win32 asynchronous mechanism. Win32 does not have any direct equivalent to either of the .NET asynchronous patterns, and just tends to use function pointers for callbacks. .NET’s Overlapped class mimics this by accepting a delegate as an argument to a method. Conceptually, this isn’t very different from the APM, it just happens not to conform exactly to the pattern.

The standard asynchronous patterns are useful, but they are somewhat low-level. If you need to coordinate multiple operations, they leave you with a lot of work to do, particularly when it comes to robust error handling or cancellation. The Task Parallel Library provides a more comprehensive scheme for working with multiple concurrent operations.

A task is some work your program needs to do. It is represented by an instance of the Task class, defined in the System.Threading.Tasks namespace. This does not define exactly how the work will be done. The work for a task might be a method to be executed, but a task could also involve asynchronous work that executes without needing to tie up a thread—the TPL has support for creating task objects that work with APM implementations, for instance. Example 16-17 shows how to create new tasks that execute code.

using System;
using System.Threading.Tasks;

namespace TplExamples
{
    class Program
    {
        static void Main(string[] args)
        {
            Task.Factory.StartNew(Go, "One");
            Task.Factory.StartNew(Go, "Two");

            Console.ReadKey();
        }

        static void Go(object name)
        {
            for (int i = 0; i < 100; ++i)
            {
                Console.WriteLine("{0}: {1}", name, i);
            }
        }
    }
}

The Task class provides a static Factory property that returns a TaskFactory object, which can be used to create new tasks. The TPL defines the TaskFactory abstraction so that it’s possible to plug in different task creation strategies. The default factory returned by Task.Factory creates new tasks that execute code on the thread pool, but it’s possible to create factories that do something else. For example, you could make a task factory that creates tasks that will run on a UI thread.

A factory’s StartNew method creates a code-based task. You pass it a delegate—it’ll accept either a method with no arguments or a method that takes a single object as an argument. If you want to pass more arguments, you can use the same lambda-based trick we saw in Example 16-4. Example 16-18 uses this to pass two arguments to Go, while using the overload of StartNew that takes a zero-argument method. (The empty () tells C# to build a zero-argument lambda, which becomes the method StartNew invokes.)

static void Main(string[] args)
{
    Task.Factory.StartNew(() => Go("One", 100));
    Task.Factory.StartNew(() => Go("Two", 500));

    Console.ReadKey();
}

static void Go(string name, int iterations)
{
    for (int i = 0; i < iterations; ++i)
    {
        Console.WriteLine("{0}: {1}", name, i);
    }
}

These last two examples look pretty similar to the thread pool examples from earlier, and they suffer from the same problem: they don’t know when the work is complete, so we’ve used the dubious solution of waiting for the user to press a key so that the program doesn’t exit until the work is done. Fortunately, tasks provide a much better solution to this: we can wait until they are finished. Task provides a Wait method that blocks until the task is complete. This is an instance method, so we’d call it once for each task. There’s also a static WaitAll method that takes an array of Task objects and blocks until they are all complete, illustrated in Example 16-19. (This method uses the params modifier on its one argument, so we can just pass each task as though it were a separate argument. The C# compiler will take the two tasks Example 16-19 passes to WaitAll and wrap them in an array for us.)

static void Main(string[] args)
{
    Task t1 = Task.Factory.StartNew(() => Go("One", 100));
    Task t2 = Task.Factory.StartNew(() => Go("Two", 500));

    Task.WaitAll(t1, t2);
}

Alternatively, we could create a parent task that contains both of these tasks as children.

static void Main(string[] args)
{
    Task t = Task.Factory.StartNew(() =>
    {
        Task.Factory.StartNew(() => Go("One", 100),
                              TaskCreationOptions.AttachedToParent);
        Task.Factory.StartNew(() => Go("Two", 500),
                              TaskCreationOptions.AttachedToParent);
    });

    t.Wait();
}

static void Main(string[] args)
{
    Task t = Task.Factory.StartNew(() => Go("One", 100))
                     .ContinueWith(t1 => Go("Two", 500))
                     .ContinueWith(t2 => Go("Three", 200));
    t.Wait();
}

void OnButtonClick(object sender, RoutedEventArgs e)
{
    TaskScheduler uiScheduler =
        TaskScheduler.FromCurrentSynchronizationContext();
    Task<string>.Factory.StartNew(GetData)
                    .ContinueWith((task) => UpdateUi(task.Result),
                                  uiScheduler);
}

string GetData()
{
    WebClient w = new WebClient();
    return w.DownloadString("http://oreilly.com/");
}

void UpdateUi(string info)
{
    myTextBox.Text = info;
}

TaskScheduler uiScheduler = TaskScheduler.FromCurrentSynchronizationContext();
Task<IPHostEntry>.Factory.FromAsync(
                    Dns.BeginGetHostEntry, Dns.EndGetHostEntry,
                    "oreilly.com", null)
    .ContinueWith((task) => UpdateUi(task.Result.AddressList[0].ToString()),
                  uiScheduler);

Cancellation of asynchronous operations is surprisingly tricky. There are lots of awkward race conditions to contend with. The operation you’re trying to cancel might already have finished by the time you try to cancel it. Or if it hasn’t it might have gotten beyond the point where it is able to stop, in which case cancellation is doomed to fail. Or work might have failed, or be about to fail when you cancel it. And even when cancellation is possible, it might take awhile to do. Handling and testing every possible combination is difficult enough when you have just one operation, but if you have multiple related tasks, it gets a whole lot harder.

Fortunately, .NET 4 introduces a new cancellation model that provides a well thought out and thoroughly tested solution to the common cancellation problems. This cancellation model is not limited to the TPL—you are free to use it on its own, and it also crops up in other parts of the .NET Framework. (The data parallelism classes we’ll be looking at later can use it, for example.)

If you want to be able to cancel an operation, you must pass it a CancellationToken. A cancellation token allows the operation to discover whether the operation has been canceled—it provides an IsCancellationRequested property—and it’s also possible to pass a delegate to its Register method in order to be called back if cancellation happens.

CancellationToken only provides facilities for discovering that cancellation has been requested. It does not provide the ability to initiate cancellation. That is provided by a separate class called CancellationTokenSource. The reason for splitting the discovery and control of cancellation across two types is that it would otherwise be impossible to provide a task with cancellation notifications without also granting that task the capability of initiating cancellation. CancellationTokenSource is a factory of cancellation tokens—you ask it for a token and then pass that into the operation you want to be able to cancel. Example 16-23 is similar to Example 16-21, but it passes a cancellation token to StartNew, and then uses the source to cancel the operation if the user clicks a Cancel button.

private CancellationTokenSource cancelSource;

void OnButtonClick(object sender, RoutedEventArgs e)
{
    cancelSource = new CancellationTokenSource();

    TaskScheduler uiScheduler =
        TaskScheduler.FromCurrentSynchronizationContext();
    Task<string>.Factory.StartNew(GetData, cancelSource.Token)
                    .ContinueWith((task) => UpdateUi(task.Result),
                                  uiScheduler);
}

void OnCancelClick(object sender, RoutedEventArgs e)
{
    if (cancelSource != null)
    {
        cancelSource.Cancel();
    }
}


string GetData()
{
    WebClient w = new WebClient();
    return w.DownloadString("http://oreilly.com/");
}

void UpdateUi(string info)
{
    cancelSource = null;
    myTextBox.Text = info;
}

In fact, cancellation isn’t very effective in this example because this particular task consists of code that makes a single blocking method call. Cancellation will usually do nothing here in practice—the only situation in which it would have an effect is if the user managed to click Cancel before the task had even begun to execute. This illustrates an important issue: cancellation is never forced—it uses a cooperative approach, because the only alternative is killing the thread executing the work. And while that would be possible, forcibly terminating threads tends to leave the process in an uncertain state—it’s usually impossible to know whether the thread you just zapped happened to be in the middle of modifying some shared state. Since this leaves your program’s integrity in doubt, the only thing you can safely do next is kill the whole program, which is a bit drastic. So the cancellation model requires cooperation on the part of the task in question. The only situation in which cancellation would have any effect in this particular example is if the user managed to click the Cancel button before the task had even begun.

If you have divided your work into numerous relatively short tasks, cancellation is more useful—if you cancel tasks that have been queued up but not yet started, they will never run at all. Tasks already in progress will continue to run, but if all your tasks are short, you won’t have to wait long. If you have long-running tasks, however, you will need to be able to detect cancellation and act on it if you want to handle cancellation swiftly. This means you will have to arrange for the code you run as part of the tasks to have access to the cancellation token, and they must test the IsCancellationRequested property from time to time.

Cancellation isn’t the only reason a task or set of tasks might stop before finishing—things might be brought to a halt by exceptions.

A task can complete in one of three ways: it can run to completion, it can be canceled, or it can fault. The Task object’s TaskStatus property reflects this through RanToCompletion, Canceled, and Faulted values, respectively, and if the task enters the Faulted state, its IsFaulted property also becomes true. A code-based task will enter the Faulted state if its method throws an exception. You can retrieve the exception information from the task’s Exception property. This returns an AggregateException, which contains a list of exceptions in its InnerExceptions property. It’s a list because certain task usage patterns can end up hitting multiple exceptions; for example, you might have multiple failing child tasks.

If you don’t check the IsFaulted property and instead just attempt to proceed, either by calling Wait or by attempting to fetch the Result of a Task<TResult>, the AggregateException will be thrown back into your code.

It’s possible to write code that never looks for the exception. Example 16-17 starts two tasks, and since it ignores the Task objects returned by StartNew, it clearly never does anything more with the tasks. If they were children of another task that wouldn’t matter—if you ignore exceptions in child tasks they end up causing the parent task to fault. But these are not child tasks, so if exceptions occur during their execution, the program won’t notice. However, the TPL tries hard to make sure you don’t ignore such exceptions—it uses a feature of the garbage collector called finalization to discover when a Task that faulted is about to be collected without your program ever having noticed the exception. When it detects this, it throws the AggregateException, which will cause your program to crash unless you’ve configured your process to deal with unhandled exceptions. (The .NET Framework runs all finalizers on a dedicated thread, and it’s this thread that the TPL throws the exception on.) The TaskScheduler class offers an UnobservedTaskException event that lets you customize the way these unhandled exceptions are dealt with.

The upshot is that you should write error handling for any nonchild tasks that could throw. One way to do this is to provide a continuation specifically for error handling. The ContinueWith method takes an optional argument whose type is the TaskContinuationOptions enumeration, which has an OnlyOnFaulted value—you could use this to build a continuation that will run only when an unanticipated exception occurs. (Of course, unanticipated exceptions are always bad news because, by definition, you weren’t expecting them and therefore have no idea what state your program is in. So you probably need to terminate the program, which is what would have happened anyway if you hadn’t written any error handling. However, you do get to write errors to your logs, and perhaps make an emergency attempt to write out unsaved data somewhere in the hope of recovering it when the program restarts.) But in general, it’s preferable to handle errors by putting normal try...catch blocks inside your code so that the exceptions never make it out into the TPL in the first place.

The Parallel class provides a couple of methods for performing data-driven parallel execution. Its For and ForEach methods are similar in concept to C# for and foreach loops, but rather than iterating through collections one item at a time, on a system with multiple logical processors available it will process multiple items simultaneously.

Example 16-24 uses Parallel.For. This code calculates pixel values for a fractal known as the Mandelbrot set, a popular parallelism demonstration because each pixel value can be calculated entirely independently of all the others, so the scope for parallel execution is effectively endless (unless machines with more logical processors than pixels become available). And since it’s a relatively expensive computation, the benefits of parallel execution are easy to see. Normally, this sort of code would contain two nested for loops, one to iterate over the rows of pixels and one to iterate over the columns in each row. In this example, the outer loop has been replaced with a Parallel.For. (So this particular code cannot exploit more processors than it calculates lines of pixels—therefore, we don’t quite have scope for per-pixel parallelism, but since you would typically generate an image a few hundred pixels tall, there is still a reasonable amount of scope for concurrency here.)

static int[,] CalculateMandelbrotValues(int pixelWidth, int pixelHeight,
    double left, double top, double width, double height, int maxIterations)
{
    int[,] results = new int[pixelWidth, pixelHeight];

    // Non-parallel version of following line would have looked like this:
    // for(int pixelY = 0; pixelY < pixelHeight; ++pixelY)
    Parallel.For(0, pixelHeight, pixelY =>
    {
        double y = top + (pixelY * height) / (double) pixelHeight;
        for (int pixelX = 0; pixelX < pixelWidth; ++pixelX)
        {
            double x = left + (pixelX * width) / (double) pixelWidth;

            // Note: this lives in the System.Numerics namespace in the
            // System.Numerics assembly.
            Complex c = new Complex(x, y);
            Complex z = new Complex();

            int iter;
            for (iter = 1; z.Magnitude < 2 && iter < maxIterations; ++iter)
            {
                z = z * z + c;
            }
            if (iter == maxIterations) { iter = 0; }
            results[pixelX, pixelY] = iter;
        }
    });

    return results;
}

This structure, seen in the preceding code:

Parallel.For(0, pixelHeight, pixelY =>
{
    ...
});

iterates over the same range as this:

for(int pixelY = 0, pixelY  < pixelHeight; ++pixelY)
{
    ...
}

The syntax isn’t identical because Parallel.For is just a method, not a language feature. The first two arguments indicate the range—the start value is inclusive (i.e., it will start from the specified value), but the end value is exclusive (it stops one short of the end value). The final argument to Parallel.For is a delegate that takes the iteration variable as its argument. Example 16-24 uses a lambda, whose minimal syntax introduces the least possible extra clutter over a normal for loop.

Parallel.For will attempt to execute the delegate on multiple logical processors simultaneously, using the thread pool to attempt to make full, efficient use of the available processors. The way it distributes the iterations across logical processors may come as a surprise, though. It doesn’t simply give the first row to the first logical processor, the second row to the second logical processor, and so on. It carves the available rows into chunks, and so the second logical processor will find itself starting several rows ahead of the first. And it may decide to subdivide further depending on the progress your code makes. So you must not rely on the iteration being done in any particular order. It does this chunking to avoid subdividing the work into pieces that are too small to handle efficiently. Ideally, each CPU should be given work in lumps that are large enough to minimize context switching and synchronization overheads, but small enough that each CPU can be kept busy while there’s work to be done. This chunking is one reason why data parallelism can sometimes be more efficient than using tasks directly—the parallelism gets to be exactly as fine-grained as necessary and no more so, minimizing overheads.

Arguably, calling Example 16-24 data parallelism is stretching a point—the “data” here is just the numbers being fed into the calculations. Parallel.For is no more or less data-oriented than a typical for loop with an int loop counter—it just iterates a numeric variable over a particular range in a list. However, you could use exactly the same construct to iterate over a range of data instead of a range of numbers. Alternatively, there’s Parallel.ForEach, which is very similar in use to Parallel.For, except, as you’d expect, it iterates over any IEnumerable<T> like a C# foreach loop, instead of using a range of integers. It reads ahead into the enumeration to perform chunking. (And if you provide it with an IList<T> it will use the list’s indexer to implement a more efficient partitioning strategy.)

There’s another way to perform parallel iteration over enumerable data: PLINQ.

Parallel LINQ (PLINQ) is a LINQ provider that enables any IEnumerable<T> to be processed using normal LINQ query syntax, but in a way that works in parallel. On the face of it, it’s deceptively simple. This:

var pq = from x in someList
         where x.SomeProperty > 42
         select x.Frob(x.Bar);

will use LINQ to Objects, assuming that someList implements IEnumerable<T>. Here’s the PLINQ version:

var pq = from x in someList.AsParallel()
         where x.SomeProperty > 42
         select x.Frob(x.Bar);

The only difference here is the addition of a call to AsParallel, an extension method that the ParallelEnumerable class makes available on all IEnumerable<T> implementations. It’s available to any code that has brought the System.Linq namespace into scope with a suitable using declaration. AsParallel returns a ParallelQuery<T>, which means that the normal LINQ to Objects implementation of the standard LINQ operators no longer applies. All the same operators are available, but now they’re supplied by ParallelEnumerable, which is able to execute certain operators in parallel.

Iterating over the results with foreach can restrict the extent to which the query can execute in parallel, because foreach asks for items one at a time—upstream parts of the query may still be able to execute concurrently, but the final results will be sequential. If you’d like to execute code for each item and to allow work to proceed in parallel even for this final processing step, PLINQ offers a ForAll operator:

pq.ForAll(x => x.DoSomething());

This will execute the delegate once for each item the query returns, and can do so in parallel—it will use as many logical processors concurrently as possible to evaluate the query and to call the delegate you provide.

This means that all the usual multithreading caveats apply for the code you run from ForAll. In fact, PLINQ can be a little dangerous as it’s not that obvious that your code is going to run on multiple threads—it manages to make parallel code look just a bit too normal. This is not always a problem—LINQ tends to encourage a functional style of programming in its queries, meaning that most of the data involved will be used in a read-only fashion, which makes dealing with threading much simpler. But code executed by ForAll is useful only if it has no side effects, so you need to be careful with whatever you put in there.