Understand Every API You Call

The guiding principle of this chapter is this:

You must understand the code executing behind every called API.

To say you have control of your performance is to assert that you know the code that executes in every critical path of your code. You should not have an opaque 3^rd-party library at the center of your inner loop—that is ceding control.

You will not always have access to the source of every method you call (though you do always have access to the assembly level!), but there is usually good documentation for all Windows APIs. With .NET, you can use one of the many IL viewing tools out there to see what the Framework is doing. (This ease of inspection does not extend to the CLR itself, which, while it is available as part of .NET Core, is written largely in dense, heavily macroed native code.)

Get used to examining Framework code for anything you are not familiar with. The more that performance is important to you, the more you need to question the implementation of APIs you do not own. Just remember to keep your pickiness proportional to the need for speed.

What follows in this chapter is a discussion of a few general areas you should be concerned with as well as some specific, common classes every program will use.

Multiple APIs for the Same Thing

You will occasionally run into a situation where you can choose among many APIs for accomplishing the same thing. A good example is XML parsing. XML parsing is never fast, exactly, but there are better ways to do it depending on your scenario. There are at least 9 different ways to parse XML in .NET:

• XmlTextReader
• XmlValidatingReader
• XDocument
• XmlDocument
• XPathNavigator
• XPathDocument
• LINQ-to-XML
• DataContractSerializer
• XmlSerializer

Which one you use depends on factors such as ease of use, productivity, suitability for the task, and performance. XmlTextReader is very fast, but it is forward-only and does no validation. XmlDocument is very convenient because it has a full object model loaded, but it is among the slowest options.

It is as true for XML parsing as it for other API choices: not all options will be equal, performance-wise. Some will be faster, but use more memory. Some will use very little memory, but may not allow certain operations. You will have to determine which features you need and measure the performance to determine which API provides the right balance of functionality vs. performance. You should prototype the options and profile them running on sample data.

Collections

.NET provides over 21 built-in collection types, including concurrent and generic versions of many popular data structures. Most programs will only need to use a combination of these and you should rarely need to create your own.

Choosing which collections to use depends on many factors, including: semantic meaning in the APIs (push/pop, enqueue/dequeue, add/remove, etc.), underlying storage mechanism and cache locality, speed of various operations on the collection such as Add and Remove, and whether you need to synchronize access to the collection. All of these factors can greatly influence the performance of your program.

byte[] fileContents = File.ReadAllBytes("foo.bin");
ArraySegment<byte> header = new ArraySegment<byte>(fileContents, 
                                                   1, 
                                                   24);
ProcessHeader(header);

const int Size = 50;

private int[,,] multiArray;

void Init()
{
    this.multiArray = new int[Size, Size, Size];
}

private const int Size = 50;

private int[][][] jaggedArray;

public void GlobalSetup()
{
    this.jaggedArray = new int[Size][][];
    for (int i = 0; i < Size; i++)
    {
        this.jaggedArray[i] = new int[Size][];
        for (int j = 0; j < Size; j++)
        {
            this.jaggedArray[i][j] = new int[Size];
        }
    }
}

public void Negate_JaggedArray()
{
    for (int i = 0; i < Size; i++)
    {
        for (int j = 0; j < Size; j++)
        {
            for (int k = 0; k < Size; k++)
            {
                this.jaggedArray[i][j][k] *= -1;
            }
        }
    }
}

public void Negate_MultiArray()
{
    for (int i = 0; i < Size; i++)
    {
        for (int j = 0; j < Size; j++)
        {
            for (int k = 0; k < Size; k++)
            {
                this.multiArray[i, j, k] *= -1;
            }
        }
    }
}

IL_000c: ldarg.0
IL_000d: ldfld int32[][][] ArrayPerf.ArrayPerfTest::jaggedArray
IL_0012: ldloc.0
IL_0013: ldelem.ref
IL_0014: ldloc.1
IL_0015: ldelem.ref
IL_0016: ldloc.2
IL_0017: ldelema [mscorlib]System.Int32
IL_001c: dup
IL_001d: ldind.i4
IL_001e: ldc.i4.m1
IL_001f: mul
IL_0020: stind.i4
IL_0021: ldloc.2
IL_0022: ldc.i4.1
IL_0023: add
IL_0024: stloc.2

IL_000c: ldarg.0
IL_000d: ldfld int32[0..., 0..., 0...] 
  ArrayPerf.ArrayPerfTest::multiArray
IL_0012: ldloc.0
IL_0013: ldloc.1
IL_0014: ldloc.2
IL_0015: call instance int32& int32[0..., 0..., 0...]::
  Address(int32, int32, int32)
IL_001a: dup
IL_001b: ldind.i4
IL_001c: ldc.i4.m1
IL_001d: mul
IL_001e: stind.i4
IL_001f: ldloc.2
IL_0020: ldc.i4.1
IL_0021: add
IL_0022: stloc.2

dict.AddOrUpdate(
  // Key I'm trying to add
  0, 
  // Delegate to call when adding--
  // return string value based on the key
  key => key.ToString(),
  // Delegate to call when already present--update existing value
  (key, existingValue) => existingValue);

dict.AddOrUpdate(
  // Key I'm trying to add
  0,
  // Value to add if new
  "0",
  // Delegate to call when already present--update existing value
  (key, existingValue) => existingValue);

string val1 = dict.GetOrAdd(
  // The key to retrieve
  0,
  // A delegate to generate the value if not present
  k => k.ToString());

string val2 = dict.GetOrAdd(
  // The key to retrieve
  0,
  // The value to add if not present
  "0");

var keytoLookup = "myKey";
var dict = new Dictionary<string, object>();
...
foreach(var kvp in dict)
{
    if (kvp.Key.ToUpper() == keyToLookup.ToUpper())
    {
        ...
    }
}

var keytoLookup = "myKey";
var dict = new Dictionary<string, object>(
             StringComparer.OrdinalIgnoreCase);
...
object val;
if (dict.TryGetValue(keyToLookup, out val))
{
    ...
}

struct MyType : IComparable<MyType>
{
    public string Name { get; set; }

    public int CompareTo(MyType other)
    {                
        return this.Name.CompareTo(other.Name);
    }
}

Strings

In .NET, strings have two problems:

1. Strings are immutable.
2. We want to mutate them.

From those two items spring most of the issues.

By immutable, we mean that, once created, strings exist forever in that state until garbage collected. This means that any modification of a string results in the creation of a new string. Fast, efficient programs generally do not modify strings in any way. Think about it: strings represent textual data, which is largely for human consumption. Unless your program is specifically for displaying or processing text, strings should be treated like opaque data blobs as much as possible. If you have the choice, always prefer non-string representations of data.

String.Compare(a, b, StringComparison.Ordinal);

String.Compare(a, b, StringComparison.OrdinalIgnoreCase);

String.Compare(a, b, StringComparison.CurrentCulture);

string result = a + b + c + d + e + f;

var sb = new StringBuilder(1024); // pre-initialize
var sb2 = new StringBuilder();    // default 16-char buffer

string message = String.Format(
  "The file {0} was {1} successfully.",
  filename, 
  operation);

string message = "The file " + filename + " was " + operation 
  + " successfully";

{
...
    ReadOnlySpan<char> subString = 
      "NonAllocatingSubstring".AsSpan().Slice(13);
    PrintSpan(subString);
...
}

private static void PrintSpan<T>(ReadOnlySpan<T> span)
{
    for (int i = 0; i < span.Length; i++)
    {
        T val = span[i];
        Console.Write(val);
        if (i < span.Length - 1) { Console.Write(", "); }
    }
    Console.WriteLine();
}

S, u, b, s, t, r, i, n, g

Avoid APIs that Throw Exceptions Under Normal Circumstances

Exceptions are expensive, as you saw in Chapter 5. As such, they should be reserved for truly exceptional circumstances. Unfortunately, there are some common APIs that defy this basic assumption.

Most basic types have a Parse method, which will throw a FormatException when the input string is in an unrecognized format. For example, Int32.Parse, DateTime.Parse, etc. Avoid these methods in favor of TryParse, which will return a bool if parsing fails.

Another example is the System.Net.HttpWebRequest class, which will throw an exception if it receives a non-200 response from a server. This bizarre behavior is thankfully corrected in the System.Net.Http.HttpClient class in .NET 4.5.

Avoid APIs That Allocate From the Large Object Heap

Recall from chapter 2 that arrays are basically the only things that can be allocated large enough to be placed on the large object heap. If your own code is allocating these, they should of course be pooled.

Unfortunately, there are .NET APIs which will do this as well. The only way you can avoid these APIs is by profiling heap allocations using PerfView, which will show the stacks allocating memory like this. You cannot always avoid it, so just be aware that there are some .NET APIs that will do this. For example, calling the Process.GetProcesses method will guarantee an allocation on the large object heap. You can avoid repeated LOH allocations by caching its results, calling it less frequently, or just avoid it completely by retrieving the information you need via interop directly from the Win32 API.

Some APIs will pool the large objects they create to avoid repeated allocation on the large object heap (StringBuilder does this, for example), but you would need to analyze their code to determine this. Profiling and going from there should be enough to steer you in the right direction.

Use Lazy Initialization

If your program uses a large or expensive-to-create object that is rarely used, or may not be used at all during a given invocation of the program, you can use the Lazy<T> class to wrap a lazy initializer around it. As soon as the Value property is accessed, the real object will be initialized according to the constructor you used to create the Lazy<T> object.

If your object has a default constructor, you can use the simplest version of Lazy<T>:

var lazyObject = new Lazy<MyExpensiveObject>();
...
if (needRealObject)
{
  MyExpensiveObject realObject = lazyObject.Value;
  ...
}

If construction is more complex, you can pass a Func<T> to the constructor.

var myObject = new Lazy<MyExpensiveObject>(
  () => Factory.CreateObject("A"));
...
MyExpensiveObject realObject = myObject.Value

Factory.CreateObject is a dummy method that produces an object of type MyExpensiveObject.

If myObject.Value is accessed from multiple threads, it is possible that each thread will need to initialize the object. By default, Lazy<T> is thread safe and only a single thread will be allowed to execute the creation delegate and set the Value property. You can modify this with a LazyThreadSafetyMode enumeration. This enumeration has three values:

• None: No thread safety. If important, you must ensure that the Lazy<T> object is accessed via a single thread in this case.
• ExecutionAndPublication: Only a single thread is allowed to execute the creation delegate and set the Value property. This is the default value if you use a constructor of Lazy without this option.
• PublicationOnly: Multiple threads can execute the creation delegate, but only a single one will initialize the Value property.

You should use Lazy<T> in place of your own singleton and double-checked locking pattern implementations.

If you have a large number of objects and Lazy<T> is too much overhead to use, you can use the static EnsureInitialized method on the LazyInitializer class. This uses Interlocked methods to ensure that the object reference is only assigned to once, but it does not ensure that the creation delegate is called only once. Unlike Lazy<T>, you must call the EnsureInitialized method yourself.

static MyObject[] objects = new MyObject[1024];      

static void EnsureInitialized(int index)
{
  LazyInitializer.EnsureInitialized(ref objects[index], 
      () => ExpensiveCreationMethod(index));
}

Note that by using the delegate here, you will have an extra allocation for the delegate object, in addition to whatever allocations ExpensiveCreationMethod performs.

The Surprisingly High Cost of Enums

You probably do not expect methods that operate on Enum values, a fundamentally integer type, to be very expensive. Unfortunately, because of the requirements of type safety, many simple operations are more expensive than you realize.

Take the Enum.HasFlag method, for example. You likely imagine the implementation to be something like the following:

public static bool HasFlag(Enum value, Enum flag)
{
  return (value & flag) != 0;
}

Unfortunately, what you actually get is something similar to:

// C# code generated by ILSpy
public bool HasFlag(Enum flag)
{
  if (flag == null)
  {
    throw new ArgumentNullException("flag");
  }
  if (!base.GetType().IsEquivalentTo(flag.GetType()))
  {
    throw new ArgumentException("Enum types do not match", 
        new object[]
      {
       flag.GetType(),
       base.GetType()
       }));
  }
  return this.InternalHasFlag(flag);
}

This is a good example of the side effects of using a general purpose framework. If you control your entire code base, then you can do better, performance-wise, by making certain assumptions and enforcing constraints not available to the .NET Framework. If you find you need to do a HasFlag test a lot, then do the check yourself:

[Flags]
enum Options
{
  Read = 0x01,
  Write = 0x02,
  Delete = 0x04
}

...

private static bool HasFlag(Options option, Options flag)
{
  return (option & flag) != 0;
}

Enum.ToString is also quite expensive for Enums that have the [Flags] attribute. One option to make it cheaper is to cache all of the ToString calls for that Enum type in a simple Dictionary<EnumType, String>. Or you can avoid writing these strings at all and get much better performance just using the actual numeric value and convert to strings offline.

For a fun exercise, see how much code is invoked when you call Enum.IsDefined. Again, the existing implementation is perfectly fine if raw performance does not matter, but you will be horrified if you find out it is a real bottleneck for you!

Story I found out about the performance problems of Enum the hard way, after a release. During a regular CPU profile I noticed that a significant portion of CPU, over 3%, was going to just Enum.HasFlag and Enum.ToString. Excising all calls to HasFlag and using a Dictionary for the cached strings reduced the overhead to negligible amounts.

Tracking Time

Time means two things:

• Absolute time of day
• Time span (how long something took)

For absolute times, .NET supplies the versatile DateTime structure. However, calling DateTime.Now is a fairly expensive operation because it has to consider time zone information. Consider calling DateTime.UtcNow instead, which is more streamlined.

Even calling DateTime.UtcNow might be too expensive for you if you need to track a lot of time stamps. If that is the case, get the time once and then track offsets instead, rebuilding the absolute time offline, using the time span measuring techniques shown here.

To measure time intervals, .NET provides the TimeSpan struct. If you subtract two DateTime structs you will get a TimeSpan struct.

However, if you need to measure very small time spans with minimal overhead, use the system’s performance counter, via System.Diagnostics.Stopwatch which will return to you a 64-bit number measuring the number of clock ticks since the CPU received power. To calculate the real time difference you take two measurements of the clock tick, subtract them, and divide by the system’s clock tick count frequency. Note that this frequency is not necessarily related to the CPU’s frequency. Most modern processors change their CPU frequency often, but the tick frequency will not be affected.

You can use the Stopwatch class like this:

var stopwatch = Stopwatch.StartNew();
//...do work...
stopwatch.Stop();
TimeSpan elapsed = stopwatch.Elapsed;
long elapsedTicks = stopwatch.ElapsedTicks;

There are also static methods to get a time stamp and the clock frequency, which may be more convenient if you are tracking a lot of time stamps and want to avoid the overhead of creating a new Stopwatch object for every interval.

long receiveTime = Stopwatch.GetTimestamp();
long parseTime = Stopwatch.GetTimestamp();
long startTime = Stopwatch.GetTimestamp();
long endTime = Stopwatch.GetTimestamp();

double totalTimeSeconds = (endTime - receiveTime) / 
    Stopwatch.Frequency;

Finally, remember that values received from the Stopwatch.GetTimestamp method are only valid in the current executing session and only for calculating relative time differences.

Combining the two types of time, you can see how to calculate offsets from a base DateTime object to get new absolute times:

DateTime start = DateTime.Now;
long startTime = Stopwatch.GetTimestamp();
long endTime = Stopwatch.GetTimestamp();

double diffSeconds = (endTime - startTime) / Stopwatch.Frequency;
DateTime end = start.AddSeconds(diffSeconds);

Regular Expressions

A regular expression is a way to search for a string with advanced pattern-matching syntax.

Some simple examples include:

Regex regex = new Regex("<value>(.*)</value>");

This finds any text between some XML <value> tags. Regular expressions can be extremely complex and difficult to understand until you master their syntax and learn how to break them down into manageable chunks.

A more complex-looking expression is:

static Regex regex = 
  new Regex("$s*[-+]?([0-9]{0,3}(,[0-9]{3})*(.[0-9]+)?)");

This extracts dollar values of the form “$-34.19”, “$52”, and a few others.

Regular expressions are not fast. Internally, .NET creates a state machine to traverse the input string and match it against the pattern’s symbols. The costs include:

• Evaluation time can be long: This depends on the input text and the pattern to match. It is quite easy to write regular expressions that perform poorly. Minor differences in the expression can make significant differences in processing speed and optimizing them in and of themselves is a whole topic unto itself.
• Assembly generation: With some options, an in-memory assembly is generated on the fly when you create a Regex object. This helps with the runtime performance, but is expensive to create the first time.
• JIT costs can be high: The code generated from a regular expression can be very long and have patterns that give the JIT compiler fits. Thankfully, recent versions of the JIT have largely improved this.

As far as .NET goes, there are a few things you can do to improve the efficiency of Regex usage:

• Ensure you are up-to-date with .NET and patches. Specifically, .NET 4.6 has a new JIT compiler that dramatically improves regular expression parsing performance.
• Create a Regex instance variable rather than using the static methods. The static methods will internally create an instance anyway, but you will not be able to reuse it. For one-offs, these are fine.
• Create the Regex object with the RegexOptions.Compiled flag. Without this, regular expressions are interpreted. When compiled, expressions are much faster to evaluate, at the expense of steeper initialization costs due to emitting code into an in-memory assembly and then JITting it. You may also want to avoid compiling expressions if you are always creating new format strings and rarely reusing earlier ones. This will pollute the process with a lot of extra code.
• Do not recreate Regex objects over and over again. Create one, save it, and reuse it to match on new input strings. Never have a pattern like this:

class Foo
{
    private static void Evaluate(string[] inputs)
    {
        for (int i = 0; i < inputs.Length; i++)
        {
            Regex regex = new Regex("<value>(.*)</value>", 
                                    RegexOptions.Compiled);
            Match match = regex.Match(input);
            ...
        }
    }
}

Do this instead:

class Foo
{
    private static readonly Regex regex = 
      new Regex("<value>(.*)</value>", 
                RegexOptions.Compiled);
    
    private static void Evaluate(string[] inputs)
    {
        for (int i = 0; i < inputs.Length; i++)
        {
            Match match = regex.Match(input);
            ...
        }
    }
}

You may also want to impose a timeout on complex expressions:

Regex regex = new Regex("<value>(.*)</value>", 
                        RegexOptions.Compiled, 
                        TimeSpan.FromSeconds(5));

You can also impose a global timeout via an application domain setting:

AppDomain.CurrentDomain.SetData("REGEX_DEFAULT_MATCH_TIMEOUT", 
                                TimeSpan.FromSeconds(10));

As an alternative to regular expressions, you can find or build your own parsing library for very simple patterns like dates or phone numbers that have few variations.

LINQ

The biggest danger with Language Integrated Query (LINQ) is that it has the potential to hide code from you—code for which you cannot be accountable because it is not present in your source file!

To illustrate this, look at a fairly simple example of a method that shifts all values in an array by a certain amount, provided they meet a certain condition:

public static int[] ShiftLinq(int[] vals, int shiftAmount)
{
    var temp = from n in vals select n + shiftAmount;
    return temp.ToArray();
}

It looks fairly innocuous, but there is a lot going on under the covers. Look at the IL this translates to:

.method public hidebysig static 
    int32[] ShiftLinq (
        int32[] vals,
        int32 shiftAmount
    ) cil managed 
{
    // Method begins at RVA 0x209c
    // Code size 37 (0x25)
    .maxstack 3
    .locals init (
      [0] class LinqCost.Program/'<>c__DisplayClass1_0'
    )

    IL_0000: newobj instance void LinqCost.Program
        /'<>c__DisplayClass1_0'::.ctor()
    IL_0005: stloc.0
    IL_0006: ldloc.0
    IL_0007: ldarg.1
    IL_0008: stfld int32 
        LinqCost.Program/'<>c__DisplayClass1_0'
          ::shiftAmount
    IL_000d: ldarg.0
    IL_000e: ldloc.0
    IL_000f: ldftn instance int32 
        LinqCost.Program/'<>c__DisplayClass1_0'
          ::'<ShiftLinq>b__0'(int32)
    IL_0015: newobj instance void class 
        [mscorlib]System.Func`2<int32, int32>
          ::.ctor(object, native int)
    IL_001a: call class [mscorlib]
        System.Collections.Generic.IEnumerable`1<!!1> 
        [System.Core]System.Linq.Enumerable
        ::Select<int32, int32>(class 
        [mscorlib]System.Collections.Generic.IEnumerable`1
          <!!0>, class [mscorlib]System.Func`2<!!0, !!1>)
    IL_001f: call !!0[] 
        [System.Core]System.Linq.Enumerable
        ::ToArray<int32>(
          class [mscorlib]
          System.Collections.Generic.IEnumerable`1<!!0>)
    IL_0024: ret
} // end of method Program::ShiftLinq

You can see that there are 37 bytes of IL code, including two allocations for closure objects and delegates, as well as two method calls. Some of those method calls will have further allocations. To top it off, the final ToArray call at the end takes an IEnumerable<T> argument, which means that, depending on the source collection, it will not know how long the array is until it gets to the end. That implies continual resizing and copying until the end, then a final copy into a precisely sized array. Whether any of this matters to you depends on the needs of your program. It is certainly far more expensive than an alternative implementation:

public static void Shift(int[] vals, int shiftAmount)
{
    for (int i = 0; i < vals.Length; i++)
    {                
        vals[i] += shiftAmount;                
    }
}

Instead of query syntax, we have a typical for-loop with an if statement. This translates to the following IL:

.method public hidebysig static 
    void Shift (
        int32[] vals,
        int32 shiftAmount
    ) cil managed 
{
    // Method begins at RVA 0x20d0
    // Code size 27 (0x1b)
    .maxstack 3
    .locals init (
        [0] int32
    )

    IL_0000: ldc.i4.0
    IL_0001: stloc.0
    IL_0002: br.s IL_0014
    // loop start (head: IL_0014)
        IL_0004: ldarg.0
        IL_0005: ldloc.0
        IL_0006: ldelema 
                   [mscorlib]System.Int32
        IL_000b: dup
        IL_000c: ldind.i4
        IL_000d: ldarg.1
        IL_000e: add
        IL_000f: stind.i4
        IL_0010: ldloc.0
        IL_0011: ldc.i4.1
        IL_0012: add
        IL_0013: stloc.0

        IL_0014: ldloc.0
        IL_0015: ldarg.0
        IL_0016: ldlen
        IL_0017: conv.i4
        IL_0018: blt.s IL_0004
    // end loop

    IL_001a: ret
} // end of method Program::Shift

The resulting IL is smaller at 27 bytes, but more importantly there are no allocations or method invocations whatsoever.

If you are paying attention, you may notice a rather significant “cheat” in the for-loop version of the code: it modifies the original array, while the LINQ version creates a new one. This is deliberate. LINQ pushes you in the direction of creating new objects rather than reuse existing ones. This is often because LINQ makes such heavy use of the IEnumerable<T> interface, which is the lowest-level interface a collection can implement. This means that semantics more appropriate to the specific data structure cannot be used, and we lose some performance in the process. However, there are methods which will check for the presence of other interfaces like ICollection or IList and make use of those methods if possible. LINQ also takes a rather functional, immutable view of data. The lesson here is that your tools will often enforce design and performance constraints upon you.

So how does the performance actually differ? Here are the benchmark result numbers:

The LINQ version is 20 times more expensive than the simpler version and because of its memory allocations, has a higher likelihood of causing garbage collections.

I want to be clear in stating that LINQ is not a bad technology—it has its place. It is phenomenally convenient at times, and many LINQ queries are perfectly performant, but it can make heavy use of delegates, interfaces, and temporary object allocation if you go crazy with temporary dynamic objects, joins, or complex where clauses.

You can often achieve some significant speedup in time by using Parallel LINQ, but keep in mind this is not actually reducing the amount of work to be done; it is just spreading it across multiple processors. For a mostly single-threaded application that just wants to reduce the time it takes to execute a LINQ query, this may be perfectly acceptable. On the other hand, if you are writing a server that is already using all of the cores to perform processing, then spreading LINQ across those same processors will not help the big picture and may even hurt it. In this case, it may be better to do without LINQ at all and find something more efficient.

If you suspect that you have some unaccounted-for complexity, run PerfView and look at the JITStats view to see the IL sizes and JIT times for methods that involve LINQ. Also look at the CPU usage of those methods once JITted.

It is worth noting that LINQ in .NET Core has made a number of efficiency improvements, such as reducing the number of allocations overall, avoiding delegate allocations, and improving algorithms all over the place to take advantage of known collection sizes (getting around the IEnumerable problem) that can offset some of the costs described here, but not all of them.

In the end, you must choose the appropriate tool for the job, and LINQ may be perfectly acceptable and convenient in most parts of your software, but inappropriate for other parts. Measure where allocations and CPU are going, and if it is in LINQ-heavy code, it is usually easy to convert it. Always let the profiler be the guide.

Reading and Writing Files

There are a number of convenience methods on the File class such as Open, OpenRead, OpenText, and OpenWrite. These are fine if performance is not critical.

If you are doing a lot of disk I/O, then you need to pay attention to the type of disk access you are doing, whether it is random, sequential, or if you need to ensure that the write has been physically written to the device before notifying the application of I/O completion. For this level of detail, you will need to use the FileStream class and a constructor overload that accepts the FileOptions enumeration. You can logically OR multiple flags together, but not all combinations are valid. None of these options are required, but they can provide hints to the operating system or file system on how to optimize file access.

using (var stream = new FileStream(
            @"C:foo.txt", 
            FileMode.Open, 
            FileAccess.Read, 
            FileShare.Read, 
            16384 /* Buffer Size*/, 
            FileOptions.SequentialScan | FileOptions.Encrypted))
{
...  
}

The options available to you are:

• None: No additional options.
• Asynchronous: Indicates that you will be doing asynchronous reading or writing to the file. This is not required to actually perform asynchronous reads and writes, but if you do not specify it, the underlying I/O will be performed synchronously without I/O completion ports. (Your thread will still execute asynchronously.) There are also overrides of the FileStream constructor that will take a Boolean parameter to specify asynchronous access.
• DeleteOnClose: Causes the OS to delete the file when the last handle to the file is closed. Use this for temporary files.
• Encrypted: Causes the file to be encrypted using the current account’s credentials.
• RandomAccess: Gives a hint to the file system to optimize caching for random access.
• SequentialScan: Gives a hint to the file system that the file is going to be read sequentially from beginning to end.
• WriteThrough: Ignores caching and goes directly to the device’s permanent storage. This generally makes I/O slower. The flag will be obeyed by the file system’s cache, but many storage devices also have onboard caches, and they are free to ignore this flag and report a successful completion before it is written to permanent storage.

Random access is bad for any device, such as a hard disk or tape, that needs to seek to the required position. Sequential access should be preferred for performance reasons. However, many computers have SSDs these days where the difference between random and sequential access is minimal or nonexistent.

Most applications will not have to care about the specific style of I/O until they are completely saturating the devices they are accessing. When that occurs, you will need to take into account the type of hardware that exists, as well as what kind of accessing you are performing, and use the above flags accordingly.

Optimizing HTTP Settings and Network Communication

If your application makes outbound HTTP calls, there are a number of settings you can change to optimize network transmission. You should exercise caution in changing these, however, as their effectiveness greatly depends on your network topology and the servers on the other end of the connection. You also need to take into account whether the target endpoints are in a data center you control, or are somewhere on the Internet. You will need to measure carefully to see if these settings benefit you or not.

To change these by default for all endpoints, modify these static properties on the ServicePointManager class:

• DefaultConnectionLimit: The number of connections per end point. Setting this higher may increase overall throughput if the network links and both endpoints can handle it.
• Expect100Continue: When a client initiates a POST or PUT command it normally waits for a 100-Continue signal from the server before proceeding to send the data. This allows the server to reject the request before the data is sent, saving bandwidth. If you control both endpoints and this situation does not apply to you, turn this off to improve latency.
• UseNagleAlgorithm: Nagling, described in RFC 896 at https://tools.ietf.org/html/rfc896.html is a way to reduce the overhead of packets on a network by combining many small packets into a single larger packet. This can be beneficial by reducing overall network transmission overhead, but it can also cause packets to be delayed. On modern networks, this value should usually be off. You can experiment with turning this off and see if there is a reduction in response times.

Some of these settings can also be applied to individual ServicePoint objects, which can be useful if you want to customize settings by endpoint, perhaps to differentiate between local datacenter endpoints and those on the Internet. In addition to the above, the ServicePoint class also lets you control additional parameters:

• ConnectionLeaseTimeout: Specifies the maximum time in milliseconds that an active connection will be kept alive. Set this to -1 to keep connections alive forever. This setting is useful for load balancing, where you will want to periodically force connections to close so that the process will re-connect to different machines. Setting this value to 0 will cause the connection to close after every request. This is not recommended because making a new HTTP connection is fairly expensive.
• ConnectionLimit: Specifies the maximum number of connections this endpoint can have.
• MaxIdleTime: Specifies the maximum time in milliseconds that a connection can remain open but idle. Set this to Timeout.Infinite to keep connections open indefinitely, regardless of whether they are active or not.
• ReceiveBufferSize: The size of the buffer used for receiving requests. The default is 8 KB. You can use a larger buffer if you regularly get large requests.
• SupportsPipelining: Allows multiple requests to be sent without waiting for a response between each one. However, the responses are sent back in order. See RFC 2616 at https://tools.ietf.org/html/rfc2616 (the HTTP/1.1 standard) for more information.

You can also force an individual HTTP request to close its current connection (after the response has been sent back) by setting the KeepAlive header to false.

Story Ensure that what you are transmitting is optimally encoded. While profiling an internal system, we noticed an extremely high memory allocation rate and CPU usage for a particular component. With some investigation, we realized that it was receiving an HTTP response, transforming the received bytes into a base64-encoded string, decoding that string into a binary blob, and then finally deserializing that blob back into a strongly typed object. It was wasting bandwidth by needlessly encoding a binary blob as a string, and wasting our CPU with multiple layers of encoding, and finally it was causing more time spent in GC with multiple large object allocations. The lesson is to send only what you need, as compactly as possible. Base64 is rarely, if ever, useful today, especially among internal components. Regardless of whether you are doing file or network I/O, encode the data as ideally as possible. For example, if you need to read a series of integers, do not waste CPU, memory, disk space, and network bandwidth wrapping that in XML.

Finally, another word of caution relating to the principle highlighted at the top of this chapter about the general purpose of much of the .NET Framework. The built-in HTTP client, while generally very good and perfectly acceptable for downloading Internet content, may not be suitable for all applications, particularly if your application is very sensitive to latencies at high percentiles, and especially with intra-datacenter requests. If you care about 95^th or 99^th percentile latencies for HTTP requests, you may have to write your own HTTP client around the underlying WinHTTP APIs to get that last bit of performance. Doing this correctly takes quite a bit of expertise in both HTTP and multithreading in .NET to get right, so you need to justify the effort.

SIMD

Single-Instruction, Multiple-Data features of processors execute a single set of operations across multiple pieces of data. Most modern x64-platform processors from both Intel and AMD support these instructions, and they are critical for parallel computations required in things like gaming, scientific, and mathematical computation. The current JIT compiler as of this writing (4.7.1) makes some limited use of these instructions and registers, mostly through the System.Numerics.Vectors NuGet package.

For an example, consider an algorithm that finds the minimum value in an array. A straightforward serial algorithm would look like this:

int min = int.MaxValue;
for (int i = 0; i < sourceArray.Length; i++)
{
    if (sourceArray[i] < min)
    {
        min = sourceArray[i];
    }
}

The equivalent SIMD version of this is considerably more complex:

var minVector = new Vector<int>(int.MaxValue);
int i = 0;
// Process array in chunks of the vector's length
for (i=0; i < Length - Vector<int>.Count; i += Vector<int>.Count)
{
    Vector<int> subArray = new Vector<int>(this.sourceArray, i);
    minVector = Vector.Min(subArray, minVector);
}

// get min of the min vector and any leftover elements
int min = Int32.MaxValue;
for (int j = 0; j < Vector<int>.Count; j++)
{
    min = Math.Min(min, minVector[j]);
}

for (i = 0; i < sourceArray.Length; i++)
{
    min = Math.Min(min, sourceArray[i]);
}

Since Vector uses hardware registers, the length is governed by that hardware and is static, thus the use of the static property Vector<int>.Count in the example above. On my machine, the SIMD version is not any faster than the non-SIMD version. However, the situation is very different for another algorithm.

Here is an algorithm that scales an array by an integer value. The non-SIMD version is very simple:

for(int i=0; i < this.sourceArray.Length; i++)
{
    this.destinationArray[i] = this.sourceArray[i] * ScaleFactor;
}

The SIMD version is arguably simpler:

this.destinationVector = this.sourceVector * ScaleFactor;

In this case, the performance difference is enormous. Here are some benchmark results for both types of algorithms:

You can see that scaling the vector is about twice as fast as the non-SIMD version.

There is no guarantee that any arbitrary SIMD-version of an algorithm is faster than its non-SIMD equivalent. Getting the algorithms right can depend on context, hardware, and the specifics of your CPU and memory patterns. There is an overhead to using SIMD instructions and the trick is making this less than the speedup you get from the parallelism. Usually, the effectiveness of a SIMD-based algorithm will depend on how effectively you can keep the algorithm operating on data within the Vector struct or other System.Numerics.Vector data structures. This is the equivalent of keeping the calculations on the hardware. When you transfer values from the Vector to standard .NET data structures, you lose the benefits of the parallelism and hardware acceleration.

Investigating Performance Issues

Many of the techniques for finding issues with .NET Framework performance are exactly the same as with your own code. When you use tools to profile CPU usage, memory allocations, exceptions, contention, and more, you will see the hot spots in the framework just like you see them in your own code.

Note that PerfView will group much of the framework together and you may need to change the view settings to get a better picture of where Framework performance is going.

Summary

As with all frameworks, you need to understand the implementation details of all the APIs you use. Do not take anything for granted.

Take care when picking collection classes. Consider API semantics, memory locality, algorithmic complexity, and space usage when choosing a collection. Prefer jagged arrays over multi-dimensional arrays. Completely avoid the older-style non-generic collections like ArrayList and HashTable. Use concurrent collections only when you need to synchronize most or all of the accesses. Consider initial capacity, key comparisons, and sorting ability with collections and the objects they store.

Pay particular attention to string usage and avoid creating extra strings. Prefer simpler APIs over more complex ones like String.Format. Use ReadOnlySpan<T> for substrings, when possible.

Avoid APIs that throw exceptions in normal circumstances, allocate from the large object heap, or have more expensive implementations than you expect.

When using regular expressions, make sure that you do not recreate the same Regex objects over and over again, and strongly consider compiling them with the RegexOptions.Compiled flag.

Pay attention to the type of I/O you are doing and use the appropriate flags when opening files to give the OS a chance to optimize performance for you. For network calls, disable Nagling and Expect100Continue. Only transmit the data you need and avoid unnecessary layers of encoding.

Avoid using reflection APIs to execute dynamically loaded code. Call this kind of code via common interfaces or through code-generated delegates.

If you are doing significant manipulation of arrays and matrices, use the System.Numerics.Vectors NuGet package to take advantage of SIMD commands.