Chapter 4. C# 4: Improving interoperability

C# 4 was an interesting release. The most dramatic change was the introduction of dynamic typing with the dynamic type. This feature makes C# statically typed (for most code) and dynamically typed (when using dynamic) in the same language. That’s rare within programming languages.

Dynamic typing was introduced for interoperability, but that’s turned out not to be relevant in many developers’ day-to-day work. The major features in other releases (generics, LINQ, async/await) have become a natural part of most C# developers’ toolkits, but dynamic typing is still used relatively rarely. I’m sure it’s useful to those who need it, and at the very least it’s an interesting feature.

The other features in C# 4 also improve interoperability, particularly with COM. Some improvements are specific to COM, such as named indexers, implicit ref arguments, and embedded interop types. Optional parameters and named arguments are useful with COM, but they can also be used in purely managed code. These two are the features from C# 4 that I use on a daily basis.

Finally, C# 4 exposes a feature of generics that was present in the CLR from v2 (the first runtime version that included generics). Generic variance is simultaneously simple and complex. At first glance, it sounds obvious: a sequence of strings is obviously a sequence of objects, for example. But then we discover that a list of strings isn’t a list of objects, dashing the expectations of some developers. It’s a useful feature, but one that’s prone to inducing headaches when you examine it closely. Most of the time, you can take advantage of it without even being aware that you’re doing so. Hopefully, the coverage in this chapter will mean that if you do need to look closer because your code isn’t working as you expect, you’ll be in a good position to fix the problem without getting confused. We’ll start off by looking at dynamic typing.

4.1. Dynamic typing

Some features come with a lot of new syntax, but after you’ve explained the syntax, there’s not much left to say. Dynamic typing is the exact opposite: the syntax is extremely simple, but I could go into almost endless detail about the impact and implementation. This section shows you the basics and then goes into some of the details before closing with a few suggestions about how and when to use dynamic typing.

4.1.1. Introduction to dynamic typing

Let’s start with an example. The following listing shows two attempts to take a substring from some text. At the moment, I’m not trying to explain why you’d want to use dynamic typing, just what it does.

Listing 4.1. Taking a substring by using dynamic typing

dynamic text = "hello world";      1
string world = text.Substring(6);  2
Console.WriteLine(world);

string broken = text.SUBSTR(6);    3
Console.WriteLine(broken);

• 1 Declares a variable with the dynamic type
• 2 Calls the Substring method; this works.
• 3 Tries to call SUBSTR; this throws an exception.

A lot is going on here for such a small amount of code. The most important aspect is that it compiles at all. If you changed the first line to declare text by using the string type, the call to SUBSTR would fail at compile time. Instead, the compiler is happy to compile it without even looking for a method called SUBSTR. It doesn’t look for Substring either. Instead, both lookups are performed at execution time.

At execution time, the second line will look for a method called Substring that can be called with an argument of 6. That method is found and returns a string, which you then assign to the world variable and print in a regular way. When the code looks for a method called SUBSTR that can be called with an argument of 6, it doesn’t find any such method, and the code fails with a RuntimeBinderException.

As mentioned in chapter 3, this process of looking up the meaning of a name in a certain context is called binding. Dynamic typing is all about changing when binding happens from compile time to execution time. Instead of just generating IL that calls a method with a precise signature determined at execution time, the compiler generates IL that performs the binding and then acts on the result. All of this is triggered by using the dynamic type.

What is the dynamic type?

Listing 4.1 declared the text variable as being of type dynamic:

dynamic text = "hello world";

What is the dynamic type? It’s different from other types you see in C#, because it exists only as far as the C# language is concerned. There’s no System.Type associated with it, and the CLR doesn’t know about it at all. Anytime you use dynamic in C#, the IL uses object decorated with [Dynamic] if necessary.

Note

If the dynamic type is used in a method signature, the compiler needs to make that information available for code compiling against it. There’s no need to do this for local variables.

The basic rules of the dynamic type are simple:

1. There’s an implicit conversion from any nonpointer type to dynamic.
2. There’s an implicit conversion from an expression of type dynamic to any nonpointer type.
3. Expressions that involve a value of type dynamic are usually bound at execution time.
4. Most expressions that involve a value of type dynamic have a compile-time type of dynamic as well.

You’ll look at the exceptions to the last two points shortly. Using this list of rules, you can look at listing 4.1 again with fresh eyes. Let’s consider the first two lines:

dynamic text = "hello world";
string world = text.Substring(6);

In the first line, you’re converting from string to dynamic, which is fine because of rule 1. The second line demonstrates all three of the other rules:

• text.Substring(6) is bound at execution time (rule 3).
• The compile-time type of that expression is dynamic (rule 4).
• There’s an implicit conversion from that expression to string (rule 2).

The conversion from an expression of type dynamic to a nondynamic type is dynamically bound, too. If you declared the world variable to be of type int, that would compile but fail at execution time with a RuntimeBinderException. If you declared it to be of type XNamespace, that would compile and then at execution time the binder would use the user-defined implicit conversion from string to XNamespace. With this in mind, let’s look at more examples of dynamic binding.

Applying dynamic binding in a variety of contexts

So far, you’ve seen dynamic binding based on the dynamic target of a method call and then a conversion, but almost any aspect of execution can be dynamic. The following listing demonstrates this in the context of the addition operator and performs three kinds of addition based on the type of the dynamic value at execution time.

Listing 4.2. Addition of dynamic values

static void Add(dynamic d)
{
    Console.WriteLine(d + d);       1
}

Add("text");                        2
Add(10);                            2
Add(TimeSpan.FromMinutes(45));      2

• 1 Performs addition based on the type at execution time
• 2 Calls the method with different values

The results of listing 4.2 are as follows:

texttext
20
01:30:00

Each kind of addition makes sense for the type involved, but in a statically typed context, they’d look different. As one final example, the following listing shows how method overloading behaves with dynamic method arguments.

Listing 4.3. Dynamic method overload resolution

static void SampleMethod(int value)
{
    Console.WriteLine("Method with int parameter");
}

static void SampleMethod(decimal value)
{
    Console.WriteLine("Method with decimal parameter");
}

static void SampleMethod(object value)
{
    Console.WriteLine("Method with object parameter");
}
static void CallMethod(dynamic d)
{
    SampleMethod(d);       1
}

CallMethod(10);            2
CallMethod(10.5m);         2
CallMethod(10L);           2
CallMethod("text");        2

• 1 Calls SampleMethod dynamically
• 2 Indirectly calls SampleMethod with different types

The output of listing 4.3 is as follows:

Method with int parameter
Method with decimal parameter
Method with decimal parameter
Method with object parameter

The third and fourth lines of the output are particularly interesting. They show that the overload resolution at execution time is still aware of conversions. In the third line, a long value is converted to decimal rather than int, despite being an integer in the range of int. In the fourth line, a string value is converted to object. The aim is that, as far as possible, the binding at execution time should behave the same way it would’ve at compile time, just using the types of the dynamic values as they’re discovered at execution time.

The result of any dynamically bound method call has a compile-time type of dynamic. When binding occurs, if the chosen method has a void return type and the result of the method was used (for example, being assigned to a variable), then binding fails. That’s the case for most dynamically bound operations: the compiler has little information about what the dynamic operation will entail. That rule has a few exceptions.

What can the compiler check in dynamically bound contexts?

If the context of a method call is known at compile time, the compiler is able to check what methods exist with the specified name. If no methods could possibly match at execution time, a compile-time error is still reported. This applies to the following:

• Instance methods and indexers where the target isn’t a dynamic value
• Static methods
• Constructors

The following listing shows various examples of calls using dynamic values that fail at compile time.

Listing 4.4. Examples of compile-time failures involving dynamic values

dynamic d = new object();
int invalid1 = "text".Substring(0, 1, 2, d);   1
bool invalid2 = string.Equals<int>("foo", d);  2
string invalid3 = new string(d, "broken");     3
char invalid4 = "text"[d, d];                  4

• 1 No String.Substring method with four paramet
• 2 No generic String.Equals method
• 3 No String constructors with two parameters accepting a string as a second argument
• 4 No String indexer with two parameters

Just because the compiler is able to tell that these particular examples are definitely broken doesn’t mean it’ll always be able to do so. Dynamic binding is always a bit of a leap into the unknown unless you’re very careful about the values involved.

The examples I’ve given would still use dynamic binding if they compiled. There are only a few cases where that’s not the case.

What operations involving dynamic values aren’t dynamically bound?

Almost everything you do with a dynamic value involves binding of some kind and finding the right method call, property, conversion, operator, and so on. There are just a few things that the compiler doesn’t need to generate any binding code for:

• Assignments to a variable of type object or dynamic. No conversion is required, so the compiler can just copy the existing reference.
• Passing an argument to a method with a corresponding parameter of type object or dynamic. That’s like assigning a variable, but the variable is the parameter.
• Testing a value’s type with the is operator.
• Attempting to convert a value with the as operator.

Although the execution-time binding infrastructure is happy to find user-defined conversions if you convert a dynamic value to a specific type with a cast or just do so implicitly, the is and as operators never use user-defined conversions, so no binding is required. In a similar way, almost all operations with dynamic values have a result that is also dynamic.

What operations involving dynamic values still have a static type?

Again, the compiler wants to help as much as it can. If an expression can always be of only one specific type, the compiler is happy to make that the compile-time type of the expression. For example, if d is a variable of type dynamic, the following are true:

• The expression new SomeType(d) has a compile-time type of SomeType, even though the constructor is bound dynamically at execution time.
• The expression d is SomeType has a compile-time type of bool.
• The expression d as SomeType has a compile-time type of SomeType.

That’s all the detail you need for this introduction. In section 4.1.4, you’ll look at unexpected twists, both at compile time and execution time. But now that you have the flavor of dynamic typing, you can look at some of its power beyond performing regular binding at execution time.

4.1.2. Dynamic behavior beyond reflection

One use for dynamic typing is to effectively ask the compiler and framework to perform reflection operations for you based on the members declared in types in the usual way. Although that’s a perfectly reasonable use, dynamic typing is more extensible. Part of the reason for its introduction was to allow better interoperability with dynamic languages that allow on-the-fly changes in binding. Many dynamic languages allow interception of calls at execution time. This has usages such as transparent caching and logging or making it look like there are functions and fields that are never declared by name in the source code.

Imaginary example of database access

As an (unimplemented) example of the kind of thing you might want to do, imagine you have a database containing a table of books, including their authors. Dynamic typing would make this sort of code possible:

dynamic database = new Database(connectionString);
var books = database.Books.SearchByAuthor("Holly Webb");
foreach (var book in books)
{
    Console.WriteLine(book.Title);
}

This would involve the following dynamic operations:

• The Database class would respond to a request for the Books property by querying the database schema for a table called Books and returning some sort of table object.
• That table object would respond to the SearchByAuthor method call by spotting that it started with SearchBy and looking for a column called Author within the schema. It would then generate SQL to query by that column using the provided argument and return a list of row objects.
• Each row object would respond to the Title property by returning the value of the Title column.

If you’re used to Entity Framework or a similar object-relational mapping (ORM), this may not sound like anything new. You can write classes fairly easily that enable the same kind of querying code or generate those classes from the schema. The difference here is that it’s all dynamic: there’s no Book or BooksTable class. It all just happens at execution time. In section 4.1.5, I’ll talk about whether that’s a good or a bad thing in general, but I hope you can at least see how it could be useful in some situations.

Before I introduce you to the types that allow all of this to happen, let’s look at two examples that are implemented. First, you’ll look at a type in the framework, and then at Json.NET.

ExpandoObject: A dynamic bag of data and methods

The .NET Framework provides a type called ExpandoObject in the namespace System.Dynamic. It operates in two modes depending on whether you’re using it as a dynamic value. The following listing gives a brief example to help you make sense of the description that follows it.

Listing 4.5. Storing and retrieving items in an ExpandoObject

dynamic expando = new ExpandoObject();
expando.SomeData = "Some data";                                 1
Action<string> action =                                         2
    input => Console.WriteLine("The input was '{0}'", input);   2
expando.FakeMethod = action;                                    2

Console.WriteLine(expando.SomeData);                            3
expando.FakeMethod("hello");                                    3

IDictionary<string, object> dictionary = expando;               4
Console.WriteLine("Keys: {0}",                                  4
    string.Join(", ", dictionary.Keys));                        4

dictionary["OtherData"] = "other";                              5
Console.WriteLine(expando.OtherData);                           5

• 1 Assigns data to a property
• 2 Assigns a delegate to a property
• 3 Accesses the data and delegate dynamically
• 4 Treats the ExpandoObject as a dictionary to print the keys
• 5 Populates data with the static context and fetches it from the dynamic value

When ExpandoObject is used in a statically typed context, it’s a dictionary of name/value pairs, and it implements IDictionary<string, object> as you’d expect from a normal dictionary. You can use it that way, looking up keys that are provided at execution time and so on.

More important, it also implements IDynamicMetaObjectProvider. This is the entry point for dynamic behavior. You’ll look at the interface itself later, but ExpandoObject implements it so you can access the dictionary keys by name within code. When you invoke a method on an ExpandoObject in a dynamic context, it’ll look up the method name as a key in the dictionary. If the value associated with that key is a delegate with appropriate parameters, the delegate is executed, and the result of the delegate is used as the result of the method call.

Listing 4.5 stored only one data value and one delegate, but you can store many with whatever names you want. It’s just a dictionary that can be accessed dynamically.

You could implement much of the earlier database example by using ExpandoObject. You’d create one to represent the Books table and then represent each book with a separate ExpandoObject, too. The table would have a key of SearchByAuthor with a suitable delegate value to execute the query. Each book would have a key of Title storing the title and so on. In practice, though, you’d want to implement IDynamicMetaObjectProvider directly or use DynamicObject. Before diving into those types, let’s take a look at another implementation: accessing JSON data dynamically.

The dynamic view of Json.NET

JSON is everywhere these days, and one of the most popular libraries for consuming and creating JSON is Json.NET.^[1] It provides multiple ways of handling JSON, including parsing straight to user-provided classes and parsing to an object model that’s closer to LINQ to XML. The latter is called LINQ to JSON with types such as JObject, JArray, and JProperty. It can be used like LINQ to XML, with access via strings, or it can be used dynamically. The following listing shows both approaches for the same JSON.

¹

Other JSON libraries are available, of course. I just happen to be most familiar with Json.NET.

Listing 4.6. Using JSON data dynamically

string json = @"                             1
    {                                        1
      'name': 'Jon Skeet',                   1
      'address': {                           1
        'town': 'Reading',                   1
        'country': 'UK'                      1
      }                                      1
    }".Replace(''', '"');                   1

JObject obj1 = JObject.Parse(json);          2

Console.WriteLine(obj1["address"]["town"]);  3

dynamic obj2 = obj1;                         4
Console.WriteLine(obj2.address.town);        4

• 1 Hardcoded sample JSON
• 2 Parses the JSON to a JObject
• 3 Uses the statically typed view
• 4 Uses the dynamically typed view

This JSON is simple but includes a nested object. The second half of the code shows how that can be accessed by either using the indexers within LINQ to JSON or using the dynamic view it provides.

Which of these do you prefer? Arguments exist for and against each approach. Both are prone to typos, whether within a string literal or the dynamic property access. The statically typed view lends itself to extracting the property names into constants for reuse, but the dynamically typed view is simpler to read when prototyping. I’ll make some suggestions for when and where dynamic typing is appropriate in section 4.1.5, but it’s worth reflecting on your initial reactions before you get there. Next we’ll take a quick look at how to do all of this yourself.

Implementing dynamic behavior in your own code

Dynamic behavior is complicated. Let’s get that out of the way to start with. Please don’t expect to come away from this section ready to write a production-ready optimized implementation of whatever amazing idea you have. This is only a starting point. That said, it should be enough to let you explore and experiment so you can decide how much effort you wish to invest in learning all the details.

When I presented ExpandoObject, I mentioned that it implements the interface IDynamicMetaObjectProvider. This is the interface signifying that an object implements its own dynamic behavior instead of just being happy to let the reflection-based infrastructure work in the normal way. As an interface, it looks deceptively simple:

public interface IDynamicMetaObjectProvider
{
    DynamicMetaObject GetMetaObject(Expression parameter);
}

The complexity lies in DynamicMetaObject, which is the class that drives everything else. Its official documentation gives a clue as to the level you need to think at when working with it:

Represents the dynamic binding and a binding logic of an object participating in the dynamic binding.

Even having used the class, I wouldn’t like to claim I fully understand that sentence, nor could I write a better description. Typically, you’d create a class deriving from DynamicMetaObject and override some of the virtual methods it provides. For example, if you want to handle method invocations dynamically, you’d override this method:

public virtual DynamicMetaObject BindInvokeMember
    (InvokeMemberBinder binder, DynamicMetaObject[] args);

The binder parameter gives information such as the name of the method being called and whether the caller expects binding to be performed case sensitively. The args parameter provides the arguments provided by the caller in the form of more DynamicMetaObject values. The result is yet another DynamicMetaObject representing how the method call should be handled. It doesn’t perform the call immediately but creates an expression tree representing what the call would do.

All of this is extremely complicated but allows for complex situations to be handled efficiently. Fortunately, you don’t have to implement IDynamicMetaObjectProvider yourself, and I’m not going to try to do so. Instead, I’ll give an example using a much friendlier type: DynamicObject.

The DynamicObject class acts as a base class for types that want to implement dynamic behavior as simply as possible. The result may not be as efficient as directly implementing IDynamicMetaObjectProvider yourself, but it’s much easier to understand.

As a simple example, you’re going to create a class (SimpleDynamicExample) with the following dynamic behavior:

• Invoking any method on it prints a message to the console, including the method name and arguments.
• Fetching a property usually returns that property name with a prefix to show you really called into the dynamic behavior.

The following listing shows how you would use the class.

Listing 4.7. Example of intended use of dynamic behavior

dynamic example = new SimpleDynamicExample();
example.CallSomeMethod("x", 10);
Console.WriteLine(example.SomeProperty);

The output should be as follows:

Invoked: CallSomeMethod(x, 10)
Fetched: SomeProperty

There’s nothing special about the names CallSomeMethod and SomeProperty, but you could’ve reacted to specific names in different ways if you’d wanted to. Even the simple behavior described so far would be tricky to get right using the low-level interface, but the following listing shows how easy it is with DynamicObject.

Listing 4.8. Implementing SimpleDynamicExample

class SimpleDynamicExample : DynamicObject
{
    public override bool TryInvokeMember(            1
        InvokeMemberBinder binder,                   1
        object[] args,                               1
        out object result)                           1
    {                                                1
        Console.WriteLine("Invoked: {0}({1})",       1
            binder.Name, string.Join(", ", args));   1
        result = null;                               1
        return true;                                 1
    }                                                1

    public override bool TryGetMember(               2
        GetMemberBinder binder,                      2
        out object result)                           2
    {                                                2
        result = "Fetched: " + binder.Name;          2
        return true;                                 2
    }                                                2
}

• 1 Handles method calls
• 2 Handles property access

As with the methods on DynamicMetaObject, you still receive binders when overriding the methods in DynamicObject, but you don’t need to worry about expression trees or other DynamicMetaObject values anymore. The return value from each method indicates whether the dynamic object successfully handled the operation. If you return false, a RuntimeBinderException will be thrown.

That’s all I’m going to show you in terms of implementing dynamic behavior, but I hope the simplicity of listing 4.8 will encourage you to experiment with DynamicObject. Even if you never use it in production, playing with it can be a lot of fun. If you want to give it a try but don’t have concrete ideas, you could always try implementing the Database example I gave at the start of this section. As a reminder, here’s the code you’d be trying to enable:

dynamic database = new Database(connectionString);
var books = database.Books.SearchByAuthor("Holly Webb");
foreach (var book in books)
{
    Console.WriteLine(book.Title);
}

Next, you’ll take a look at the code the C# compiler generates when it encounters dynamic values.

4.1.3. A brief look behind the scenes

You’re probably aware by now that I enjoy looking at the IL that the C# compiler uses to implement its various features. You’ve already looked at how captured variables in lambda expressions can result in extra classes being generated and how lambda expressions converted to expression trees result in calls to methods in the Expression class. Dynamic typing works a little bit like expression trees in terms of creating a data representation of the source code, but on a larger scale.

This section goes into even less detail than the previous one. Although the details are interesting, you almost certainly won’t need to know them.^[2] The good news is that it’s all open source, so you can go as low-level as you want to if you find yourself tantalized by this brief introduction to the topic. We’ll start off by considering which subsystem is responsible for what aspect of dynamic typing.

²

And to be honest, I don’t know enough details to do the whole topic justice.

Who does what?

Normally when you consider a C# feature, it’s natural to divide responsibility into three areas:

• The C# compiler
• The CLR
• The framework libraries

Some features are purely in the domain of the C# compiler. Implicit typing is an example of this. The framework doesn’t need to provide any types to support var, and the runtime is blissfully unaware of whether you used implicit or explicit typing.

At the other end of the spectrum is generics, which require significant compiler support, runtime support, and framework support in terms of the reflection APIs. LINQ is somewhere in between: the compiler provides the various features you saw in chapter 3, and the framework provides not only the implementation of LINQ to Objects but also the API for expression trees. On the other hand, the runtime didn’t need to change. For dynamic typing, the picture is a little more complicated. Figure 4.1 gives a graphical representation of the elements involved.

Figure 4.1. Graphical representation of components involved in dynamic typing

The CLR didn’t require changes, although I believe there were optimizations from v2 to v4 that were somewhat driven by this work. The compiler is obviously involved in generating different IL, and we’ll look at an example of this in a moment. For framework/library support, there are two aspects. The first is the Dynamic Language Runtime (DLR), which provides language-agnostic infrastructure such as DynamicMetaObject. That’s responsible for executing all the dynamic behavior. But a second library isn’t part of the core framework itself: Microsoft.CSharp.dll.

Note

This library ships with the framework but isn’t part of the system framework libraries as such. I find it helpful to think of it as if it were a third-party dependency, where the third party happens to be Microsoft. On the other hand, the Microsoft C# compiler is fairly tightly coupled to it. It doesn’t fit into any box particularly neatly.

This library is responsible for anything C# specific. For example, if you make a method call in which one argument is a dynamic value, it’s this library that performs the overload resolution at execution time. It’s a copy of the part of the C# compiler responsible for binding, but it does so in the context of all the dynamic APIs.

If you’ve ever seen a reference to Microsoft.CSharp.dll in your project and wondered what it was for, that’s the reason. If you don’t use dynamic typing anywhere, you can safely remove the reference. If you do use dynamic typing but remove the reference, you’ll get a compile-time error as the C# compiler generates calls into that assembly. Speaking of code generated by the C# compiler, let’s have a look at some now.

The IL generated for dynamic typing

We’re going to go right back to our initial example of dynamic typing but make it even shorter. Here are the first two lines of dynamic code I showed you:

dynamic text = "hello world";
string world = text.Substring(6);

Pretty simple, right? There are two dynamic operations here:

• The call to the Substring method
• The conversion from the result to a string

The following listing is a decompiled version of the code generated from those two lines. I’ve included the surrounding context of a class declaration and Main method just for clarity.

Listing 4.9. The result of decompiling two simple dynamic operations

using Microsoft.CSharp.RuntimeBinder;
using System;
using System.Runtime.CompilerServices;

class DynamicTypingDecompiled
{
  private static class CallSites                                  1
  {
    public static CallSite<Func<CallSite, object, int, object>>
       method;
    public static CallSite<Func<CallSite, object, string>>
       conversion;
  }

  static void Main()
  {
    object text = "hello world";
    if (CallSites.method == null)                                 2
    {
      CSharpArgumentInfo[] argumentInfo = new[]
      {
        CSharpArgumentInfo.Create(
          CSharpArgumentInfoFlags.None, null),
        CSharpArgumentInfo.Create(
          CSharpArgumentInfoFlags.Constant |
            CSharpArgumentInfoFlags.UseCompileTimeType,
          null)
      };
      CallSiteBinder binder =
        Binder.InvokeMember(CSharpBinderFlags.None, "Substring",
          null, typeof(DynamicTypingDecompiled), argumentInfo);
      CallSites.method =
        CallSite<Func<CallSite, object, int, object>>.Create(binder);
    }

    if (CallSites.conversion == null)                             3
    {
      CallSiteBinder binder =
        Binder.Convert(CSharpBinderFlags.None, typeof(string),
          typeof(DynamicTypingDecompiled));
      CallSites.conversion =
        CallSite<Func<CallSite, object, string>>.Create(binder);
    }
    object result = CallSites.method.Target(                      4
      CallSites.method, text, 6);                                 4

    string str =                                                  5
      CallSites.conversion.Target(CallSites.conversion, result);
  }
}

• 1 Cache of call sites
• 2 Creates a call site for the method call if necessary
• 3 Creates a call site for the conversion if necessary
• 4 Invokes the method call site
• 5 Invokes the conversion call site

I apologize for this formatting. I’ve done what I can to make it readable, but it’s a lot of code that involves a lot of long names. The good news is, you’re almost certain to never need to look at code like this except for the sake of interest. One point to note is that CallSite is in the System.Runtime.CompilerServices namespace as it’s language neutral, whereas the Binder class being used is from Microsoft.CSharp .RuntimeBinder.

As you can tell, a lot of call sites are involved. Each call site is cached by the generated code, and multiple levels of caching are within the DLR as well. Binding is a reasonably involved process. The cache within the call site improves performance by storing the result of each binding operation to avoid redundant work while being aware that the same call could end up with different binding results if some of the context changes between calls.

The result of all this effort is a system that’s remarkably efficient. It doesn’t perform quite as well as statically typed code, but it’s surprisingly close. I expect that in most cases where dynamic typing is an appropriate choice for other reasons, its performance won’t be a limiting factor. To wrap up the coverage of dynamic typing, I’ll explain a few limitations you may encounter and then give a little guidance around when and how dynamic typing is an effective choice.

4.1.4. Limitations and surprises in dynamic typing

Integrating dynamic typing into a language that was designed from the start to be statically typed is difficult. It’s no surprise that in a few places, the two don’t play nicely together. I’ve put together a list of some of the aspects of dynamic typing that include limitations or potential surprises to encounter at execution time. The list isn’t exhaustive, but it covers the most commonly seen problems.

The dynamic type and generics

Using the dynamic type with generics can be interesting. Rules are applied at compile time about where you can use dynamic:

• A type can’t specify that it implements an interface using dynamic anywhere in a type argument.
• You can’t use dynamic anywhere in type constraints.
• A class can specify a base class that uses dynamic in a type argument, even as part of an interface type argument.
• You can use dynamic as an interface type argument for variables.

Here are some examples of invalid code:

class DynamicSequence : IEnumerable<dynamic>
class DynamicListSequence : IEnumerable<List<dynamic>>
class DynamicConstraint1<T> : IEnumerable<T> where T : dynamic
class DynamicConstraint2<T> : IEnumerable<T> where T : List<dynamic>

But all of these are valid:

class DynamicList : List<dynamic>
class ListOfDynamicSequences : List<IEnumerable<dynamic>>
IEnumerable<dynamic> x = new List<dynamic> { 1, 0.5 }.Select(x => x * 2);

Extension methods

The execution-time binder doesn’t resolve extension methods. It could conceivably do so, but it’d need to keep additional information about every relevant using directive at every method call site. It’s important to note that this doesn’t affect statically bound calls that happen to use a dynamic type somewhere within a type argument. So, for example, the following listing compiles and runs with no problems.

Listing 4.10. A LINQ query over a list of dynamic values

List<dynamic> source = new List<dynamic>
{
    5,
    2.75,
    TimeSpan.FromSeconds(45)
};
IEnumerable<dynamic> query = source.Select(x => x * 2);
foreach (dynamic value in query)
{
    Console.WriteLine(value);
}

The only dynamic operations here are the multiplication (x * 2) and the overload resolution in Console.WriteLine. The call to Select is bound as normal at compile time. As an example of what will fail, let’s try making the source itself dynamic and simplify the LINQ operation you’re using to Any(). (If you kept using Select as before, you’d run into another problem that you’ll look at in a moment.) The following listing shows the changes.

Listing 4.11. Attempting to call an extension method on a dynamic target

dynamic source = new List<dynamic>
{
    5,
    2.75,
    TimeSpan.FromSeconds(45)
};
bool result = source.Any();

I haven’t included the output part, because execution doesn’t reach there. Instead, it fails with a RuntimeBinderException because List<T> doesn’t include a method called Any.

If you want to call an extension method as if its target were a dynamic value, you need to do so as a regular static method call. For example, you could rewrite the last line of listing 4.11 to the following:

bool result = Enumerable.Any(source);

The call will still be bound at execution time, but only in terms of overload resolution.

Anonymous functions

Anonymous functions have three limitations. For the sake of simplicity, I’ll show them all with lambda expressions.

First, anonymous methods can’t be assigned to a variable of type dynamic, because the compiler doesn’t know what kind of delegate to create. It’s fine if you either cast or use an intermediate statically typed variable (and then copy the value), and you can invoke the delegate dynamically, too. For example, this is invalid:

dynamic function = x => x * 2;
Console.WriteLine(function(0.75));

But this is fine and prints 1.5:

dynamic function = (Func<dynamic, dynamic>) (x => x * 2);
Console.WriteLine(function(0.75));

Second, and for the same underlying reason, lambda expressions can’t appear within dynamically bound operations. This is the reason I didn’t use Select in listing 4.11 to demonstrate the problem with extension methods. Here’s what listing 4.11 would’ve looked like otherwise:

dynamic source = new List<dynamic>
{
    5,
    2.75,
    TimeSpan.FromSeconds(45)
};
dynamic result = source.Select(x => x * 2);

You know that wouldn’t work at execution time because it wouldn’t be able to find the Select extension method, but it doesn’t even compile because of the use of the lambda expression. The workaround for the compile-time issue is the same as before: just cast the lambda expression to a delegate type or assign it to a statically typed variable first. That would still fail at execution time for extension methods such as Select, but it would be fine if you were calling a regular method such as List<T>.Find, for example.

Finally, lambda expressions that are converted to expression trees must not contain any dynamic operations. This may sound slightly odd, given the way the DLR uses expression trees internally, but it’s rarely an issue in practice. In most cases where expression trees are useful, it’s unclear what dynamic typing means or how it could possibly be implemented.

As an example, you can attempt to tweak listing 4.10 (with the statically typed source variable) to use IQueryable<T>, as shown in the following listing.

Listing 4.12. Attempting to use a dynamic element type in an IQueryable<T>

List<dynamic> source = new List<dynamic>
{
    5,
    2.75,
    TimeSpan.FromSeconds(45)
};
IEnumerable<dynamic> query = source
    .AsQueryable()
    .Select(x => x * 2);               1

• 1 This line now fails to compile.

The result of the AsQueryable() call is an IQueryable<dynamic>. This is statically typed, but its Select method accepts an expression tree rather than a delegate. That means the lambda expression (x => x * 2) would have to be converted to an expression tree, but it’s performing a dynamic operation, so it fails to compile.

Anonymous types

I mentioned this issue when I first covered anonymous types, but it bears repeating: anonymous types are generated as regular classes in IL by the C# compiler. They have internal access, so nothing can use them outside the assembly they’re declared in. Normally that’s not an issue, as each anonymous type is typically used only within a single method. With dynamic typing, you can read properties of instances of anonymous types, but only if that code has access to the generated class. The following listing shows an example of this where it is valid.

Listing 4.13. Dynamic access to a property of an anonymous type

static void PrintName(dynamic obj)
{
    Console.WriteLine(obj.Name);
}

static void Main()
{
    var x = new { Name = "Abc" };
    var y = new { Name = "Def", Score = 10 };
    PrintName(x);
    PrintName(y);
}

This listing has two anonymous types, but the binding process doesn’t care whether it’s binding against an anonymous type. It does check that it has access to the properties it finds, though. If you split this code across two assemblies, that would cause a problem; the binder would spot that the anonymous type is internal to the assembly where it’s created and throw a RuntimeBinderException. If you run into this problem and can use [InternalsVisibleTo] to allow the assembly performing the dynamic binding to have access to the assembly where the anonymous type is created, that’s a reasonable workaround.

Explicit interface implementation

The execution-time binder uses the execution-time type of any dynamic value and then binds in the same way as if you’d written that as the compile-time type of a variable. Unfortunately, that doesn’t play nicely with the existing C# feature of explicit interface implementation. When you use explicit interface implementation, that effectively says that the member being implemented is available only when you’re using the interface view over the object instead of the type itself.

It’s easier to show this than to explain it. The following listing uses List<T> as an example.

Listing 4.14. Example of explicit interface implementation

List<int> list1 = new List<int>();
Console.WriteLine(list1.IsFixedSize);     1

IList list2 = list1;
Console.WriteLine(list2.IsFixedSize);     2

dynamic list3 = list1;
Console.WriteLine(list3.IsFixedSize);     3

• 1 Compile-time error
• 2 Succeeds; prints False
• 3 Execution-time error

List<T> implements the IList interface. The interface has a property called IsFixedSize, but the List<T> class implements that explicitly. Any attempt to access it via an expression with a static type of List<T> will fail at compile time. You can access it via an expression with a static type of IList, and it’ll always return false. But what about accessing it dynamically? The binder will always use the concrete type of the dynamic value, so it fails to find the property, and a RuntimeBinderException is thrown. The workaround here is to convert the dynamic value back to the interface (via casting or a separate variable) if you know that you want to use an interface member.

I’m sure that anyone who works with dynamic typing on a regular basis would be able to regale you with a long list of increasingly obscure corner cases, but the preceding items should keep you from being surprised too often. We’ll complete our coverage of dynamic typing with a little guidance about when and how to use it.

4.1.5. Usage suggestions

I’ll be up front about this: I’m generally not a fan of dynamic typing. I can’t remember the last time I used it in production code, and I’d do so only warily and after a lot of testing for correctness and performance.

I’m a sucker for static typing. In my experience, it gives four significant benefits:

• When I make mistakes, I’m likely to discover them earlier—at compile time rather than execution time. That’s particularly important with code paths that may be hard to test exhaustively.
• Editors can provide code completion. This isn’t particularly important in terms of speed of typing, but it’s great as a way of exploring what I might want to do next, particularly if I’m using a type I’m unfamiliar with. Editors for dynamic languages can provide remarkable code-completion facilities these days, but they’ll never be quite as precise as those for statically typed languages, because there just isn’t as much information available.
• It makes me think about the API I’m providing, in terms of parameters, return types, and so on. After I’ve made decisions about which types to accept and return, that acts as ready-made documentation: I need to add comments only for anything that isn’t otherwise obvious, such as the range of acceptable values.
• By doing work at compile time instead of execution time, statically typed code usually has performance benefits over dynamically typed code. I don’t want to emphasize this too much, as modern runtimes can do amazing things, but it’s certainly worth considering.

I’m sure a dynamic typing aficionado would be able to give you a similar list of awesome benefits of dynamic typing, but I’m not the right person to do so. I suspect those benefits are more readily available in a language designed with dynamic typing right from the start. C# is mostly a statically typed language, and its heritage is clear, which is why the corner cases I listed earlier exist. That said, here are a few suggestions about when you might want to use dynamic typing.

Simpler reflection

Suppose you find yourself using reflection to access a property or method; you know the name at compile time, but you can’t refer to the static type for whatever reason. It’s much simpler to use dynamic typing to ask the runtime binder to perform that access than to do it directly with the reflection API. The benefit increases if you’d otherwise need to perform multiple steps of reflection. For example, consider a code snippet like this:

dynamic value = ...;
value.SomeProperty.SomeMethod();

The reflection steps involved would be as follows:

1. Fetch the PropertyInfo based on the type of the initial value.
2. Fetch the value of that property and remember it.
3. Fetch the MethodInfo based on the type of the property result.
4. Execute the method on the property result.

By the time you’ve added validation to check that the property and method both exist, you’re looking at several lines of code. The result would be no safer than the dynamic approach shown previously, but it would be a lot harder to read.

Common members without a common interface

Sometimes you do know all the possible types of a value in advance, and you want to use a member with the same name on all of them. If the types implement a common interface or share a common base class that declares the member, that’s great, but that doesn’t always happen. If each of them declares that member independently (and if you can’t change that), you’re left with unpleasant choices.

This time, you don’t need to use reflection, but you might need to perform several repetitive steps of check the type, cast, access the member. C# 7 patterns make this significantly simpler, but it can still be repetitive. Instead, you can use dynamic typing to effectively say “Trust me, I know this member will be present, even though I can’t express it in a statically typed way.” I’d be comfortable doing this within tests (where the cost of being wrong is a test failure), but in production code I’d be much more cautious.

Using a library built for dynamic typing

The .NET ecosystem is pretty rich and is getting better all the time. Developers are creating all kinds of interesting libraries, and I suspect some may embrace dynamic typing. For example, I can imagine a library designed to allow for easy prototyping with REST- or RPC-based APIs with no code generation involved. That could be useful in the initial phase of development while everything is quite fluid before generating a statically typed library for later development.

This is similar to the Json.NET example you looked at earlier. You may well want to write classes to represent your data model after that model is well-defined, but when prototyping, it may be simpler to change the JSON and then the code that’s accessing it dynamically. Likewise, you’ll see later how COM improvements mean that often you can end up working with dynamic typing instead of performing a lot of casting.

In a nutshell, I think it still makes sense to use static typing where it’s simple to do so, but you should accept dynamic typing as a potentially useful tool for some situations. I encourage you to weigh the pros and cons in each context. Code that’s acceptable for a prototype or even in test code may not be suitable for production code, for example.

Beyond code that you might write for professional purposes, the ability to respond with dynamic behavior by using DynamicObject or IDynamicMetaObjectProvider certainly gives a lot of scope for fun development. However much I may shy away from dynamic typing myself, it’s been well designed and implemented in C# and provides a rich avenue for exploration.

Our next feature is somewhat different, although both will come together when you look at COM interoperability. We’re back to static typing and one specific aspect of it: providing arguments for parameters.

4.2. Optional parameters and named arguments

Optional parameters and named arguments have a limited scope: given a method, constructor, indexer, or delegate that you want to call, how do you provide the arguments for the call? Optional parameters allow the caller to omit an argument entirely, and named arguments allow the caller to make it clear to both the compiler and any human reader which parameter an argument is intended to relate to.

Let’s start with a simple example and then dive into the details. In this whole section, I’m going to consider only methods. The same rules apply to all the other kinds of members that can have parameters.

4.2.1. Parameters with default values and arguments with names

The following listing shows a simple method with three parameters, two of which are optional. Multiple calls to the method then demonstrate different features.

Listing 4.15. Calling a method with optional parameters

static void Method(int x, int y = 5, int z = 10)       1
{
    Console.WriteLine("x={0}; y={1}; z={2}", x, y, z); 2
}

...

Method(1, 2, 3);                                       3
Method(x: 1, y: 2, z: 3);                              3
Method(z: 3, y: 2, x: 1);                              3
Method(1, 2);                                          4
Method(1, y: 2);                                       4
Method(1, z: 3);                                       5
Method(1);                                             6
Method(x: 1);                                          6

• 1 One required parameter, two optional
• 2 Just print the parameter values.
• 3 x=1; y=2; z=3
• 4 x=1; y=2; z=10
• 5 x=1; y=5; z=3
• 6 x=1; y=5; z=10

Figure 4.2 shows the same method declaration and one method call, just to make the terminology clear.

Figure 4.2. Syntax of optional/required parameters and named/positional arguments

The syntax is simple:

• A parameter can specify a default value after its name with an equal sign between the name and the value. Any parameter with a default value is optional; any parameter without a default value is required. Parameters with ref or out modifiers aren’t permitted to have default values.
• An argument can specify a name before the value with a colon between the name and the value. An argument without a name is called a positional argument.

The default value for a parameter must be one of the following expressions:

• A compile-time constant, such as a numeric or string literal, or the null literal.
• A default expression, such as default(CancellationToken). As you’ll see in section 14.5, C# 7.1 introduces the default literal, so you can write default instead of default(CancellationToken).
• A new expression, such as new Guid() or new CancellationToken(). This is valid only for value types.

All optional parameters must come after all required parameters, with an exception for parameter arrays. (Parameter arrays are parameters with the params modifier.)

Warning

Even though you can declare a method with an optional parameter followed by a parameter array, it ends up being confusing to call. I urge you to avoid this, and I won’t go into how calls to such methods are resolved.

The purpose of making a parameter optional is to allow the caller to omit it if the value it would supply is the same as the default value. Let’s look at what how the compiler handles a method call that can involve default parameters and/or named arguments.

4.2.2. Determining the meaning of a method call

If you read the specification, you’ll see that the process of working out which argument corresponds to which parameter is part of overload resolution and is intertwined with type inference. This is more complicated than you might otherwise expect, so I’m going to simplify things here. We’ll focus on a single method signature, assume it’s the one that has already been chosen by overload resolution, and take it from there.

The rules are reasonably simple to list:

• All positional arguments must come before all named arguments. This rule is relaxed slightly in C# 7.2, as you’ll see in section 14.6.
• Positional arguments always correspond to a parameter in the same position in the method signature. The first positional argument corresponds to the first parameter, the second positional argument corresponds to the second parameter, and so on.
• Named arguments match by name instead of position: an argument named x corresponds to a parameter named x. Named arguments can be specified in any order.
• Any parameter can have only one corresponding argument. You can’t specify the same name in two named arguments, and you can’t use a named argument for a parameter that already has a corresponding positional argument.
• Every required parameter must have a corresponding argument to provide a value.
• Optional parameters are permitted not to have a corresponding argument, in which case the compiler will supply the default value as an argument.

To see these rules in action, let’s consider our original simple method signature:

static void Method(int x, int y = 5, int z = 10)

You can see that x is a required parameter because it doesn’t have a default value, but y and z are optional parameters. Table 4.1 shows several valid calls and their results.

Table 4.1. Examples of valid method calls for named arguments and optional parameters

Call	Resulting arguments	Notes
Method(1, 2, 3)	x=1; y=2; z=3	All positional arguments. Regular call from before C# 4.
Method(1)	x=1; y=5; z=10	Compiler supplies values for y and z, as there are no corresponding arguments.
Method()	n/a	Invalid: no argument corresponds to x.
Method(y: 2)	n/a	Invalid: no argument corresponds to x.
Method(1, z: 3)	x=1; y=5; z=3	Compiler supplies value for y as there’s no corresponding argument. It was skipped by using a named argument for z.
Method(1, x: 2, z: 3)	n/a	Invalid: two arguments correspond to x.
Method(1, y: 2, y: 2)	n/a	Invalid: two arguments correspond to y.
Method(z: 3, y: 2, x: 1)	x=1; y=2; z=3	Named arguments can be in any order,

There are two more important aspects to note when it comes to evaluating method calls. First, arguments are evaluated in the order they appear in the source code for the method call, left to right. In most cases, this wouldn’t matter, but if argument evaluation has side effects, it can. As an example, consider these two calls to our sample method:

int tmp1 = 0;
Method(x: tmp1++, y: tmp1++, z: tmp1++);    1

int tmp2 = 0;
Method(z: tmp2++, y: tmp2++, x: tmp2++);    2

• 1 x=0; y=1; z=2
• 2 x=2; y=1; z=0

The two calls differ only in terms of the order of their named arguments, but that affects the values that are passed into the method. In both cases, the code is harder to read than it might be. When side effects of argument evaluation are important, I encourage you to evaluate them as separate statements and assign to new local variables that are then passed directly to the method as arguments, like this:

int tmp3 = 0;
int argX = tmp3++;
int argY = tmp3++;
int argZ = tmp3++;
Method(x: argX, y: argY, z: argZ);

At this point, whether you name the arguments doesn’t change the behavior; you can choose whichever form you find most readable. The separation of argument evaluation from method invocation makes the order of argument evaluation simpler to understand, in my opinion.

The second point to note is that if the compiler has to specify any default values for parameters, those values are embedded in the IL for the calling code. There’s no way for the compiler to say “I don’t have a value for this parameter; please use whatever default you have.” That’s why the default values have to be compile-time constants, and it’s one of the ways in which optional parameters affect versioning.

4.2.3. Impact on versioning

Versioning of public APIs in libraries is a hard problem. It’s really hard and significantly less clear-cut than we like to pretend. Although semantic versioning says that any breaking change means you need to move to a new major version, pretty much any change can break some code that depends on the library, if you’re willing to include obscure cases. That said, optional parameters and named arguments are particularly tricky for versioning. Let’s have a look at the various factors.

Parameter name changes are breaking

Suppose you have a library containing the method that you previously looked at, but it’s public:

public static Method(int x, int y = 5, int z = 10)

Now suppose you want to change that to the following in a new version:

public static Method(int a, int b = 5, int c = 10)

That’s a breaking change; any code that uses named arguments when calling the method will be broken, as the names they specified before no longer exist. Check your parameter names as carefully as you check your type and member names!

Default value changes are at least surprising

As I’ve noted, default values are compiled into the IL of the calling code. When that’s within the same assembly, changing the default value doesn’t cause a problem. When it’s in a different assembly, a change to the default value will be visible only when the calling code is recompiled.

That’s not always a problem, and if you anticipate that you might want to change the default value, it’s not entirely unreasonable to state that explicitly in the method documentation. But it could definitely surprise some developers using your code, particularly if complicated dependency chains are involved. One way of avoiding this is to use a dedicated default value that always means “Let the method choose at execution time.” For example, if you have a method that would normally have an int parameter, you could use Nullable<int> instead, with a default value of null meaning “the method will choose.” You can change the implementation of the method later to make a different choice, and every caller using the new version will get the new behavior, whether they’ve recompiled or not.

Adding overloads is fiddly

If you thought overload resolution was tricky in a single-version scenario, it becomes a lot worse when you’re trying to add an overload without breaking anyone. All original method signatures must be present in the new version to avoid breaking binary compatibility, and all calls against the original methods should either resolve to the same calls, or at least equivalent calls, in the new version. Whether a parameter is required or optional isn’t part of the method signature itself; you don’t break binary compatibility by changing an optional parameter to be required, or vice versa. But you might break source compatibility. If you’re not careful, you can easily introduce ambiguity in overload resolution by adding a new method with more optional parameters.

If two methods are both applicable within overload resolution (both make sense with respect to the call) and neither is better than the other in terms of the argument-to-parameter conversions involved, then default parameters can be used as a tiebreak. A method that has no optional parameters without corresponding arguments is “better” than a method with at least one optional parameter without a corresponding argument. But a method with one unfilled parameter is no better than a method with two such parameters.

If you can possibly get away without adding overloads to methods when optional parameters are involved, I strongly advise that you do so—and, ideally, bear that in mind from the start. One pattern to consider for methods that might have a lot of options is to create a class representing all those options and then take that as an optional parameter in method calls. You can then add new options by adding properties to the options class without changing the method signature at all.

Despite all these caveats, I’m still in favor of optional parameters when they make sense to simplify calling code for common cases, and I’m a big fan of the ability to name arguments to clarify calling code. This is particularly relevant when multiple parameters of the same type could be confused with each other. As one example, I always use them when I need to call the Windows Forms MessageBox.Show method. I can never remember whether the title of the message box or the text comes first. IntelliSense can help me when I’m writing the code, but it’s not as obvious when I’m reading it, unless I use named arguments:

MessageBox.Show(text: "This is text", caption: "This is the title");

Our next topic is one that many readers may have no need for and other readers will use every day. Although COM is a legacy technology in many contexts, a huge amount of code still uses it.

4.3. COM interoperability improvements

Before C# 4, VB was simply a better language to use if you wanted to interoperate with COM components. It’s always been a somewhat more relaxed language, at least if you ask it to be, and it has had named arguments and optional parameters from the start. C# 4 makes life much simpler for those working with COM. That said, if you’re not using COM, you won’t miss out on anything important by skipping this section. None of the features I go into here is relevant outside COM.

Note

COM is the Component Object Model introduced by Microsoft in 1993 as a cross-language form of interoperability on Windows. A full description is beyond the scope of this book, but you’re likely to know about it if you need to know about it. The most commonly used COM libraries are probably those for Microsoft Office.

Let’s start with a feature that goes beyond the language. It’s mostly about deployment, although it also impacts how the operations are exposed.

4.3.1. Linking primary interop assemblies

When you code against a COM type, you use an assembly generated for the component library. Usually, you use a primary interop assembly (PIA) generated by the component publisher. You can use the Type Library Importer tool (tlbimp) to generate this for your own COM libraries.

Before C# 4, the complete PIA had to be present on the machine where the code finally ran, and it had to be the same version as the one that you compiled against. This either meant shipping the PIA along with your application or trusting that the right version would already be installed.

From C# 4 and Visual Studio 2010 onward, you can choose to link the PIA instead of referencing it. In Visual Studio, in the property page for the reference, this is the Embed Interop Types option.

When this option is set to True, the relevant parts of the PIA are embedded directly into your assembly. Only the bits you use within your application are included. When the code runs, it doesn’t matter whether the exact same version of the component you used to compile against is present on the client machine, so long as it has everything that your application needs. Figure 4.3 shows the difference between referencing (the old way) and linking (the new way) in terms of how the code runs.

Figure 4.3. Comparing referencing and linking

In addition to deployment changes, linking the PIA affects how the VARIANT type is treated within the COM type. When the PIA is referenced, any operations returning a VARIANT value would be exposed using the object type in C#. You’d then have to cast that to the appropriate type to use its methods and properties.

When the PIA is linked instead, dynamic is returned instead of object. As you saw earlier, there’s an implicit conversion from an expression of type dynamic to any nonpointer type, which is then checked at execution time. The following listing shows an example of opening Excel and populating 20 cells in a range.

Listing 4.16. Setting a range of values in Excel with implicit dynamic conversion

var app = new Application { Visible = true };
app.Workbooks.Add();
Worksheet sheet = app.ActiveSheet;
Range start = sheet.Cells[1, 1];
Range end = sheet.Cells[1, 20];
sheet.Range[start, end].Value = Enumerable.Range(1, 20).ToArray();

Listing 4.16 silently uses some of the features coming up later, but for the moment focus on the assignments to sheet, start, and end. Each would need a cast normally, as the value being assigned would be of type object. You don’t have to specify the static types for the variables; if you used var or dynamic for the variable types, you’d be using dynamic typing for more operations. I prefer to specify the static type where I know what I expect it to be, partly for the implicit validation this performs and partly to enable IntelliSense in the code that follows.

For COM libraries that use VARIANT extensively, this is one of the most important benefits of dynamic typing. The next COM feature also builds on a new feature in C# 4 and takes optional parameters to a new level.

4.3.2. Optional parameters in COM

Some COM methods have a lot of parameters, and often they’re all ref parameters. This meant that prior to C# 4, a simple act like saving a file in Word could be extremely painful.

Listing 4.17. Creating a Word document and saving it before C# 4

object missing = Type.Missing;                             1

Application app = new Application { Visible = true };      2
Document doc = app.Documents.Add                           3
    ref missing, ref missing,                              3
    ref missing, ref missing);                             3
Paragraph para = doc.Paragraphs.Add(ref missing);          3
para.Range.Text = "Awkward old code";                      3

object fileName = "demo1.docx";                            4
doc.SaveAs2(ref fileName, ref missing,                     4
    ref missing, ref missing, ref missing,                 4
    ref missing, ref missing, ref missing,                 4
    ref missing, ref missing, ref missing,                 4
    ref missing, ref missing, ref missing,                 4
    ref missing, ref missing);                             4

doc.Close(ref missing, ref missing, ref missing);          5
app.Application.Quit(                                      5
    ref missing, ref missing, ref missing);                5

• 1 Placeholder variable for ref parameters
• 2 Starts Word
• 3 Creates and populates a document
• 4 Saves the document
• 5 Closes Word

A lot of code is required just to create and save a document, including 20 occurrences of ref missing. It’s hard to see the useful part of the code within the forest of arguments you don’t care about.

C# 4 provides features that all work together to make this much simpler:

• Named arguments can be used to make it clear which argument should correspond to which parameter, as you’ve already seen.
• Just for COM libraries, values can be specified directly as arguments for ref parameters. The compiler will create a local variable behind the scenes and pass that by reference.
• Just for COM libraries, ref parameters can be optional and then omitted in the calling code. Type.Missing is used as the default value.

With all of these features in play, you can transform listing 4.17 into much shorter and cleaner code.

Listing 4.18. Creating a Word document and saving it using C# 4

Application app = new Application { Visible = true };
Document doc = app.Documents.Add();                     1
Paragraph para = doc.Paragraphs.Add();
para.Range.Text = "Simple new code";

doc.SaveAs2(FileName: "demo2.docx");                    2

doc.Close();
app.Application.Quit();

• 1 Optional parameters omitted everywhere
• 2 Named argument used for clarity

This is a dramatic transformation in readability. All 20 occurrences of ref missing are gone, as is the variable itself. As it happens, the argument you pass to SaveAs2 corresponds to the first parameter of the method. You could use a positional argument instead of a named argument, but specifying the name adds clarity. If you also wanted to specify a value for a later parameter, you could do so by name without providing values for all the other parameters in between.

That argument to SaveAs2 also demonstrates the implicit ref feature. Instead of having to declare a variable within an initial value of demo2.docx and then pass that by reference, you can pass the value directly, as far as our source code is concerned. The compiler handles turning it into a ref parameter for you. The final COM-related feature exposes another aspect where VB is slightly richer than C#.

4.3.3. Named indexers

Indexers have been present in C# since the beginning. They’re primarily used for collections: retrieving an element from a list by index or retrieving a value from a dictionary by key, for example. But C# indexers are never named in source code. You can write only the default indexer for the type. You can specify a name by using an attribute, and that name will be consumed by other languages, but C# doesn’t let you differentiate between indexers by name. At least, it didn’t until C#4.

Other languages allow you to write and consume indexers with names, so you can access different aspects of an object via indexes using the name to make it clear what you want. C# still doesn’t do this for regular .NET code, but it makes an exception just for COM types. An example will make this clearer.

The Application type in Word exposes a named indexer called SynonymInfo. It’s declared like this:

SynonymInfo SynonymInfo[string Word, ref object LanguageId = Type.Missing]

Prior to C# 4, you could call the indexer as if it were a method called get_SynonymInfo, but that’s somewhat awkward. In C# 4, you can access it by name, as shown in the following listing.

Listing 4.19. Accessing a named indexer

Application app = new Application { Visible = false };

object missing = Type.Missing;                                      1
SynonymInfo info = app.get_SynonymInfo("method", ref missing);      1
Console.WriteLine("'method' has {0} meanings", info.MeaningCount);

info = app.SynonymInfo["index"];                                    2
Console.WriteLine("'index' has {0} meanings", info.MeaningCount);

• 1 Accessing synonyms prior to C# 4
• 2 Simpler code using a named indexer

Listing 4.19 shows how optional parameters can be used in named indexers as well as regular method calls. The code for before C# 4 has to declare a variable and pass it by reference to the awkwardly named method. With C# 4, you can use the indexer by name, and you can omit the argument for the second parameter.

That was a brief run through the COM-related features in C# 4, but I hope the benefits are obvious. Even though I don’t work with COM regularly, the changes shown here would make me a lot less despondent if I ever need to in the future. The extent of the benefit will depend on how the COM library you’re working with is structured. For example, if it uses a lot of ref parameters and VARIANT return types, the difference will be more significant than a library with few parameters and concrete return types. But even just the option of linking the PIA could make deployment significantly simpler.

We’re coming toward the end of C# 4 now. The final feature can be a bit tricky to get your head around, but it’s also one you may use without even thinking about it.

4.4. Generic variance

Generic variance is easier to show than to describe. It’s about safely converting between generic types based on their type arguments and paying particular attention to the direction in which data travels.

4.4.1. Simple examples of variance in action

We’ll start with an example using a familiar interface, IEnumerable<T>, which represents a sequence of elements of type T. It makes sense that any sequence of strings is also a sequence of objects, and variance allows that:

IEnumerable<string> strings = new List<string> { "a", "b", "c" };
IEnumerable<object> objects = strings;

That may seem so natural that you’d be surprised if it failed to compile, but that’s exactly what would’ve happened before C# 4.

Note

I’m using string and object consistently in these examples because they’re classes that all C# developers know about and aren’t tied to any particular context. Other classes with the same base class/derived class relationship would work just as well.

There are potentially more surprises to come; not everything that sounds like it should work does work, even with C# 4. For example, you might try to extend the reasoning about sequences to lists. Is any list of strings a list of objects? You might think so, but it’s not:

IList<string> strings = new List<string> { "a", "b", "c" };
IList<object> objects = strings;                               1

• 1 Invalid: no conversion from IList<string> to IList<object>

What’s the difference between IEnumerable<T> and IList<T>? Why isn’t this allowed? The answer is that it wouldn’t be safe, because the methods within IList<T> allow values of type T as inputs as well as outputs. Every way you can use an IEnumerable<T> ends up with T values being returned as output, but IList<T> has methods like Add that accept a T value as input. That would make it dangerous to allow variance. You can see this if you try to extend our invalid example a little:

IList<string> strings = new List<string> { "a", "b", "c" };
IList<object> objects = strings;
objects.Add(new object());                                  1
string element = strings[3];                                2

• 1 Adds an object to the list
• 2 Retrieves it as a string

Every line other than the second one makes sense on its own. It’s fine to add an object reference to an IList<object>, and it’s fine to take a string reference from an IList<string>. But if you can treat a list of strings as a list of objects, those two abilities come into conflict. The language rules that make the second line invalid are effectively protecting the rest of the code.

So far, you’ve seen values being returned as output (IEnumerable<T>) and values being used as both input and output (IList<T>). In some APIs, values are always used only as input. The simplest example of this is the Action<T> delegate, where you pass in a value of type T when you invoke the delegate. Variance still applies here, but in the opposite direction. This can be confusing to start with.

If you have an Action<object> delegate, that can accept any object reference. It can definitely accept a string reference, and the language rules allow you to convert from Action<object> to Action<string>:

Action<object> objectAction = obj => Console.WriteLine(obj);
Action<string> stringAction = objectAction;
stringAction("Print me");

With those examples in hand, I can define some terminology:

• Covariance occurs when values are returned only as output.
• Contravariance occurs when values are accepted only as input.
• Invariance occurs when values are used as input and output.

Those definitions are deliberately slightly vague for now. They’re more about the general concepts than they are about C#. We can tighten them up after you’ve looked at the syntax C# uses to specify variance.

4.4.2. Syntax for variance in interface and delegate declarations

The first thing to know about variance in C# is that it can be specified only for interfaces and delegates. You can’t make a class or struct covariant, for example. Next, variance is defined separately for each type parameter. Although you might loosely say “IEnumerable<T> is covariant,” it would be more precise to say “IEnumerable<T> is covariant in T.” That then leads to syntax for interface and delegate declarations in which each type parameter has a separate modifier. Here are the declarations for the IEnumerable<T> and IList<T> interfaces and Action<T> delegate:

public interface IEnumerable<out T>
public delegate void Action<in T>
public interface IList<T>

As you can see, the modifiers in and out are used to specify the variance of a type parameter:

• A type parameter with the out modifier is covariant.
• A type parameter with the in modifier is contravariant.
• A type parameter with no modifiers is invariant.

The compiler checks that the modifier you’ve used is suitable given the rest of the declaration. For example, this delegate declaration is invalid because a covariant type parameter is used as input:

public delegate void InvalidCovariant<out T>(T input)

And this interface declaration is invalid because a contravariant type parameter is used as output:

public interface IInvalidContravariant<in T>
{
    T GetValue();
}

Any single type parameter can have only one of these modifiers, but two type parameters in the same declaration can have different modifiers. For example, consider the Func<T, TResult> delegate. That accepts a value of type T and returns a value of type TResult. It’s natural for T to be contravariant and TResult to be covariant. The delegate declaration is as follows:

public TResult Func<in T, out TResult>(T arg)

In everyday development, you’re likely to use existing variant interfaces and delegates more often than you declare them. A few restrictions exist in terms of the type arguments you can use. Let’s look at them now.

4.4.3. Restrictions on using variance

To reiterate a point made earlier, variance can be declared only in interfaces and delegates. That variance isn’t inherited by classes or structs implementing interfaces; classes and structs are always invariant. As an example, suppose you were to create a class like this:

public class SimpleEnumerable<T> : IEnumerable<T>   1
{
                                                    2
}

• 1 The out modifier isn’t permitted here.
• 2 Implementation

You still couldn’t convert from SimpleEnumerable<string> to a SimpleEnumerable<object>. You could convert from SimpleEnumerable<string> to IEnumerable<object> using the covariance of IEnumerable<T>.

Let’s assume you’re dealing with a delegate or interface with some covariant or contravariant type parameters. What conversions are available? You need definitions to explain the rules:

• A conversion involving variance is called a variance conversion.
• Variance conversion is one example of a reference conversion. A reference conversion is one that doesn’t change the value involved (which is always a reference); it only changes the compile-time type.
• An identity conversion is a conversion from one type to the same type as far as the CLR is concerned. This might be the same type from a C# perspective, too (from string to string, for example), or it might be between types that are different only as far as the C# language is concerned, such as from object to dynamic.

Suppose you want to convert from IEnumerable<A> to IEnumerable<B> for some type arguments A and B. That’s valid if there’s an identity or implicit reference conversion from A to B. For example, these conversions are valid:

• IEnumerable<string> to IEnumerable<object>: there’s an implicit reference conversion from a class to its base class (or its base class’s base class, and so forth).
• IEnumerable<string> to IEnumerable<IConvertible>: there’s an implicit reference conversion from a class to any interface it implements.
• IEnumerable<IDisposable> to IEnumerable<object>: there’s an implicit reference conversion from any reference type to object or dynamic.

These conversions are invalid:

• IEnumerable<object> to IEnumerable<string>: there’s an explicit reference conversion from object to string, but not an implicit one.
• IEnumerable<string> to IEnumerable<Stream>: the string and Stream classes are unrelated.
• IEnumerable<int> to IEnumerable<IConvertible>: there’s an implicit conversion from int to IConvertible, but it’s a boxing conversion rather than a reference conversion.
• IEnumerable<int> to IEnumerable<long>: there’s an implicit conversion from int to long, but it’s a numeric conversion rather than a reference conversion.

As you can see, the requirement that the conversion between type arguments is a reference or identity conversion affects value types in a way that you might find surprising.

That example using IEnumerable<T> has only a single type argument to consider. What about when you have multiple type arguments? Effectively, they’re checked pairwise from the source of the conversion to the target, making sure that each conversion is appropriate for the type parameter involved.

To put this more formally, consider a generic type declaration with n type parameters: T<X₁, ..., X_n>. A conversion from T<A₁, ..., A_n> to T<B₁, ..., B_n> is considered in terms of each type parameter and pair of type arguments in turn. For each i between 1 and n:

• If X_i is covariant, there must be an identity or implicit reference conversion from A_i to B_i.
• If X_i is contravariant, there must be an identity or implicit reference conversion from B_i to A_i.
• If X_i is invariant, there must be an identity conversion from A_i to B_i.

To put this into a concrete example, let’s consider Func<in T, out TResult>. The rules mean the following:

• There’s a valid conversion from Func<object, int> to Func<string, int> because
  • The first type parameter is contravariant, and there’s an implicit reference conversion from string to object.
  • The second type parameter is covariant, and there’s an identity conversion from int to int.
• There’s a valid conversion from Func<dynamic, string> to Func<object, IConvertible> because
  • The first type parameter is contravariant, and there’s an identity conversion from dynamic to object.
  • The second type parameter is covariant, and there’s an implicit reference conversion from string to IConvertible.
• There’s no conversion from Func<string, int> to Func<object, int> because
  • The first type parameter is contravariant, and there’s no implicit reference conversion from object to string.
  • The second type parameter doesn’t matter; the conversion is already invalid because of the first type parameter.

Don’t worry if all of this is a bit overwhelming; 99% of the time you won’t even notice you’re using generic variance. I’ve provided this detail to help you just in case you receive a compile-time error and don’t understand why.^[3] Let’s wrap up by looking at a couple of examples of when generic variance is useful.

³

If this proves insufficient for a particular error, I suggest turning to the third edition, which has even more detail.

4.4.4. Generic variance in practice

A lot of the time, you may end up using generic variance without even being conscious of doing so, because things just work as you’d probably want them to. There’s no particular need to be aware that you’re using generic variance, but I’ll point out a couple of examples of where it’s useful.

First, let’s consider LINQ and IEnumerable<T>. Suppose you have strings that you want to perform a query on, but you want to end up with a List<object> instead of a List<string>. For example, you may need to add other items to the list afterward. The following listing shows how before covariance, the simplest way to do this would be to use an extra Cast call.

Listing 4.20. Creating a List<object> from a string query without variance

IEnumerable<string> strings = new[] { "a", "b", "cdefg", "hij" };
List<object> list = strings
    .Where(x => x.Length > 1)
    .Cast<object>()
    .ToList();

That feels annoying to me. Why create a whole extra step in the pipeline just to change the type in a way that’ll always work? With variance, you can specify a type argument to the ToList() call instead, to specify the type of list you want, as in the following listing.

Listing 4.21. Creating a List<object> from a string query by using variance

IEnumerable<string> strings = new[] { "a", "b", "cdefg", "hij" };
List<object> list = strings
    .Where(x => x.Length > 1)
    .ToList<object>();

This works because the output of the Where call is an IEnumerable<string>, and you’re asking the compiler to treat the input of the ToList() call as an IEnumerable<object>. That’s fine because of variance.

I’ve found contravariance to be useful in conjunction with IComparer<T>, the interface for ordering comparisons of another type. As an example, suppose you have a Shape base class with an Area property and then Circle and Rectangle derived classes. You can write an AreaComparer that implements IComparer<Shape>, and that’s fine for sorting a List<Shape> in place using List<T>.Sort(). But if you have a List<Circle> or a List<Rectangle>, how do you sort that? Various workarounds existed before generic variance, but the following listing shows how it’s trivial now.

Listing 4.22. Sorting a List<Circle> with an IComparer<Shape>

List<Circle> circles = new List<Circle>
{
    new Circle(5.3),
    new Circle(2),
    new Circle(10.5)
};
circles.Sort(new AreaComparer());
foreach (Circle circle in circles)
{
    Console.WriteLine(circle.Radius);
}

The full source for the types used by listing 4.22 is in the downloadable code, but they’re as simple as you’d expect them to be. The key point is that you can convert AreaComparer to IComparer<Circle> for the Sort method call. That wasn’t the case before C# 4.

If you declare your own generic interfaces or delegates, it’s always worth considering whether the type parameters can be covariant or contravariant. I wouldn’t normally try to force the issue if it doesn’t fall out that way naturally, but it’s worth taking a moment to think about it. It can be annoying to use an interface that could have variant type parameters but where the developer just hadn’t considered whether it might be useful to someone.

Summary

• C# 4 supports dynamic typing, which defers binding from compile time to execution time.
• Dynamic typing supports custom behavior via IDynamicMetaObjectProvider and the DynamicObject class.
• Dynamic typing is implemented with both compiler and framework features. The framework optimizes and caches heavily to make it reasonably efficient.
• C# 4 allows parameters to specify default values. Any parameter with a default value is an optional parameter and doesn’t have to be provided by the caller.
• C# 4 allows arguments to specify the name of the parameter for which it’s intended to provide the value. This works with optional parameters to allow you to specify arguments for some parameters but not others.
• C# 4 allows COM primary interop assemblies (PIAs) to be linked rather than referenced, which leads to a simpler deployment model.
• Linked PIAs expose variant values via dynamic typing, which avoids a lot of casting.
• Optional parameters are extended for COM libraries to allow ref parameters to be optional.
• Ref parameters in COM libraries can be specified by value.
• Generic variance allows safe conversions for generic interfaces and delegates based on whether values act as input or output.