The wrox.com code downloads for this chapter are found at www.wrox.com/go/proalm3ed on the Download Code tab. The files are in the Chapter 19 download folder and individually named as shown throughout this chapter.
Programmatic unit testing involves writing code to verify a system at a lower and more granular level than with other types of testing. It is used by programmers for programmers, and is quickly becoming standard practice at many organizations. All editions of Visual Studio include unit testing features that are fully integrated with the IDE and with other features (such as reporting and source control). Developers no longer need to rely on third-party utilities (such as NUnit) to perform their unit testing, although they still have the option to use them and, in fact, can integrate them into Visual Studio using the test adapter framework.
This chapter describes the concepts behind unit testing, why it is important, and how to create effective unit test suites. You learn about the syntax of writing unit tests, and you see how to work with Visual Studio's integrated features for executing and analyzing those tests. The discussion then goes into more detail about the classes available to you when writing your unit tests, including the core
class and many important attributes.Assert
You find out how Visual Studio enables the generation of unit tests from existing code, as well as the generation of member structures when writing unit tests. And you delve into Microsoft Fakes, a technology in Visual Studio 2013 that enables you to shim and stub your code for easier testing. Finally, you take a brief look at the test adapter framework in Visual Studio 2013 and how you can use that framework to utilize third-party testing frameworks in your testing process.
You've likely encountered a number of traditional forms of testing. Your quality assurance staff may run automated or manual tests to validate behavior and appearance. Load tests may be run to establish that performance metrics are acceptable. Your product group might run user acceptance tests to validate that systems do what the customers expect. Unit testing takes another view. Unit tests are written to ensure that code performs as the programmer expects.
Unit tests are generally focused at a lower level than other testing, establishing that underlying features work as expected. For example, an acceptance test might walk a user through an entire purchase. A unit test might verify that a
class correctly defends against adding an item with a negative quantity.ShoppingCart
Unit testing is an example of white box testing, where knowledge of internal structures is used to identify the best ways to test the system. This is a complementary approach to black box testing, where the focus is not on implementation details but rather on overall functionality compared to specifications. You should leverage both approaches to effectively test your applications.
Unit testing as a concept has been around for decades. However, in recent times, the process of performing unit tests by writing code to execute those tests has become popular. This form of programmatic unit testing is now what many people refer to as a “unit test” — and sometimes people use the term “unit test” to cover all forms of testing conducted using the programmatic unit testing frameworks, even if those tests are actually not tests of the unit of code, but are actually full integration tests.
A common reaction to unit testing is to resist the approach because the tests seemingly make more work for a developer. However, unit testing offers many benefits that may not be obvious at first.
The act of writing tests often uncovers design or implementation problems. The unit tests serve as the first users of your system, and they frequently identify design issues or functionality that is lacking. The act of thinking about tests causes the developer to question the requirements of the application, and, therefore, seek clarification from the business very early in the lifecycle of the software development project. This makes things easy and inexpensive to rectify as the clarification is received.
After a unit test is written, it serves as a form of living documentation for the use of the target system. Other developers can look to an assembly's unit tests to see example calls into various classes and members. An important benefit of unit tests for framework APIs is that the tests introduce a dependency at compile time, making it trivial to determine if any code changes have affected the contract represented by the API.
Perhaps one of the most important benefits is that a well-written test suite provides the original developer with the freedom to pass the system off to other developers for maintenance and further enhancement, knowing that their intentions of how the code would be used are fully covered by tests. Should those developers introduce a bug in the original functionality, there is a strong likelihood that those unit tests can detect that failure and help diagnose the issue. In addition, because there is a full set of unit tests making up the regression tests, it is a simple task for the maintenance team to introduce a new test that demonstrates the bug first, and then confirm that it is correctly fixed by the code modification. Meanwhile, the original developer can focus on current tasks.
It takes the typical developer time and practice to become comfortable with unit testing. After a developer has saved enough time by using unit tests, he latches on to them as an indispensable part of the development process.
Unit testing does require more explicit coding, but this cost will be recovered, and typically exceeded, when you spend much less time debugging your application. In addition, some of this cost is typically already hidden in the form of a test console or Windows-based applications that a developer might have previously used as a test harness. Unlike these informal testing applications, which are frequently discarded after initial verification, unit tests become a permanent part of the project, and ideally run each time a change is made to help ensure that the system still functions as expected. Tests are stored in source control as part of the same solution with the code they verify and are maintained along with the code under test, making it easier to keep them synchronized.
It is difficult to overstate the importance of comprehensive unit test suites. They enable developers to hand off a system to other developers with confidence that any changes they make should not introduce undetected side effects. However, because unit testing provides only one view of a system's behavior, no amount of unit testing should ever replace integration, acceptance, and load testing.
Because unit tests are themselves code, you are generally unlimited in the approaches you can take when writing them. However, you should follow some general guidelines:
The final proof of your unit testing's effectiveness is when it saves you more time during development and maintenance than you spent creating the tests. Experience has shown that you will realize this savings many times over.
Unit testing is not a new concept. Before Visual Studio introduced integrated unit testing, developers needed to rely on third-party frameworks. The de facto standard for .NET unit testing has been an Open Source package called NUnit. NUnit has its roots as a .NET port of the Java-based JUnit unit testing framework. JUnit is itself a member of the extended xUnit family.
There are many similarities between NUnit and the unit testing framework in Visual Studio. The structure and syntax of tests and the execution architecture are conveniently similar. If you have existing suites of NUnit-based tests, it is generally easy to convert them for use with Visual Studio.
Visual Studio's implementation of unit testing is not merely a port of NUnit. Microsoft has added a number of features including IDE integration, code generation, new attributes, and enhancements to the
class. The implementation is part of a broader testing platform across both Visual Studio and Team Foundation Server. For example, these tests can be run through the test controller/agent system, associated with test case work items, and queued with automated tests to run from Microsoft Test Manager.Assert
Unit testing is a feature available in all editions of Visual Studio. Unit tests can be written against all types of applications, from console applications to Windows Store apps. This section describes how to create, execute, and manage unit tests.
Unit tests are normal code, identified as unit tests through the use of attributes. Like NUnit 2.0 and later, Visual Studio uses .NET reflection to inspect assemblies to find unit tests.
You also use attributes to identify other structures used in your tests and to indicate desired behaviors.
This section takes a slower approach to creating a unit test than you will in your normal work. This gives you a chance to examine details you could miss using only the built-in features that make unit testing easier. Later in this chapter, you look at the faster approaches.
In order to have something to test, create a new C# class library project named
. Rename the default ExtendedMath
to Class1.cs
. You will add code to compute the Fibonacci Sequence for a given number. The Fibonacci Sequence, as you may recall, is a series of numbers where each term is the sum of the prior two terms. The first six terms, starting with an input factor of 1, are 1, 1, 2, 3, 5, and 8.Functions.cs
Open
and insert the following code:Functions.cs
namespace ExtendedMath { public static class Functions { public static int Fibonacci(int factor) { if (factor < 2) { return (factor); } int x = Fibonacci(--factor); int y = Fibonacci(--factor); return x + y; } } }
You are now ready to create unit tests to verify the
implementation. Unit tests are recognized as tests only if they are contained in separate projects called test projects. Test projects can contain any of the test types supported in Visual Studio. Add a test project named Fibonacci
to your solution by adding a new project and selecting the Unit Test Project template. If the test project includes any sample tests for you (such as ExtendedMathTesting
) then you can safely delete them. Because you will be calling objects in your UnitTest1.cs
project, make a reference to that class library project from the test project. You may notice that a reference to the ExtendedMath
assembly has already been made for you. This assembly contains many helpful classes for creating units tests. You'll use many of these throughout this chapter.Microsoft.VisualStudio.QualityTools.UnitTestFramework.dll
After you have created a new test project, add a new class file (not a unit test; that file type is covered later) called
. You use this class to contain the unit tests for the FunctionsTest.cs
class. You use objects from the Functions
project and the ExtendedMath
assembly mentioned earlier, so add UnitTestFramework
statements at the top so that the class members do not need to be fully qualified:using
using Microsoft.VisualStudio.TestTools.UnitTesting;
To enable Visual Studio to identify a class as potentially containing unit tests, you must assign the
attribute. If you forget to add the TestClass
attribute, the unit tests methods in your class are not recognized. Unit tests are required to be hosted within public classes, so don't forget to include the TestClass
descriptor for the class.public
To indicate that the
class contains unit tests, add the FunctionsTest
attribute to its declaration:TestClass
namespace ExtendedMath
{
[TestClass]
public class FunctionsTest
{
}
}
Note also that the parentheses after an attribute are optional if you are not passing parameters to the attribute. For example,
and [TestClass()]
are equivalent.[TestClass]
Having identified the class as a container of unit tests, you're ready to add your first unit test. A unit test method must be public, nonstatic, accept no parameters, and have no return value. To differentiate unit test methods from ordinary methods, they must be decorated with the
attribute.TestMethod
Add the following code inside the
class:FunctionsTest
[TestMethod] public void FibonacciTest() { }
You have the shell of a unit test, but how do you test? A unit test indicates failure to Visual Studio by throwing an exception. Any test that does not throw an exception is considered to have passed, except in the case of the
attribute, which is described later.ExpectedException
The unit testing framework defines the
object. This object exposes many members, which are central to creating unit tests. You learn more about Assert
later in the chapter.Assert
Add the following code to the
:FibonacciTest
[TestMethod] public void FibonacciTest() {const int FACTOR = 8;
const int EXPECTED = 21;
int actual = ExtendedMath.Functions.Fibonacci(FACTOR);
Assert.AreEqual(EXPECTED, actual);
}
This uses the
method to compare two values, the value you expect and the value generated by calling the Assert.AreEqual
method. If they do not match, an exception is thrown, causing the test to fail.Fibonacci
When you run these tests, you see the Test Explorer window. Success is indicated with a green check mark and failure with a red X. A special inconclusive result (described later in this chapter in the section “Using the Assert Methods”) is represented by a question mark.
To see a failing test, change the
constant from EXPECTED
to 21
and rerun the test. The Test Explorer window shows the test as failed. The error message section at the bottom of the window provides details about the failure. In this case, the error message shows the following:22
Assert.AreEqual failed. Expected:<22>, Actual:<21>
This indicates that either the expected value is wrong, or the implementation of the
algorithm is wrong. Fortunately, because unit tests verify a small amount of code, the job of finding the source of bugs is made easier.Fibonacci
After you have created a unit test and rebuilt your project, Visual Studio automatically inspects your projects for unit tests. All unit tests that are found are displayed in the Test Explorer window, shown in Figure 19.1. This window is used for managing and running tests. From this window you have multiple options, such as running all your unit tests, running only tests that have not been run yet, and viewing unit test run results.
You also have the capability to run a unit test directly from code. To do that, open the unit test and navigate to the method. Right-click the unit test method in the code and, from the context menu, select Run Tests. The selected test method will execute.
Because unit tests are simply methods with special attributes applied to them, they can be debugged just like other code.
You can set breakpoints anywhere in your code, not just in your unit tests. For example, the
calls into the FibonacciTest
method. You could set a breakpoint in either method and have execution pause when that line of code is reached.ExtendedMath.Fibonacci
However, setting program execution does not pause at your breakpoints unless you run your unit test in debugging mode. The Test Explorer window enables you to right-click a test and select Debug Selected Tests. The selected unit tests are run in debug mode, pausing execution at any enabled breakpoints and giving you a chance to evaluate and debug your unit test or implementation code as necessary.
This section describes in detail the attributes and methods available for creating unit tests. You can find all the classes and attributes mentioned in this section in the
namespace.Microsoft.VisualStudio.TestTools.UnitTesting
Often, you need to configure a resource that is shared among your tests. Examples might be a database connection, a log file, or a shared object in a known default state. You might also need ways to clean up from the actions of your tests, such as closing a shared stream or rolling back a transaction.
The unit test framework offers attributes to identify such methods. They are grouped into three levels: Test, Class, and Assembly. The levels determine the scope and timing of execution for the methods they decorate. Table 19.1 describes these attributes.
Table 19.1 Unit Test Framework Attributes
Attributes | Frequency and Scope |
,
|
Executed before ( ) or after ( ) any of the class's unit tests are run |
,
|
Executed a single time before or after any of the tests in the current class are run |
,
|
Executed a single time before or after any number of tests in any of the assembly's classes are run |
Having methods with these attributes is optional, but do not define more than one of each attribute in the same context. Also, keep in mind that you cannot guarantee the order in which your unit tests will be run, and that should govern what functionality you place in each of these methods.
Use the
attribute to create a method that is executed one time before every unit test method in the current class. Similarly, TestInitialize
marks a method that is always run immediately after each test. Like unit tests, methods with these attributes must be public, nonstatic, accept no parameters, and have no return values.TestCleanup
Following is an example test for a simplistic shopping cart class. It contains two tests and defines the
and TestInitialize
methods:TestCleanup
using Microsoft.VisualStudio.TestTools.UnitTesting; [TestClass] public class ShoppingCartTest { private ShoppingCart cart;[TestInitialize]
public void TestInitialize() { cart = new SomeClass(); cart.Add(new Item("Test")); }[TestCleanup]
public void TestCleanup() { // Not required - here for illustration cart.Dispose(); } [TestMethod] public void TestCountAfterAdd() { int expected = cart.Count + 1; cart.Add(new Item("New Item")); Assert.AreEqual(expected, cart.Count); } [TestMethod] public void TestCountAfterRemove() { int expected = cart.Count - 1; cart.Remove(0); Assert.AreEqual(expected, cart.Count); } }
When you run both tests,
and TestInitialize
are both executed twice. TestCleanup
is run immediately before each unit test and TestInitialize
immediately after.TestCleanup
The
and ClassInitialize
attributes are used very similarly to ClassCleanup
and TestInitialize
. The difference is that these methods are guaranteed to run once and only once no matter how many unit tests are executed from the current class. Unlike TestCleanup
and TestInitialize
, these methods are marked static and accept a TestCleanup
instance as a parameter.TestContext
The importance of the
instance is described later in this chapter.TestContext
The following code demonstrates how you might manage a shared logging target using class-level initialization and cleanup with a logging file:
private System.IO.File logFile;[ClassInitialize]
public static void ClassInitialize(TestContext context) { // Code to open the logFile object }[ClassCleanup]
public static void ClassCleanup(TestContext context) { // Code to close the logFile object }
You could now reference the
object from any of your unit tests in this class, knowing that it will automatically be opened before any unit test is executed and closed after the final test in the class has completed.logFile
The following code shows the flow of execution if you run both tests again:
ClassInitialize TestInitialize TestCountAfterAdd TestCleanup TestInitialize TestCountAfterRemove TestCleanup ClassCleanup
Where you might use
and ClassInitialize
to control operations at a class level, use the ClassCleanup
and AssemblyInitialize
attributes for an entire assembly. For example, a method decorated with AssemblyCleanup
is executed once before any test in that current assembly, not just those in the current class. As with the class-level initialize and cleanup methods, these must be static and accept a AssemblyInitialize
parameter:TestContext
[AssemblyInitialize]
public static void AssemblyInitialize(TestContext context) { // Assembly-wide initialization code }[AssemblyCleanup]
public static void AssemblyCleanup(TestContext context) { // Assembly-wide cleanup code }
Consider using
and AssemblyInitialize
in cases where you have common operations spanning multiple classes. Instead of having many per-class initialize and cleanup methods, you can refactor these to single assembly-level methods.AssemblyCleanup
The most common way to determine success in unit tests is to compare an expected result against an actual result. The
class features many methods that enable you to make these comparisons quickly.Assert
Of the various
methods, you will likely find the most use for Assert
and AreEqual
. As their names imply, you are comparing an expected value to a supplied value. If the operands are not value-equivalent (or are equivalent for AreNotEqual
) then the current test fails.AreNotEqual
A third, optional argument can be supplied: a string that will be displayed along with your unit test results, which you can use to describe the failure. Additionally, you can supply parameters to be replaced in the string, just as the
method supports. The string message should be used to explain why failing that String.Format
is an error. If you have multiple Assert
s in a single test method, then it is very useful to provide a failure message string on every Assert
so that you can very quickly identify which Assert
failed:Assert
[TestMethod] public void IsPrimeTest() { const int FACTOR = 5; const bool EXPECTED = true; bool actual = CustomMath.IsPrime(FACTOR);Assert.AreEqual(EXPECTED, actual, “The number {0} should have been computed as
prime, but was not.”, FACTOR);
}
and Assert.AreEqual
have many parameter overloads, accepting types such as AreNotEqual
, string
, double
, int
, float
, and object
types. Take the time to review the overloads in the Object Browser.generic
When using these methods with two string arguments, one of the overrides allows you to optionally supply a third argument. This is a boolean, called
, that indicates whether the comparison should be case-insensitive. The default comparison is case-sensitive.ignoreCase
Working with floating-point numbers involves a degree of imprecision. You can supply an argument that defines a delta by which two numbers can differ yet still pass a test — for example, say you're computing square roots and decide that a “drift” of plus or minus 0.0001 is acceptable:
[TestMethod] public void SquareRootTest() { const double EXPECTED = 3.1622;const double DELTA = 0.0001;
double actual = CustomMath.SquareRoot(10);Assert.AreEqual(EXPECTED, actual, DELTA, “Root not within acceptable range”);
}
and AreSame
function in much the same manner as AreNotSame
and AreEqual
. The important difference is that these methods compare the references of the supplied arguments. For example, if two arguments point to the same object instance, then AreNotEqual
passes. Even when the arguments are exactly equivalent in terms of their state, AreSame
fails if they are not, in fact, the same object. This is the same concept that differentiates AreSame
from object.Equals
.object.ReferenceEquals
A common use for these methods is to ensure that properties return expected instances, or that collections handle references correctly. The following example adds an item to a collection and ensures that what you get back from the collection's indexer is a reference to the same item instance:
[TestMethod]
public void CollectionTest()
{
CustomCollection cc = new CustomCollection();
Item original = new Item("Expected");
cc.Add(original);
Item actual = cc[0];
Assert.AreSame(original, actual);
}
As you can probably guess,
and IsTrue
are used simply to ensure that the supplied expression is true or false as expected. Returning to the IsFalse
example, you can restate it as follows:IsPrimeNumberTest
[TestMethod] public void IsPrimeNumberTest() { const int FACTOR = 5;Assert.IsTrue(CustomMath.IsPrime(FACTOR), “The number {0} should have been
computed as prime, but was not.”, FACTOR);
}
Similar to
and IsTrue
, these methods verify that a given object type is either IsFalse
or not null
. Revising the collection example, this ensures that the item returned by the indexer is not null
:null
[TestMethod]
public void CollectionTest()
{
CustomCollection cc = new CustomCollection();
cc.Add(new Item("Added"));
Item item = cc[0];
Assert.IsNotNull(item);
}
simply ensures that a given object is an instance of an expected type. For example, suppose you have a collection that accepts entries of any type. You want to ensure that an entry you're retrieving is of the expected type, as shown here:IsInstanceOfType
[TestMethod]
public void CollectionTest()
{
UntypedCollection untyped = new UntypedCollection();
untyped.Add(new Item("Added"));
untyped.Add(new Person("Rachel"));
untyped.Add(new Item("Another"));
object entry = untyped[1];
Assert.IsInstanceOfType(entry, typeof(Person));
}
As you can no doubt guess,
tests to ensure that an object is not of the specified type.IsNotInstanceOfType
Use
to immediately fail a test. For example, you may have a conditional case that should never occur. If it does, call Assert.Fail
and an Assert.Fail
is thrown, causing the test to abort with failure. You may find AssertFailedException
useful when defining your own custom Assert.Fail
methods.Assert
enables you to indicate that the test result cannot be verified as a pass or fail. This is typically a temporary measure until a unit test (or the related implementation) has been completed. Assert.Inconclusive
can also be used to indicate that more work is needed to complete a unit test.Assert.Inconclusive
The
namespace includes a class, called Microsoft.VisualStudio.TestTools.UnitTesting
, that contains useful methods for testing the contents and behavior of collection types.CollectionAssert
Table 19.2 describes the methods supported by
.CollectionAssert
Table 19.2 CollectionAssert Methods
Method | Description |
|
Ensures that all elements are of an expected type |
|
Ensures that no items in the collection are
|
|
Searches a collection, failing if a duplicate member is found |
|
Ensures that two collections have reference-equivalent members |
|
Ensures that two collections do not have reference-equivalent members |
|
Ensures that two collections have value-equivalent members |
|
Ensures that two collections do not have value-equivalent members |
|
Searches a collection, failing if the given object is not found |
|
Searches a collection, failing if a given object is found |
|
Ensures that the first collection has members not found in the second |
|
Ensures that all elements in the first collection are found in the second |
|
Determines whether the specified instances are the same instance |
The following example uses some of these methods to verify various behaviors of a collection type,
. When this example is run, none of the assertions fails, and the test results in success. Note that proper unit testing would spread these checks across multiple smaller tests:CustomCollection
[TestMethod] public void CollectionTests() { CustomCollection list1 = new CustomCollection(); list1.Add("alpha"); list1.Add("beta"); list1.Add("delta"); list1.Add("delta");CollectionAssert.AllItemsAreInstancesOfType(list1, typeof(string));
CollectionAssert.AllItemsAreNotNull(list1);
CustomCollection list2 = (CustomCollection)list1.Clone();CollectionAssert.AreEqual(list1, list2);
CollectionAssert.AreEquivalent(list1, list2);
CustomCollection list3 = new CustomCollection(); list3.Add("beta"); list3.Add("delta");CollectionAssert.AreNotEquivalent(list3, list1);
CollectionAssert.IsSubsetOf(list3, list1);
CollectionAssert.DoesNotContain(list3, “alpha”);
CollectionAssert.AllItemsAreUnique(list3);
}
The final assertion,
, would have failed if tested against AllItemsAreUnique(list3)
because that collection has two entries of the string list1
.“delta”
Similar to
, the CollectionAssert
class contains methods that enable you to easily make assertions based on common text operations. Table 19.3 describes the methods supported by StringAssert
.StringAssert
Table 19.3 StringAssert Methods
Method | Description |
|
Searches a string for a substring and fails if not found |
|
Applies a regular expression to a string and fails if any matches are found |
|
Fails if the string does not end with a given substring |
|
Applies a regular expression to a string and fails if no matches are found |
|
Fails if the string does not begin with a given substring |
|
Determines whether the specified object instances are considered equal |
|
Determines whether the specified instances are in the same instance |
Following are some simple examples of these methods. Each of these assertions will pass:
[TestMethod] public void TextTests() { StringAssert.Contains("This is the searched text", "searched"); StringAssert.EndsWith("String which ends with searched", "ends with searched"); StringAssert.Matches("Search this string for whitespace", new System.Text.RegularExpressions.Regex(@"s+")); StringAssert.DoesNotMatch("Doesnotcontainwhitespace", new System.Text.RegularExpressions.Regex(@"s+")); StringAssert.StartsWith("Starts with correct text", "Starts with"); }
and Matches
accept a string and an instance of DoesNotMatch
. In the preceding example, a simple regular expression that looks for at least one whitespace character was used. System.Text.RegularExpressions.Regex
finds whitespace and Matches
does not find whitespace, so both pass.DoesNotMatch
Normally, a unit test that throws an exception is considered to have failed. However, you'll often want to verify that a class behaves correctly by throwing an exception. For example, you might provide invalid arguments to a method to verify that it properly throws an exception.
The
attribute indicates that a test succeeds only if the indicated exception is thrown. Not throwing an exception or throwing an exception of a different type results in test failure.ExpectedException
The following unit test expects that an
will be thrown:ObjectDisposedException
[TestMethod]
[ExpectedException(typeof(ObjectDisposedException))]
public void ReadAfterDispose()
{
CustomFileReader cfr = new CustomFileReader("target.txt");
cfr.Dispose();
string contents = cfr.Read(); // Should throw ObjectDisposedException
}
The
attribute supports a second, optional string argument. The ExpectedException
property of the thrown exception must match this string or the test fails. This enables you to differentiate between two different instances of the same exception type.Message
For example, suppose you are calling a method that throws a
for several different files. To ensure that it cannot find one specific file in your testing scenario, supply the message you expect as the second argument to FileNotFoundException
. If the exception thrown is not ExpectedException
and its FileNotFoundException
property does not match that text, the test fails.Message
You may define custom properties for your unit tests. For example, you may want to specify the author of each test and be able to view that property from the Test List Editor.
Use the
attribute to decorate a unit test, supplying the name of the property and a value:TestProperty
[TestMethod]
[TestProperty(“Author”, “Deborah”)]
public void ExampleTest()
{
// Test logic
}
Now, when you view the properties of that test, you see a new entry,
, with the value Author
. If you change that value from the Properties window, the attribute in your code is automatically updated.Deborah
Unit tests normally have a reference to a
instance. This object provides runtime features that might be useful to tests, such as details of the test itself, the various directories in use, and several methods to supplement the details stored with the test's results. TestContext
is also very important for data-driven unit tests, as you see later.TestContext
Several methods are especially useful to all unit tests. The first,
, enables you to insert text into the results of your unit test. This can be useful for supplying additional information about the test, such as parameters, environment details, and other debugging data that would normally be excluded from test results.WriteLine
Here is a simple example of a unit test that accesses the
to send a string containing the test's name to the results:TestContext
[TestClass] public class TestClass { private TestContext testContextInstance; public TestContext TestContext { get { return testContextInstance; } set { testContextInstance = value; } } [TestMethod] public void TestMethod1() { TestContext.WriteLine("This is test {0}", TestContext.TestName); }
The
method enables you to add a file, at runtime, to the results of the test run. The file you specify is copied to the results directory alongside other results content. For example, this may be useful if your unit test is validating an object that creates or alters a file, and you would like that file to be included with the results for analysis.AddResultFile
Finally, the
and BeginTimer
methods enable you to create one or more named timers within your unit tests. The results of these timers are stored in the test run's results.EndTimer
One of the many features people have asked for from Visual Studio is for it to ship with a mocking framework. A mocking framework enables you to provide a fake implementation of a type or object, along with logic that verifies how calls were made to the mocked object. There are several good mocking frameworks currently available in the community, including Moq, Rhino, and NMock. Although these tools have strong followings and a good reputation, there was still a need to provide a mocking framework to customers who may be unable to utilize open source or third-party tools. Hence, the Microsoft Fakes framework in Visual Studio 2013.
Developers often need to test individual components of their code in isolation from other components. Commonly, this is performed using dummy implementations of code that are not currently being tested. In reality, it can be very difficult to implement this dummy code because the actual code being tested is expecting real code on the other end. The Fakes framework helps developers create, maintain, and inject dummy implementation of components into the developer's unit test, making it quick and easy to isolate specific unit tests from the actual environment.
Currently, the Fakes framework focuses on two kinds of test fakes for .NET programming: stubs and shims.
Stubs are concrete implementations of interfaces and abstract classes that can be passed into the system being tested. A developer provides method implementations via .NET delegates or lambdas. A stub is realized by a distinct type that is generated by the Fakes framework. As such, all stubs are strongly typed. You cannot use stubs for static or non-overridable methods. Instead, you should use shims in those instances.
Shims are runtime method interceptors. They enable you to provide your own implementation for almost any method available to your code in .NET, including types and methods from the .NET base class libraries.
Stub types and shim types are built on different underlying technologies. As such, they have different requirements, properties, and use cases. Table 19.4 provides a list of the different aspects to consider when choosing between a stub and a shim.
Table 19.4 Stubs versus Shims
Aspect | Stub/Shim | Reason |
Performance | Stub | The runtime code-rewriting used by shims introduces some performance issues at runtime. Stubs do not do this. |
Static Methods | Shim | Stub can only influence overridable methods. They cannot be used for static, non-virtual, and sealed virtual methods. |
Internal Types | Stub/Shim | Both stubs and shims can be used with internal types made accessible through the attribute. |
Private Methods | Shim | Shim types can replace private methods if all the types on the method signature are visible. |
Interfaces/Abstract Methods | Stub | Stubs implement interfaces and abstract methods that can be used for testing. Shims can't do this because they don't have method bodies. |
The general recommendation is to use stubs to isolate dependencies within your code base by hiding components behind interfaces. You should use shims to isolate third-party components that don't provide a testing API.
Stubs are a part of the Fakes framework that enables you to easily isolate unit tests from the environment. You do this by generating a Fakes assembly, based on an actual target assembly. When the Fakes assembly is generated, a stub type is created for each non-sealed class and interface in the target assembly that contains virtual or abstract methods, properties, or events. The stub type provides a default implementation of each virtual member and adds a delegate property that you can customize to provide specific behavior.
For this example, you are going to make a list of books. To get started, create a new C# class library project named
. Rename the FakesUsingStubs
file to be Class1.cs
. Add the following code to the Book.cs
file to create a Book.cs
class:Book
public class Book { public int Isbn {get;set;} public int ListItemId {get;set;} public Book (int isbn, int listItemId) { Isbn = isbn; ListItemId = listItemId; } }
Now add the following class,
. This class contains a list of books and has a method, BookListToStub
, for adding a new book to the list:AddBookToList
public class BookListToStub { public int ListId {get;set;} public int CustomerId {get;set;} private List<Book> _books = new List<Book>(); public ReadOnlyCollection<Book> Books {get; set;} private IListSave _listSave; public BookListToStub(int listId, int customerId, IListSave listSave) { ListId = listId; CustomerId = customerId; _listSave = listSave; Books = new ReadOnlyCollection<Book>(_books); } public void AddBookToList(int isbn) { var bookItemId = _listSave.SaveListItem(ListId, isbn); _books.Add(new Book(isbn, bookItemId)); } }
The saving functionality is implemented using a class called
that implements the interface ListSave
. Add this class to your project:IListSave
public interface IListSave { int SaveListItem(int listId, int isbn); } public class ListSave : IListSave { public int SaveListItem(int listId, int isbn) { throw new NotImplementedException("Forgot to add SQL Code"); } }
As you can see from the preceding snippet of code, the actual code to perform the save has not been implemented yet. Normally, that would make testing the save functionality difficult. Microsoft Fakes enables you to stub out the saving functionality so that you can test the rest of the code even though the saving functionality does not currently exist.
Right-click your solution and add a new C# unit test project to the solution, named
Rename the default unit test file to FakesUsingStubs.Tests.
. Add a reference to the BookListToStubTests.cs
project by right-clicking the References folder, selecting Add Reference from the context menu, and then selecting the FakesUsingStubs
assembly. To create your shims and stubs, right-click the FakesUsingStubs
assembly reference, and select Add Fakes Assembly (only available with Visual Studio Ultimate 2013 and Visual Studio Premium 2013. When you do this, several things happen behind the scenes:FakesUsingStubs
.fakes
is created in the Fakes folder within your project. This file controls how your fakes are generated.StubX
and ShimX
based on the type they target.FakeAssemblies
.To complete your testing, you need to mock out the database call to isolate the logic in the
method. Open the AddBookToList
file, and add the following test method:BookListToStubTests.cs
[TestMethod] public void AddBook_BookShouldBeAddedToList() { int bookItemId = 77; int listId = 1; int customerId = 25; int isbn = 12345; //Stub IListSave var listSave = new Fakes.StubIListSave(); listSave.SaveListItemInt32Int32 = (l,i) => bookItemId; var list = new BookListToStub(listId, customerId, listSave); list.AddBookToList(isbn); var book = list.Books[0]; Assert.AreEqual(isbn, book.Isbn); }
When you created the Fakes assembly, you created a stub method for the
interface that can be overridden using a delegate. In this case, you have created a delegate that returns the IListSave
, which is what you would expect from the save functionality that is not implemented. If you execute the unit test, it executes with no errors.bookItemId
Shims are runtime method interceptors. They enable you to provide your own implementation for almost any method available to your code in .NET, including types and methods from the .NET base class libraries.
For this example, you are again going to make a list of books, but with some changes to the code. Instead of implementing an interface, you are going to implement a static class for the data access layer. To get started, created a new C# class library project named
. Rename the FakesUsingShims
file to be Class1.cs
. Add the following code to the Book.cs
file, to create a Book.cs
class:Book
public class Book { public int Isbn {get;set;} public int ListItemId {get;set;} public Book (int isbn, int listItemId) { Isbn = isbn; ListItemId = listItemId; } }
Now add the following class,
. This class contains a list of books and has a method, BookListToShim
, for adding a new book to the list. Notice this method now makes use of a new class, AddBookToList
, which contains the saving functionality:DAL
public class BookListToShim { public int ListId {get;set;} public int CustomerId {get;set;} private List<Book> _books = new List<Book>(); public ReadOnlyCollection<Book> Books {get; set;} public BookListToShim(int listId, int customerId) { ListId = listId; CustomerId = customerId; Books = new ReadOnlyCollection<Book>(_books); } public void AddBookToList(int isbn) { var bookItemId = DAL.SaveListItem(ListId, isbn); _books.Add(new Book(isbn, bookItemId)); } }
Create a new C# class file named
. Add the following code to implement the saving functionality:DAL.cs
public static class DAL { public static int SaveListItem(int listId, int isbn) { throw new NotImplementedException("Forgot to add SQL Code"); } }
As before, the actual code to perform the save has not been implemented yet. Microsoft Fakes allows you to shim out the saving functionality, enabling you to test the rest of the code even though the saving functionality does not currently exist.
Right-click your solution and add a new C# unit test project to the solution, named
. Rename the default unit test file to FakesUsingShims.Tests
. Add a reference to the BookListToShimTests.cs
project by right-clicking the References folder, selecting Add Reference from the context menu, and then selecting the FakesUsingShims
assembly reference. Right-click the FakesUsingShims
assembly reference and select Add Fakes Assembly to generate the shim and stub code.FakesUsingShims
To complete your testing, you need to mock out the database call to isolate the logic in the
method. Open the AddBookToList
file, and add the following test method:BookListToShimTests.cs
[TestMethod] public void AddBook_BookShouldBeAddedToList() { int bookItemId = 77; int listId = 1; int customerId = 25; int isbn = 12345; using (ShimsContext.Create()) { Fakes.ShimDAL.SaveListItemInt32Int32 = (l,i) => bookItemId; var list = new BookListToShim(listId, customerId); list.AddBookToList(isbn); var book = list.Books[0]; Assert.AreEqual(isbn, book.Isbn); } }
In the test method, you create a
, which enables you to scope the amount of shimming you are implementing. The rest of the code is very similar to the stubs example, and works in the same way, with a delegate being used to override the ShimsContext
method.SaveListItem
The unit testing framework in Visual Studio 2013 is extensible. This allows third-party unit testing frameworks, such as NUnit or XUnit, to create test adapters for Visual Studio unit testing. Now a developer can use any testing framework that provides an adapter for unit test creation.
You can download most third-party unit testing frameworks using the Visual Studio Extension Manager inside of Visual Studio, or you can get them directly from the Visual Studio Gallery on the MSDN website.
To download a third-party test framework adapter using the Visual Studio Extension Manager, open Visual Studio and select Tools ⇒ Extensions and Updates. The Extension and Updates dialog box opens. In the dialog box, select Online ⇒ Visual Studio Gallery ⇒ Tools ⇒ Testing, as shown in Figure 19.2.
Select the Unit Test framework to install, and click the Download button. If you already have a testing framework installed, you see a green check mark instead of a download button. Clicking the Download button automatically downloads and installs the testing framework. You need to restart Visual Studio 2013 before you can start using the testing framework.
After you have restarted Visual Studio, you can start creating unit tests using the new test framework, and you can execute those unit tests inside of Visual Studio 2013.
There are a couple of limitations with using test adapters. For example, even though you can run them using the Agile Test Runner as part of the Team Foundation Build process, you can't associate the automation with a test case. This means that you can't use them as part of your test controller/agent infrastructure.
Microsoft has brought the advantages of unit testing to the developer by fully integrating features with the Visual Studio development environment. If you're new to unit testing, this chapter has provided an overview of what unit testing is, and how you can create effective unit tests. This chapter examined the creation and management of unit tests and detailed the methods and attributes available in the unit test framework. You should be familiar with attributes for identifying your tests, as well as many of the options that the
class offers for testing behavior of code.Assert
You also learned about the Microsoft Fakes framework, and how you can use shims and stubs to help you test your code more effectively while isolating different systems in your environment. Finally, the chapter covered test adapters, how test adapters enable you to utilize third-party testing frameworks, and how you can install test adapters.
You should become familiar with the benefits of unit testing, keeping in mind that unit tests are not a replacement for other forms of testing, but they are a very strong supplement.
Obviously, testing is an important aspect to prove that your code is ready to be deployed into production. However, just because the code passes all the unit tests doesn't mean that it is necessarily ready to ship.
Chapter 20 examines the code analysis tools in Visual Studio 2013 that help you quickly look for common mistakes, security issues, or even violations of standards. You also find out how to use code metrics to identify parts of the systems that may prove difficult to maintain, how code cloning can help you find duplicate code to refactor in your solution, and how the new CodeLens feature provides detailed information directly in the code editor.