Photo by Alfred T. Palmer. Library of Congress, Prints & Photographs Division, FSA-OWI Collection, Reproduction Number: LC-USE6-D-005032.
All the parts of an airplane engine, which has just undergone severe tests in a Midwest plant, are spread out for minute inspection. Continental Motors, Michigan, February 1942.
Sometimes a Smoke Test (13) is not enough to test a change in detail when you are working on a module, especially when you are working on new code. This pattern shows you how to test detailed changes so that you can ensure the quality of your codeline.
Checking that a class, module, or function still works after you make a change is a basic procedure that will help you maintain stability in your software development. It is also easier to understand what can go wrong at a low-level interface than it is at a system level. On the other hand, testing small-scale units all the time can become tedious.
Integration is where most of the problems become visible, but when you have the results of a failed integration test, you are still left with the question, What broke? Also, testing integration-level functions can take longer to set up because they require many pieces of the system to be stable. You want to be able to see whether any incremental change to your code broke something, so being able to run the tests as often as you like has benefits. You also want to run comprehensive tests on the item you are changing before check-in.
Since a smoke test is by its nature somewhat superficial, you want to be able to ensure that each part of a system works reasonably well. When a system test or a smoke test fails, you want to figure out what part of the system broke. You want to be able to run quick tests in development to see the effect of a change. Additional testing layers add time. Tests that are too complex take more effort to debug than the value they add is worth.
We want to isolate integration issues from local changes, and we want to test the contracts that each element provides locally.
A unit test tests fine-grained elements of a component to see that they obey their contract. A good unit test has the following properties (Beck 2000).
It is automatic and self-evaluating. A unit test can report a boolean result automatically. A user should not have to look at the detailed test results unless there is an error.
It is fine-grained. Any significant interface method on a class should test using known inputs. It is not necessary to write tests to verify trivial methods such as accessors and setters. To put it simply, the test tests things that might break.
It is isolated. A unit test does not interact with other tests. Otherwise, one test that fails may cause others to fail.
It tests the contract. The test should be self-contained so that external changes do not affect the results. If an external interface changes, you should update the test to reflect this.
It is simple to run. You should be able to run a unit test by using a simple command line or a graphical tool. No setup should be involved.
You should run unit tests
While you are coding
Just before checking in a change and after updating your code to the current version
You can also run all your unit tests when you are trying to find a problem with a smoke test or a regression test, or in response to a user problem report.
Try to use a testing framework such as JUnit (or CppUnit, PyUnit, and other derived frameworks). This will enable you to focus on the unit tests and not distract yourself with testing infrastructure.
Unit testing is indispensable when making changes to the structure of the code that should not affect behavior, such as when you are refactoring (Fowler 1999).
Grady Booch in Object Solutions suggests that during evolution you carry out unit testing of all new classes and objects but also apply regression testing to each new complete release (Booch 1996).
I've found that if I can't come up with a good unit test for a class or set of classes, I should make sure that my design is not overly complicated and not too abstract.
Writing unit tests can be tedious. You should try to use a framework such as JUnit to simplify some of the tedious parts of writing test cases.
If your public interface is narrow, but you want to test other functions, you need to decide whether to open your interface to allow for testing or do something else. There are a number of approaches to this problem.
Unit testing is a key part of Extreme Programming (Beck 2000; Jeffries et al. 2000).
To simplify your testing, try the testing framework JUnit (http://www.junit.org) for Java programming and the related frameworks for many other languages, available at http://www.xprogramming.com.
The Art of Software Testing, by Glen Meyers (Myers 1979), is a classic book on testing and has a good discussion on black-box versus white-box testing. The book is based on work on mainframe systems, but it is still useful.