High-quality programs have automated tests. We need to use everything at our disposal to be sure that our software works. The golden rule is this: to be deliverable, a feature must have an automated unit test. Without an automated unit test, a feature cannot be trusted to work and should not be used. According to Kent Beck, in Extreme Programming Explained:

There are the following two essential points regarding the automated testing of program features:

• Automated: This means that there's no human judgment involved. The testing involves a script that compares actual responses to expected responses.
• Features: Elements of the software are tested in isolation to be sure that they work separately. In some contexts, features are quite broad concepts involving functionality observed by the user. When unit testing, features are often much smaller, and refer to behaviors of single software components. Each unit is enough software to implement a given feature. Ideally, it's a Python class. However, it can also be a larger unit, such as a module or package.

Python has two built-in testing frameworks, making it easy to write automated unit tests. We'll look at using both doctest and unittest for automated testing. doctest offers a very simple approach to test writing. The unittest package is much more sophisticated.

We'll also look at the popular pytest framework, which is simpler to use than unittest. In some cases, we'll write tests using the unittest.TestCase class, but execute the tests using pytest's sophisticated test discovery. In other cases, we'll do almost everything with pytest.

We'll look at some of the design considerations required to make testing practical. It's often necessary to decompose complex classes for testability. As we noted in Chapter 15, Design Principles and Patterns, we want expose dependencies to have a flexible design; the dependency inversion principle will also help to create testable classes.

For more ideas, read about Ottinger and Langr's FIRST unit test properties—Fast, Isolated, Repeatable, Self-Validating, and Timely. For the most part, Repeatable and Self-Validating require an automated test framework. Timely means that the test is written before the code under test. For more information, refer to http://pragprog.com/magazines/2012-01/unit-tests-are-first.

In this chapter, we will cover the following topics:

• Defining and isolating units for testing
• Using doctest to define test cases
• Using setup and teardown
• The TestCase class hierarchy
• Using externally defined expected results
• Using pytest and fixtures
• Automated integration testing or automated performance testing