We can use the unittest package to perform testing that isn't focused on a single, isolated class definition. As noted previously, we can use unittest automation to test a unit that is an integration of multiple components. This kind of testing can only be performed on software that has passed unit tests on isolated components. There's no point in trying to debug a failed integration test when a component's unit test didn't work correctly.

Performance testing can be done at several levels of integration. For a large application, performance testing with the entire build may not be completely helpful. One traditional view is that a program spends 90 percent of its time executing just 10 percent of the available code. Therefore, we don't often need to optimize an entire application; we only need to locate the small fraction of the program that represents the real performance bottleneck.

In some cases, it will be clear that we have a data structure that involves searching. We know that removing searching will lead to a tremendous improvement in performance. As we saw in Chapter 6, Using Callables and Contexts, implementing memoization can lead to dramatic performance improvements by avoiding recalculation.

In order to perform proper performance testing, we need to follow this three-step work cycle:

1. Use a combination of design reviews and code profiling to locate the parts of the application that are likely to be a performance problem. Python has two profiling modules in the standard library. Unless there are more complex requirements, cProfile will locate the part of the application that requires focus.
2. Create an automated test scenario with unittest to demonstrate any actual performance problems. Collect the performance data with timeit or time.perf_counter().
3. Optimize the code for the selected test case until the performance is acceptable.

The point is to automate as much as possible and avoid vaguely tweaking things in the hope of an improvement in performance. Most of the time, a central data structure or algorithm (or both) must be replaced, leading to extensive refactoring. Having automated unit tests makes wholesale refactoring practical.

An awkward situation can arise when a performance test lacks specific pass-fail criteria. There can be a drive to make software faster without a concrete definition of fast enough. It's always simpler when there are measurable performance objectives. Given a concrete objective, then formal, automated testing can be used to assert both that the results are correct and that the time taken to get those results is acceptable.

For performance testing, we might use something like the following code:

import unittest
import timeit


class Test_Performance(unittest.TestCase):

    def test_simpleCalc_shouldbe_fastEnough(self):
        t = timeit.timeit(
            stmt="""RateTimeDistance(rate=1, time=2)""",
            setup="""from Chapter_4.ch04_ex3 import RateTimeDistance""",
        )
        print("Run time", t)
        self.assertLess(t, 10, f"run time {t} >= 10")

This use of unittest can create an automated performance test. As the timeit module executes the given statement 1,000,000 times, this should minimize the variability in the measurements from the background work on the computer that does the testing.

In the preceding example, each execution of the RTD constructor is required to take less than 1/100,000 of a second. A million executions should take less than 10 seconds.