So far, all the tests we have written used ;unittest.TestCase classes and unittest.main() to run them. As your project grows, you will have more and more tests modules around.

To automatically discover and run all the tests in a project, the unittest package has introduced a Test Discovery feature in Python 3.2, which finds and runs tests given a few options. This feature has been around for a while in projects like Nose (https://nose.readthedocs.io) and pytest, and that's what inspired the unittest package in the standard library.

Which runner to use is a matter of taste, and as long as you stick to writing your tests in TestCase classes, your tests will be compatible with all of them.

That said, the pytest project is very popular in the Python community, and since it offers extensions, people have started to write useful tools around it. Its runner is also quite efficient, as it starts to run the tests while they are still discovered in the background, making it a little faster than the others. Its output in the console is also beautiful and bright.

To use it in your project, you can simply install the pytest package with pip, and use the provided pytest command line. In the following example, the pytest command runs all the modules that start with test_:

$ pytest test_* 
============= test session starts ================================ 
platform darwin -- Python 3.5.2, pytest-3.0.5, py-1.4.32, pluggy-0.4.0 
rootdir: /Users/tarek/Dev/github.com/microbook/code, inifile: 
collected 7 items 
 
test_app.py . 
test_app_webtest.py . 
test_bugzilla.py ... 
test_error.py .. 
 
============= 7 passed in 0.55 seconds ============================= 

The pytest package comes with a lot of extensions, which are listed at http://plugincompat.herokuapp.com/.

Two useful extensions are pytest-cov and pytest-flake8. The first one uses the coverage tool (https://coverage.readthedocs.io) to display the test coverage of your project, and the second one runs the Flake8 (https://gitlab.com/pycqa/flake8) linter to make sure that your code is following the PEP8 style, and has no unused imports.

Here's an invocation example with some style issues left on purpose:

$ pytest --cov=flask_basic --flake8 test_* 
============= test session starts ================================ 
platform darwin -- Python 3.5.2, pytest-3.0.5, py-1.4.32, pluggy-0.4.0 
rootdir: /Users/tarek/Dev/github.com/microbook/code, inifile: 
plugins: flake8-0.8.1, cov-2.4.0 
collected 11 items 
 
test_app.py F. 
test_app_webtest.py F. 
test_bugzilla.py F... 
 
---------- coverage: platform darwin, python 3.5.2-final-0 ----------- 
Name             Stmts   Miss  Cover 
------------------------------------ 
flask_basic.py       6      1    83% 
============= FAILURES ===================================== 
______________ FLAKE8-check ___________________________________ 
test_app.py:18:1: E305 expected 2 blank lines after class or function definition, found 1 
test_app.py:21:1: W391 blank line at end of file 
 
______________ FLAKE8-check ___________________________________ 
test_app_webtest.py:29:1: W391 blank line at end of file 
 
______________ FLAKE8-check ___________________________________ 
test_bugzilla.py:26:80: E501 line too long (80 > 79 characters) 
test_bugzilla.py:28:80: E501 line too long (82 > 79 characters) 
test_bugzilla.py:40:1: W391 blank line at end of file 
 
============= 3 failed, 7 passed, 0 skipped in 2.19 seconds ============= 

Another useful tool that can be used in conjunction with pytest is Tox (http://tox.readthedocs.io).

If your projects need to run on several version of Python, or if you only want to make sure that your code can work on the latest Python 2 and Python 3 versions, Tox can automate the creation of separate environments to run your tests.

Telling Tox to run your project on Python 2.7 and Python 3.5 is done by installing Tox (using the pip installs tox command), and then creating a tox.ini configuration file in the root of your project. Tox makes the assumption that your project is a Python package, and therefore, has a setup.py file in the root directory alongside the tox.ini file, but that's the only requirement.

The tox.ini file contains the command lines to run the tests along with the Python versions it should be run against:

    [tox] 
    envlist = py27,py35 
 
    [testenv] 
    deps = pytest 
       pytest-cov 
       pytest-flake8    
 
    commands =  pytest --cov=flask_basic --flake8 test_* 

When Tox is executed by calling the tox command, it will create a separate environment for each Python version, deploy your package and its dependencies in it, and run the tests in it using the pytest command.

You can run a single environment with tox -e, which is very handy when you want to run the tests quickly. For instance, tox -e py35 will just run pytest under Python 3.5.

Even if you support a single Python version, using Tox will ensure that your project can be installed in a current Python environment, and that you've correctly described all the dependencies.

Using this tool is highly recommended.