Organizing code into src, scripts, tests, and docs

As we noted in the previous section, there's no essential need for a complex directory structure in a Python project. The ideal structure follows the standard library, and is a relatively flat list of modules. This will include overheads such as the  setup.py and README files. This is pleasantly simple and easy to work with.

When the modules and packages get more complex, we'll often need to impose some structure. For complex applications, one common approach is to segregate Python code into a few bundles. To make the examples concrete, let's assume that our application is called my_app:

  • my_app/src: This directory has all of the working application code. All of the various modules and packages are here. In some cases, there's only a single top-level package name in this src directory. In other cases, there may be many modules or packages listed under src. The advantage of a src directory is the simplicity of doing static analysis with mypy, pylint, or pyflakes.
  • my_app/scripts or my_app/bin: This directory can have any scripts that form an OS-level command-line API. These scripts can be copied to the Python scripts directory by setup.py. As noted previously, these should be like the __main__.py module; they should be very short, and they can be thought of as OS filename aliases for Python code.
  • my_app/tests: This directory can have the various test modules. Most of the modules will have names beginning with test_ so they can be discovered by pytest automatically.
  • my_app/docs: This directory will have the documentation. We'll look at this in Chapter 20, Quality and Documentation.

The top-level directory name, my_app, might be augmented with a version number to permit having multiple branches or versions available, leading to my_app-v1.1 as a top-level directory name. A better strategy is to use a sophisticated version control tool, such as git. This can manage having multiple versions of software in a single directory structure. Using git commands to switch branches works out better than trying to have multiple branches in adjacent directories.

The top-level directory will contain the setup.py file to install the application into Python's standard library structure. See Distributing Python Modules (https://docs.python.org/3/distributing/index.html) for more information. Additionally, of course, a README.rst file would be placed in this directory. Other common files found here are the tox.ini file, for configuring the overall testing environment, and the environment.yaml file, for building the Conda environment for using the application.

When the application modules and test modules are in separate directories, we need to refer to the application as an installed module when running tests. We can use the PYTHONPATH environment variable for this. We can run the test suite as in the following code:

PYTHONPATH=my_app/src python3 -m test 

We set an environment variable on the same line where we execute a command. This may be surprising, but it's a first-class feature of the bash shell. This allows us to make a very localized override to the PYTHONPATH environment variable.

The next section shows how to install Python modules.

..................Content has been hidden....................

You can't read the all page of ebook, please click here login for view all page.
Reset