We will first explore the basics of creating Python source tarballs and then the creation of Debian packages from those tarballs.
5.1 Creating a Python Source Tarball
To create a Python source tarball, you have to write a setup.py script that uses distutils or setuptools. Then python setup.py sdist creates the tarball in the right format.
distutils is part of the Python standard library but lacks some commonly used features. setuptools adds these features by extending distutils. Which of these tools you use is mostly a matter of taste and context.
This imports the setup function from setuptools and calls it with metadata about the package—name, version, the list of Python packages to include, and a list of dependencies.
Author: Used for the maintainer’s names. author_email is for a contact’s e-mail address.
url: This should be a link to the project’s web site.
package_data: Used for adding non-Python files to the tarball.
description: This is for a one-paragraph description of the package’s purpose.
python_requires: Used to specify what Python versions your package supports.
scripts: This can hold a list of Python files that are installed as runnable scripts, not just Python packages.
When the setup.py file is in place, you can run python setup.py sdist, and it creates a tarball in the dist directory. The file is named like the name in setup.py, followed by a dash, the version number, and then the suffix .tar.gz. In our example, it’s dist/webcount-0.1.tar.gz.
5.2 Debian Packaging with dh-virtualenv
The official Debian repositories come with more than 40,000 software packages and include software written in all common programming languages. To support this scale and diversity, tooling has been developed to make it easy to get started with packaging but also supports many hooks for customization.
This tooling, which mostly lives in the devscripts package, reads files from the debian directory for metadata and build instructions.
While a complete description of the debhelper tooling would be a big enough topic for a separate book, I want to provide enough information here to get you started.
Getting Started with Packaging
The dh-make package provides a tool for creating a skeleton debian directory, with some metadata already filled in and sample files to base your own versions on. The rest of the tooling then utilizes the files inside the debian packages, to build a binary archive from your source code.
If you are following this example in your own development environment, ensure that the dh-make package is installed before continuing.
5.3 The control File
File debian/control: Metadata for the Debian Package
Source: python-webcount
This declares the build dependency dh-virtualenv, which you need to install, in order to build the Debian package.
Directing the Build Process
The Debian maintainers use the command dpkg-buildpackage or debuild to build the Debian package. Among other things, these tools invoke the debian/rules script with the current action as the argument. The action can be such things as configure, build, test, or install.
Being a makefile, the indentation here must be actual tabulator characters, not a series of spaces.
Declaring Python Dependencies
File requirements.txt
to specify a URL to your own pypi mirror, which dh-virtualenv then uses to fetch the packages.
Building the Package
The -b option instructs dpkg-buildpackage to only build the binary package (which is the deployable unit we want), and -us and -uc skip the signing process that Debian developers use to upload their packages to the Debian mirrors.
The command must be invoked in the root directory of the project (so, the directory that contains the debian directory), and when successful, it puts the generated .deb file into the parent directory of the root directory.
Creating the python-matheval Package
Packaging matheval as the Debian package python-matheval works similarly to webcount. The main difference is that matheval is a service that should be running all the time.
debian/control File for the python-matheval Package
Together, the familiar dpkg-buildpackage invocation creates a Debian package that, on installation, automatically starts the web service and restarts it when a new version of the package is installed.
Tradeoffs of dh-virtualenv
The dh-virtualenv tool makes it pretty easy to create Debian packages with all Python dependencies packaged into them. This is very convenient for the developer, because it means he/she can start using Python packages without having to create separate Debian packages from them.
It also means that you can depend on several different versions of Python packages in multiple applications installed on the same machine—something you cannot easily do if you use the system-wide Python packages.
On the other hand, this “fat packaging” means that if one of the Python packages contains a security flaw, or an otherwise critical bug, you must rebuild and deploy all Debian packages that contain a copy of the flawed code.
Finally, dh-virtualenv packages are tied to the Python version that was used on the build server. So, if a package is built for Python 3.5, for example, it won’t work with Python 3.6. If you are transitioning from one Python version to the next, you have to build packages for both in parallel.
5.4 Summary
We build packages in two steps: first, a Python source tarball based on Python setuptools, then a binary Debian package through dh-virtualenv. Both steps use a few files, mostly based on declarative syntax. The end result is a mostly self-contained Debian package that just needs a matching Python version installed on the target machine.