Once a Debian package is built, it must be distributed to the servers it is to be installed on. Debian, as well as basically all other operating systems, uses a pull model for that. The package and its metadata are stored on a server that the client can communicate with and request the metadata and the package.
The sum of metadata and packages is called a repository. In order to distribute packages to the servers that need them, we must set up and maintain such a repository.
6.1 Signatures
In Debian land, packages are cryptographically signed, to ensure they aren’t tampered with on the repository server or during transmission. Thus, the first step is to create a key pair that is used to sign this particular repository. (If you already have a PGP key for signing packages, you can skip this step.)
The following assumes that you are working with a pristine system user that does not have a GnuPG key ring yet and will be used to maintain the Debian repository. It also assumes that you have the gnupg package installed in version 2 or later versions.
First, create a file called
key-control-file-gpg2, with the following contents:
%no-protection
Key-Type: RSA
Key-Length: 1024
Subkey-Type: RSA
Name-Real: Aptly Signing Key
Expire-Date: 0
%commit
%echo done
Substitute
[email protected] with your own e-mail address or an e-mail address for the project you are working for, then run the following command:
$ gpg --gen-key --batch key-control-file-gpg2
The output of this command contains a line like the following:
gpg: key D163C61A6C25A6B7 marked as ultimately trusted
The string of hex digits
D163C... is the key ID, and this
differs for every run. Use it to export the public key, which we’ll need later on.
$ gpg --export --armor D163C61A6C25A6B7 > pubkey.asc
6.2 Preparing the Repository
I use Aptly for creating and managing the repository. It is a command-line application with no server component.
To initialize a repository, I first have to come up with a name. Here, I call it
myrepo.
$ aptly repo create -distribution=stretch
-architectures=amd64,i386,all -component=main myrepo
Local repo [myrepo] successfully added.
You can run 'aptly repo add myrepo ...' to add packages
to repository.
$ aptly publish repo -architectures=amd64,i386,all myrepo
Warning: publishing from empty source, architectures list
should be complete, it can't be changed after publishing
(use -architectures flag)
Loading packages...
Generating metadata files and linking package files...
Finalizing metadata files...
Signing file 'Release' with gpg, please enter your
passphrase when prompted:
Clearsigning file 'Release' with gpg, please enter your
passphrase when prompted:
Local repo myrepo has been successfully published.
Please set up your webserver to serve directory
'/home/aptly/.aptly/public' with autoindexing.
Now you can add following line to apt sources:
deb http://your-server/ stretch main
Don't forget to add your GPG key to apt with apt-key.
You can also use `aptly serve` to publish your repositories over HTTP quickly.
Now that the repository has
been created, you can add a package by running
$ aptly repo add myrepo python_webcount_0.1-1_all.deb
$ aptly publish update myrepo
This updates the files in .aptly/public to be a valid Debian repository that includes the newly added package.
6.3 Automating Repository Creation and Package Addition
For use inside the deployment pipeline, it is handy to have the repositories, and the addition of packages to these repositories, created with a single command. It also makes sense to have separate repositories for the different environments. Hence, we need a repository each for the testing, staging, and production environments. A second dimension is the distribution for which a package is built.
Here is a small program (Listing
6-1) that, given an environment, a distribution, and a list of file names of Debian packages, creates the repository in the path
$HOME/aptly/$environment/$distribution, adds the packages, and then updates the public files of the repositories:
#!/usr/bin/env python3
import json
import os
import os.path
import subprocess
import sys
assert len(sys.argv) >= 4,
'Usage: add-package <env> <distribution> <.deb-file>+'
env, distribution = sys.argv[1:3]
packages = sys.argv[3:]
base_path = os.path.expanduser('~') + '/aptly'
repo_path = '/'.join((base_path, env, distribution))
config_file = '{}/{}-{}.conf'.format(base_path, env,
distribution)
def run_aptly(*args):
aptly_cmd = ['aptly', '-config=' + config_file]
subprocess.call(aptly_cmd + list(args))
def init_config():
os.makedirs(base_path, exist_ok=True)
contents = {
'rootDir': repo_path,
'architectures': ['amd64', 'all'],
}
with open(config_file, 'w') as conf:
json.dump(contents, conf)
def init_repo():
if os.path.exists(repo_path + '/db'):
return
os.makedirs(repo_path, exist_ok=True)
run_aptly('repo', 'create',
'-distribution=' + distribution, 'myrepo')
run_aptly('publish', 'repo', 'myrepo')
def add_packages():
for pkg in packages:
run_aptly('repo', 'add', 'myrepo', pkg)
run_aptly('publish', 'update', distribution)
if __name__ == '__main__':
init_config();
init_repo();
add_packages();
Listing 6-1add-package, a Tool for Creating and Populating Debian Repositories
It can be used as
$ ./add-package testing stretch python-matheval_0.1-1_all.deb
to add the python-matheval_0.1-1_all.deb file to the Stretch repository for environment testing, and it automatically creates that repository, if it does not yet exist.
6.4 Serving the Repositories
As is, the repositories are only available on one machine. The easiest way to make them available to more machines is to serve the public directory as static files through HTTP.
If you use Apache as the web server, the virtual host configuration to serve these files could look like Listing
6-2.
ServerName apt.example.com
DocumentRoot /home/aptly/aptly/
Alias /debian/testing/stretch/
/home/aptly/aptly/testing/stretch/public/
Alias /debian/production/stretch/
/home/aptly/aptly/production/stretch/public/
# more repositories go here
Options +Indexes +FollowSymLinks
Require all granted
LogLevel notice
CustomLog /var/log/apache2/apt/access.log combined
ErrorLog /var/log/apache2/apt/error.log
ServerSignature On
Listing 6-2Apache 2 Configuration for Serving Debian Repositories
After creating the logging directory (mkdir -p /var/log/apache2/apt/), enabling the virtual host (a2ensite apt.conf), and restarting Apache, the Debian repository is ready.
If, instead, you prefer
lighttpd,
you could use a configuration snippet such as that in Listing
6-3.
dir-listing.encoding = "utf-8"
server.dir-listing = "enable"
alias.url = (
"/debian/testing/stretch/" =>
"/home/aptly/aptly/testing/stretch/public/",
"/debian/production/stretch/" =>
"/home/aptly/aptly/production/stretch/public/",
# more repositories go here
)
Listing 6-3lighttpd Configuration for Serving Debian Repositories
Configuring a Machine to Use the Repository
When a machine uses one of the new repositories, it first has to trust the cryptographic key with which the repositories are signed.
Copy the PGP public key (
pubkey.asc) to the machine that will use the repository and import it.
Then add the actual package source.
$ echo "deb http://apt.example.com/ stretch main"
> /etc/apt/source.list.d/myrepo.list
After an
apt-get update, the contents of the repository are available, and an
apt-cache policy python-matheval shows the
repository as a possible source for this package.
$ apt-cache policy python-webcount
python-webcount:
Installed: (none)
Candidate: 0.1-1
Version table:
*** 0.1-1 0
990 http://apt.example.com/ stretch/main amd64 Packages
100 /var/lib/dpkg/status
This concludes the whirlwind tour through Debian repository management and, thus, package distribution.
6.5 Summary
Debian package installers such as apt-get and aptitude from the APT software suite read metadata and download packages from repositories. Software such as Aptly manages these repositories.
Cryptographic signatures authenticate packages and catch man-in-the-middle attacks and transport errors that modify software packages. You must to create a GPG key and supply it to Aptly and configure the target machines to trust this key.
