Automating deployment with Fabric

A big win for development team productivity can be automating as much of the routine tasks as possible. The scope of tasks that could be automated is practically unlimited; if it happens more than once, you might want to think about it. The reason for this is simple: deployment tasks are often repetitive and prone to error. Doing it correctly once, capturing the process and automating it prevents errors, reduces efforts, and minimizes time spent away from focusing on your application.

Fabric is a very recent tool written in Python that attempts to provide a simple framework for automating deployment activities. It allows us to automate common elements of a deployment process, including connecting to and issuing commands on servers via SSH, uploading and downloading files, and gathering input from the console when needed. Documentation and more information are available at http://docs.fabfile.org/.

Writing a Fabfile

Creating a set of deployment functions for our application is just a matter of writing some Python. Often this code will live in a file called fabfile.py somewhere in our project tree. The contents of this file can be whatever we want; it's just normal Python, but it will typically import the fabric module to allow us to perform our deployment work.

What's great about Fabric is that it lets us define all of our operations relative to an environment. The environment is a global dictionary containing the import information regarding our deployment situation. This includes things such as host names, user names, passwords, and so on. Thus we can write a set of deployment functions for our application and reuse them in a simple way, just by changing our environment values.

For example, to execute our Fabric script across multiple servers, we can define the environment's hosts list to include the server hostnames or IP address at the top of our fabfile.py:

env.hosts = ['ebook.server.com', '172.16.1.10']	

With our environment defined, Fabric will automatically run our deployment functions across both of these servers, including logging in to SSH. In real-world scenarios, additional information, such as user names, can be included in the environment. By default, the current user is used as the value for username and passwords are obtained interactively.

The fabric module is loaded with useful functions. For example, the fabric.operations module includes:

  • sudo: Used to run a remote command as the super user
  • put: Used to upload a file to the remote server
  • run: Used to run a remove command as the logged-in user
  • get: Used to download a file from the remote server
  • local: Used to run a command locally

These are all just Python functions that behave as you would probably expect. Some examples include:

put('media/js/site_wide.js', '/var/www/media/js/site_wide.js') 
put('media/img/*.jpg', '/var/www/media/img/') 
sudo('svn update /opt/svn/repos/myrepos') 
run('touch django.wsgi') 
local('django-admin.py test all')

With these basic set of operations, you can begin to compile a set of functions before the deployment tasks. This might include a test function to run Django unit tests, a minify function if you're minifying JavaScript, a function to compress and upload test data or import SQL commands to your database, and almost anything else.

Using the fab tool

Fabric provides a command line tool called fab that allows us to quickly run our Fabric operations from a shell prompt. The fab tool takes any number of arguments, each of which corresponds to a function we've written in our fabfile.

The fabfile library is auto discovered when using the fab tool. It will search the current directory for a fabfile.py or, in newer versions of Fabric, search for a fabfile/ module. The fabfile/ subdirectory in our project must contain __init__.py for this auto-discovery to succeed.

The fab command will execute our fabfile operations in the order we provide on the command line. An example usage follows:

$ fab run_tests prepare_deployment deploy

This will first execute our fabfile's run_tests function, then prepare_deployment, then deploy. For simplicity we could have wrapped all three functions inside the deploy operation and the result would be equivalent. But what happens when say, run_tests fails? Fabric will automatically detect Python exceptions and cease execution of the current and future operations.

In addition to Python exceptions originating in our functions, Fabric will also halt when a remote or local shell command does not return cleanly, after a failed file transfer or broken sudo command, for example. This behavior is configurable, of course, by changing the environment setting warn_only to True.

Fabric for production deployments

Fabric is a very Pythonic deployment tool. Other excellent tools exist that play a similar role, such as Ruby's Capistrano (which can be used for deploying anything, not just Ruby). But many developers still use the manual, SSH-and-type-some-commands method. This works, but automating these repetitive tasks can turn a half-dozen shell commands and a lot of thought into a single command that we can run while grabbing coffee (or automate for a continuous integration strategy).

Consider the typical production deployment. For Django applications there can be several complicating factors. One of the biggest is media. Media is typically served in a different fashion than the application itself. This might mean media files are served via a content-delivery network or a separate media server. Either way, changes to Django media files sometimes necessitate special steps in the deployment process.

Cascading Style Sheets (CSS) are good examples. CSS files frequently contain file paths to background images and other content, but do not have access to Django's MEDIA_URL template variable, because they are not rendered by Django. Relative file paths can solve this problem much of the time, as long as media is served up identically on production, staging, and development servers.

But when media can't be served identically, perhaps because production servers use a content-delivery network that we have a limited ability to customize, we face a deployment headache. A tool like Fabric can solve this problem in concert with an appropriate search-and-replace type script. It is as simple as writing custom deploy functions that extract our application code, deploy to the corresponding site, and run a site-specific media processing script to make any necessary adjustments.

Another example is using Fabric to manage deployments from source code repositories. The server-centric functions we've written to include special handlers for media files could also intelligently manage the deployment interaction with a source code repository. This means we could write Fabric functions to roll out new code releases and roll back to previous releases automatically. A simplification like this is ideal because it provides a simple way for anyone (even non-technical staff) to manage production servers.

One-off and complicated situations like this are often unavoidable. If developers are burdened by a complicated deployment process, they are almost certainly going to perform less testing and encounter more problems during roll outs of new code. This is where a tool like Fabric can become especially powerful.

..................Content has been hidden....................

You can't read the all page of ebook, please click here login for view all page.
Reset