Virtualenv

In the previous section we saw how zc.buildout could be used to generate an isolated project environment, including project files and dependencies. Buildout automated the process using a configuration file and a special filesystem hierarchy. It is a powerful tool, but may not suit every developer's style. Getting an environment up and running is a fair amount of work and making changes to try out new things requires editing the configuration file and rebuilding everything.

Environment isolation is a definite need for any serious Python or Django developer, though. It becomes very difficult to work in a situation where multiple projects are overlapping and there are conflicting package needs. Developing enhancements to our applications while maintaining a working production copy, like upgrading to a newer version of Django, for example, is impossible without some tool to manage our packages.

A lot of developers will take the obvious approach of creating special local package directories in their own workspace and attaching this location to the $PYTHONPATH environment variable or sys.path, swapping around a .pth file in the system site-packages location, or managing a collection of symbolic links.

If you've ever tried these approaches you'll probably recognize the difficulties this creates. First among them is that distributed Python applications that use setuptools or distutils and a setup.py script require a lot of special attention to install. Second, tools like easy_install install to the system site-packages by default. Both of these kill productivity and defeat the purpose of having a package index.

Buildout solves this problem, but it's not the only solution. Virtual environments take a different approach. Instead of writing configuration files and running build scripts, you can configure a virtual environment, and then use all the standard Python practices for installing packages and configuring tools. No changes to $PYTHONPATH or sys.path, no symbolic links, no funny business.

A virtual environment in some ways resembles the isolated environments we saw with buildout. A virtual environment is isolated from all other virtual environments, but not from the system environment. This means if we have installed packages to our system site-packages location, by default they will be available in our virtual environments as well. Otherwise packages are not available until we install them to the virtual environment.

There are currently several different virtual environment tools. Among these, virtualenv has gained a large following and provides an excellent all-around solution to the virtual environment problem. Virtualenv was written by top Python developer Ian Bicking and is available from PyPI at http://pypi.python.org/pypi/virtualenv.

Creating an environment

Once installed, we can use the shell command virtualenv to manipulate environments. Every developer will have different preferences, but it's likely that you'll want to store all of your virtual environments in a single location, perhaps /home/username/virtualenvs.

When we change to our virtualenvs directory, we can issue the virtualenv command, following by a destination directory. This could be the name of our project or some other identifier to distinguish the environment we're creating. Let's create an environment for this book's coleman project:

$ virtualenv coleman

The results of this command are shown in the following screenshot:

Creating an environment

We've just created a complete virtual environment. If we navigate to our environment directory, we'll notice three subdirectories: bin, include, and lib. So far this is similar to what happened when we used buildout to create an environment.

Inside our virtual environment's bin, we see a copy of our Python interpreter. This interpreter is an actual interpreter binary, copied from our system's Python installation. This is unlike buildout, which creates wrappers around our system Python and provides it with an updated set of paths.

Even though the virtualenv interpreter is not editable, it does have knowledge of the virtual environment. If we were to run it as a Python shell, the path variable will reflect our virtual environment installation path. In order for this interpreter to be recognized as the default when we type python into a shell, however, we must first activate the environment.

Inside the bin location there is a shell script called activate. In order to activate the environment we have to run this shell script using the source command. Source is a built-in bash command that runs a script in our current shell (as opposed to starting a sub-shell). Running activate with source updates our shell environment and also adds a useful indicator to our shell prompt to inform us of what virtual environment we're currently using, as shown below:

Creating an environment

The activate command has done two things: updated our shell's $PATH environment variable to include our virtual environment's bin location and updated a $VIRTUAL_ENV shell environment variable that points to our virtual environment directory.

When we ran virtualenv earlier, it actually performed some magic whereby our virtual environment's interpreter can detect our activated environment and adjust its path accordingly. The results of these operations are shown in the following screenshot:

Creating an environment

Once we have activated an environment, we can deactivate it in a similar way by just typing deactivate at the shell prompt. Our virtual environment indicator should disappear from our prompt and we'll be back to using the system Python interpreter. We can activate other virtual environments by navigating to their location in the shell and running the new environment's bin/activate.

Working in the environment

Now that we've created a virtual environment for our project and activated it using the bin/activate script, we can go about our work. Because virtualenv has updated our shell path, every time we run python, our virtual environment's python will run.

Since we can now use the python command at will, something cool happens: we can begin installing packages using setup.py or easy_install. As long as our virtual environment is activated, we can easy_install to our heart's content and it all goes into our virtual environment.

In general, it is good practice when using virtual environments to keep the system site-packages location as light as possible. This prevents confusion and other potential mishaps. Our easy_installs from an activated virtual environment will include the system site-packages when looking for dependencies and such, but it is still recommended that most packages be installed virtually.

This will lead to some duplication if you are working on multiple projects that use the same set of packages. For these cases you may consider installing the package system-wide, especially if it's particularly important or difficult to install. The MySQLdb package and other database interfaces are good examples of candidates for system-wide installation.

Django would be an example of something not to install system wide, unless you are absolutely certain you won't be developing two separate projects that are running against different versions of the framework. Even if you are, still consider keeping all packages installed virtually. The disk space is there, why not use it?

Another important point to note is that the package manager for your operating system may, either by default or through a dependency negotiation, have installed packages to your Python's system site-packages without you necessarily being aware. It's a good idea to check out what packages and modules are installed in your system before switching to virtualenv. In fact, you may wish to empty your system site-packages altogether. You can uninstall any module by deleting its .egg, directory, and other metadata files.

Virtualenvwrapper

Doug Hellman has released a set of extension scripts for virtualenv called virtualenvwrapper. These are a set of shell functions that enhance the virtual environment configuration, primarily by organizing and structuring our environment locations. It includes pre and post hooks for several operations, including creating an environment, activating a new environment, and deactivating an environment.

The activation hooks allow for lots of interesting possibilities. For example, you could attach a post-activate hook that automatically runs a shell script or AppleScript to launch and configure your text editor whenever you change virtual environments.

The virtualenvwrapper scripts are a highly recommended enhancement to virtualenv. They are available from: http://www.doughellmann.com/projects/virtualenvwrapper/.

..................Content has been hidden....................

You can't read the all page of ebook, please click here login for view all page.
Reset