Traditionally, very large sites have provided a directory of the information they've made available using a navigational device known as a sitemap. The intent was originally to help users find information they were looking for with the least amount of effort. Turns out, however, there were better solutions to this problem, namely search engines.

Why dig through a long list of irrelevant information when you can simply type in some keywords and get a much more accurate list of potential matches? Sitemaps made a lot of sense before search engine technology was available on a wide scale, but as we discussed in Chapter 6, Searching the Product Catalog, adding search to our applications is now super easy. Even sites that don't offer a search engine can be searched using Google.

So why does Django include a module for automatic sitemap generation? These sitemaps are a little different from the traditional sitemaps, which were usually designed in HTML for human consumption. The sitemaps produced by Django are XML files that are designed to inform machines about the layout and content of your site.

When we say machines, we're really talking about search engines and, more precisely, Google. Google uses sitemaps to build its index of your site. This is very important for achieving a high position in Google's search results.

Django's sitemap module is in django.constrib.sitemaps. To get started, we need to add this to our INSTALLED_APPS setting and create a URL in our root URL configuration. The URL requires us to pass in a dictionary that resembles the feed dictionary we discussed in the previous section. It takes a string and maps it to a Sitemap class for a specific section of our site. For example, we may have a Products sitemap that lists all of our product pages as well as other sitemaps that list manufacturers, special deals, blog posts from our corporate blog, or any other piece of content we've put on the Web.

A sitemap class for our Products would look something like this:

from django.contrib.sitemaps import Sitemap
from coleman.products.models import Product

class ProductSitemap(Sitemap):
    def items(self):
        return Product.objects.all()

We could filter our Product module instead of calling all and limit it by some useful metric. Say, for example, we had a discontinued field but the Product object remained in our database. We may not want Google to index discontinued products so we could return Product.objects.filter(discontinued=False) in the sitemap items method.

The items returned for the sitemap will be assumed to have a get_absolute_url method. This method will be used by the Django sitemap framework to construct the URLs for all of our objects in the sitemap. It is important to make sure your get_absolute_url methods are correct and functional for any object you want indexed in a sitemap.

Once we've generated a sitemap for our site, we want to make sure to tell Google about it. The way to do this is register for a Google Webmaster account at http://google.com/webmaster. In addition to submitting our sitemap, Google's Webmaster Tools let us see all kinds of interesting metrics about our site and how Google evaluates it. This includes what keywords we score highly on and where any incoming links originate. It also lets us track how changes to our site affect our position in the Google index.

If you're building an e-commerce site with lots of content, a very large product catalog, for example, it is highly recommended that you generate a sitemap and submit it to Google. Several books have been written about "Search Engine Optimization" that include lots of search engine magic. Building a sitemap is among the best and most realistic tactics for improving your search result position.