Django comes with a sitemap framework, which allows you to generate sitemaps for your site dynamically. A sitemap is an XML file that tells search engines the pages of your website, their relevance, and how frequently they are updated. By using a sitemap, you will help crawlers indexing your website's content.

The Django sitemap framework depends on django.contrib.sites, which allows you to associate objects to particular websites that are running with your project. This comes handy when you want to run multiple sites using a single Django project. To install the sitemap framework, we need to activate both the sites and the sitemap applications in our project. Edit the settings.py file of your project and add django.contrib.sites and django.contrib.sitemaps to the INSTALLED_APPS setting. Also define a new setting for the site ID, as follows:

SITE_ID = 1

# Application definition
INSTALLED_APPS = (
    # ...
    'django.contrib.sites',
    'django.contrib.sitemaps',
)

Now, run the following command to create the tables of the Django sites application in the database:

python manage.py migrate

You should see an output that contains this line:

Applying sites.0001_initial... OK

The sites application is now synced with the database. Now, create a new file inside your blog application directory and name it sitemaps.py. Open the file and add the following code to it:

from django.contrib.sitemaps import Sitemap
from .models import Post

class PostSitemap(Sitemap):
    changefreq = 'weekly'
    priority = 0.9

    def items(self):
        return Post.published.all()

    def lastmod(self, obj):
        return obj.publish

We create a custom sitemap by inheriting the Sitemap class of the sitemaps module. The changefreq and priority attributes indicate the change frequency of your post pages and their relevance in your website (maximum value is 1). The items() method returns the QuerySet of objects to include in this sitemap. By default, Django calls the get_absolute_url() method on each object to retrieve its URL. Remember that we created this method in Chapter 1, Building a Blog Application, to retrieve the canonical URL for posts. If you want to specify the URL for each object, you can add a location method to your sitemap class. The lastmod method receives each object returned by items() and returns the last time the object was modified. Both changefreq and priority method can also be either methods or attributes. You can see the complete sitemap reference in the official Django documentation located at https://docs.djangoproject.com/en/1.8/ref/contrib/sitemaps/.

Finally, we just need to add our sitemap URL. Edit the main urls.py file of your project and add the sitemap like this:

from django.conf.urls import include, url
from django.contrib import admin
from django.contrib.sitemaps.views import sitemap
from blog.sitemaps import PostSitemap

sitemaps = {
    'posts': PostSitemap,
}

urlpatterns = [
    url(r'^admin/', include(admin.site.urls)),
    url(r'^blog/',
        include('blog.urls'namespace='blog', app_name='blog')),
    url(r'^sitemap.xml$', sitemap, {'sitemaps': sitemaps}, 
        name='django.contrib.sitemaps.views.sitemap'),
]

Here, we include the required imports and define a dictionary of sitemaps. We define a URL pattern that matches with sitemap.xml and uses the sitemap view. The sitemaps dictionary is passed to the sitemap view. Now open http://127.0.0.1:8000/sitemap.xml in your browser. You should see XML code like the following:

<?xml version="1.0" encoding="UTF-8"?>
<urlset xmlns="http://www.sitemaps.org/schemas/sitemap/0.9">
  <url>
    <loc>http://example.com/blog/2015/09/20/another-post/</loc>
    <lastmod>2015-09-29</lastmod>
    <changefreq>weekly</changefreq>
    <priority>0.9</priority>
  </url>
  <url>
    <loc>http://example.com/blog/2015/09/20/who-was-django-reinhardt/</loc>
    <lastmod>2015-09-20</lastmod>
    <changefreq>weekly</changefreq>
    <priority>0.9</priority>
  </url>
</urlset>

The URL for each post has been built calling its get_absolute_url() method. The lastmod attribute corresponds to the post publish date field as we specified in our sitemap, and the changefreq and priority attributes are also taken from our PostSitemap class. You can see that the domain used to build the URLs is example.com. This domain comes from a Site object stored in the database. This default object has been created when we synced the sites framework with our database. Open http://127.0.0.1:8000/admin/sites/site/ in your browser. You should see something like this:

This is the list display admin view for the sites framework. Here, you can set the domain or host to be used by the sites framework and the applications that depend on it. In order to generate URLs that exist in our local environment, change the domain name to 127.0.0.1:8000 as shown in the following screenshot and save it:

We point to our local host for development purposes. In a production environment, you will have to use your own domain name for the sites framework.