HttpRequest.method

The HTTP specification outlines a variety of verbs that can be used to describe the type of request being performed. This is typically referred to as its method, with different request methods having specific expectations of how they should be handled. In Django, the method being used for the request is represented as the method attribute of the HttpRequest object. It will be included as a standard string, with the method name in all uppercase letters.

Each method describes what the server should do with the resource identified by the URL. Most Web applications will only implement GET and POST, but a few others are worth explaining here as well. Further details on these—and others not listed here—can be found in the HTTP specification,² as well as many other resources on the Web.

• DELETE—Requests that the resource be deleted. Web browsers don’t implement this method, so its use is limited to Web service applications. In typical Web browser applications, such operations are done with a POST request, since GET requests aren’t allowed to have side effects, such as removal of the resource.
• GET—Retrieves the resource specified by the URL. This is, by far, the most common type of request made on the Web, as every standard retrieval of a Web page is done with a GET request. As noted in the “Safe Methods” section, GET requests are assumed to have no side effects on the server; they should retrieve the specified resource and do nothing else.
• HEAD—Retrieves some information about the resource without getting the entire contents. Specifically, the response to a HEAD request should return exactly the same headers as a GET request, only without anything in the body of the response. Web browsers don’t implement this method, but since the server-side operation is essentially just a GET request without a response body, it is rarely missed. In Web service applications, a HEAD request can be a low-bandwidth way to retrieve information about a resource, such as whether it exists, when it was last updated or the size of its content.
• POST—Requests that the attached data be stored in some way related to the resource specified by the URL. This could mean comments on a blog post or news article, answers to a question, replies to a Web-based email or any number of other related situations.

  This definition is only valid in Web service environments, where a differentiation can be made between PUT and POST. In standard Web browsers, only GET and POST are reliably available, so POST is used for any situation that modifies information on the server. Using POST to submit data from a form is little more than a footnote in the official HTTP specification, but is the most popular use of the method.

• PUT—Requests that the attached data be stored at the resource specified by the URL. This could be seen as a “create” or “replace” operation, depending on whether the resource already exists. This method isn’t traditionally available in Web browsers, though, so its use is limited to Web service applications. In a standard Web browser, the operation specified by PUT is done with a POST request instead.

HttpRequest.path

This attribute contains the complete path that was requested, without any query-string parameters attached. This can be used to identify the resource being requested, without relying on which view will be called or how it will behave.

Accessing Submitted Data

Any time a request comes in, it can potentially be accompanied by a variety of data provided by the Web browser. Processing this information is key to making a Web site dynamic and interactive, so Django makes it easy and flexible. Just as there are many ways to submit data to a Web server, there are as many ways to access that data once it arrives.

Data that comes in using the standard query-string format³ sent by most browsers is automatically parsed into a special type of dictionary class called QueryDict. This is an immutable subclass of MultiValueDict, which means that it functions mostly like a dictionary, but with a few added options for handling multiple values for each key in the dictionary.

The most significant detail of QueryDict is that it’s instantiated with a query-string from an incoming request. For more information on the details of how to access values in a QueryDict, see the details for MultiValueDict in Chapter 9.

HttpRequest.META

When a request comes in, there is a significant amount of information related to the request that doesn’t come through in a query-string and isn’t available in the GET or POST attributes on the request. Instead, data regarding where the request came from and how it got to the server is stored in the request’s META attribute. Details of which values are available in META can be found in PEP-333.

In addition, each request is accompanied by a number of headers, which describe various options the client would like to make known. Exactly what these types of headers can contain is specified in the HTTP specification,⁴ but they typically control things like a preferred language, allowable content-types and information about the Web browser.

These headers are also stored in META, but in a form slightly altered from how they came in originally. All HTTP header names become uppercase, are prefixed with HTTP_ and have all of their dashes replaced with underscores.

• Host becomes HTTP_HOST.
• Referer becomes HTTP_REFERER.
• X-Forwarded-For becomes HTTP_X_FORWARDED_FOR.

HttpRequest.COOKIES

Since each HTTP request is a fresh connection between the client and the server, cookies are used as a way to identify clients that make multiple requests. In a nutshell, cookies are little more than a way to send a name and associated value to a Web browser, which that browser will then send back each time it makes a new request to the Web site.

While cookies are set during the response phase of the process, as documented under HttpResponse, the task of reading cookies from an incoming request is quite simple. The COOKIES attribute of the request is a standard Python dictionary mapping names of cookies to the values that were previously sent.

Keep in mind that this dictionary will contain entries for all cookies sent by the browser, even if they were set by another application on the same server. The HttpResponse section later in this chapter covers the specific rules of how a browser decides which cookies to send with a particular request and how to control that behavior.

HttpRequest.get_signed_cookie(key[, …])

If you store information in a cookie that could be used against you if it were tampered with, you can opt to sign your cookies and validate those signatures when reading the cookies back in. The signatures themselves are provided using the HttpResponse.set_signed_cookie() method described later in this chapter, but when reading them in a request, you’ll need to use this method.

You can control the behavior of your cookie retrieval using a few additional arguments:

• default=RAISE_ERROR—This argument allows you to specify a default value that should be returned if the requested key was not found or was invalid. This is equivalent to passing a default value into a standard dictionary’s get() method. If you don’t supply a value, this method will raise a standard KeyError if the key is missing or django.core.signing.BadSignature if the signature is invalid.
• salt=''—This is a complement to the same argument in the set_signed_cookie() method. It allows you to use the same key in different aspects of your application, perhaps on multiple domains, without risk of the signature being reused from one use to another. This must match the value you provide when setting the cookie in order for the signature check to match.
• max_age=None—By default, cookie signatures also have an expiration associated with them to avoid them being reused longer than intended. If you supply a max_age that exceeds the age of a given cookie, you’ll get a django.core.signing.SignatureExpired exception. By default, this won’t check the expiration date when validating the signature.

HttpRequest.get_host( )

Many server configurations allow a single Web application to respond to requests sent to multiple different domain names. To help with these situations, the get_host() method of the incoming request allows a view to identify the name that the Web browser used to reach the Web site.

In addition to the host name used to make the request, the value returned from this method will include a port number if the server was configured to respond on a nonstandard port.

HttpRequest.get_full_path( )

In addition to the host information, the get_full_path() method returns the entire path portion of the URL; everything after the protocol and domain information. This includes the full path that was used to determine which view to use, as well as any query-string that was provided.

HttpRequest.build_absolute_uri(location=None)

This method generates an absolute URL for the provided location, if any. If no location is supplied explicitly, the request’s current URL is returned, including the query-string. The exact behavior of the method if the location is provided depends on what value is passed in.

• If the value contains a fully-qualified URL—including the protocol—that URL is already absolute and is returned as provided.
• If the value begins with a forward slash (/), it is appended to the protocol and domain information of the current URL, then returned. This will generate an absolute URL for the provided path, without having to hard-code the server information.
• Otherwise, the value is assumed to be a path relative to the request’s current URL, and the two will be joined together using Python’s urlparse.urljoin() utility function.

HttpRequest.is_secure( )

This simple method returns True if the request came in using the Secure Sockets Layer (SSL) protocol or False if the request was unsecured.

HttpRequest.is_ajax( )

Useful for “Web 2.0” sites, this method returns True if the request has an X-Requested-With header with a value of “XMLHttpRequest”. Most JavaScript libraries designed to make calls to the server will provide this header, providing a convenient way to identify them.

HttpRequest.encoding

This is a simple attribute representing the encoding to be used when accessing the GET and POST attributes described earlier. Values in those dictionaries are forced to unicode objects using this encoding, if one is set. By default, its value is None, which will use the default encoding of utf-8 when accessing values.

In most cases, this attribute can be left as is, with most input being converted properly using the default encoding. Specific applications may have different needs, so if the application expects input with a different encoding, simply set this attribute to a value that will decode those values properly.

Creating a Response

Unlike the request, the author of a view has full control over how its response is created, allowing a variety of options. The standard HttpResponse class is instantiated rather simply, but accepts three arguments to customize its behavior. None of these are required; options described later in this section can set these values in other ways.

• content—This accepts text—or other content—to be used as the body of the request.
• status—This sets the HTTP status code⁵ to be sent with the request.
• content_type—This controls the Content-Type header to be sent with the request. If this is supplied, make sure it also contains the charset value when appropriate.

  >>> from django.http import HttpResponse
  >>> print HttpResponse()
  Content-Type: text/html; charset=utf-8
   
   
  >>> print HttpResponse(content_type='application/xml; charset=utf-8')
  Content-Type: application/xml; charset=utf-8
   
   
  >>> print HttpResponse('content')
  Content-Type: text/html; charset=utf-8
   
  content

There is also a mimetype argument, provided for backwards-compatibility with older Django applications, but content_type should be used instead. It’s still important to keep mimetype in mind, though, as it means that status and content_type should be specified as keyword arguments, if supplied at all.

Dictionary Access to Headers

Once a response has been created, it’s simple to customize the headers that will be sent out along with its content, using standard dictionary syntax. This is quite straightforward and works just as you’d expect. The only notable variation from a standard dictionary is that all key comparisons are case-insensitive.

>>> from django.http import HttpResponse
>>> response = HttpResponse('test content')
>>> response['Content-Type']
'text/html; charset=utf-8'
>>> response['Content-Length']
Traceback (most recent call last):
  ...
KeyError: 'content-length'
>>> response['Content-Length'] = 12
>>> for name, value in response.items():
...     print '%s is set to %r' % (name, value)
...
Content-Length is set to '12'
Content-Type is set to 'text/html; charset=utf-8'

File-Like Access to Content

In addition to the ability to specify body content as a string when creating the response object, content can be created by many third-party libraries that know how to write to open files. Django’s HttpResponse implements a few file protocol methods—most notably write()—that enable it to be treated as a write-only file for many of these libraries. This technique can be especially useful when using Django to generate binary content, such as PDF files, dynamically within views.

One important thing to note regarding file-like access to the response body is that not all file protocol methods are implemented. This means that certain libraries, such as Python’s own zipfile.ZipFile class, which require those extra methods, will fail with an AttributeError, indicating which method was missing. This is by design, as HTTP responses aren’t true files, so there is no predictable way to implement those methods.

HttpResponse.status_code

This attribute contains the numerical status code representing the type of response being sent to the client. As described earlier, this can be set immediately when instantiating the response object, but as a standard object attribute, it can also be set any time after the response has been created.

This should only be set to known HTTP response status codes. See the HTTP specification for details on valid status codes. This status can be set while instantiating the response, but it can also be set as a class attribute on a subclass, which is how Django configures many of its specialized responses.

HttpResponse.set_cookie(key, value=''[, …])

When looking to store values across multiple requests, cookies are the tool of choice, passing values to the Web browser through special headers, which are then sent back to the server on subsequent requests. By calling set_cookie() with a key and a value, the HTTP response sent to the client will contain a separate header, telling the browser what to store and when to send it back to the server.

In addition to just the key and value, set_cookie() can take a few extra arguments that configure when the browser should send the cookie back to the server. While a quest for readability suggests that these arguments be specified using keywords, this list uses their positional order. More details on what values are allowed for each of these options can be found in the official specification for HTTP state management.⁶

• max_age=None—Corresponding to the max-age option from the specification, this specifies the number of seconds the cookie should remain active.
• expires=None—Not all browsers accept and respect max-age as required by the official specification but instead follow an early pattern set out by Netscape. The expires attribute takes an exact date when the cookie should expire, rather than an offset in seconds. The specified date is in the following format: Sun, 15-Jun-2008 12:34:56 GMT.
• path='/'—This specifies a base path under which the browser should send this cookie back to the server. That is, if the path of the URL being requested begins with the value specified here, the browser will send the cookie’s value along with the request.
• domain=None—Similar to path, this specifies the domain under which the cookie will be sent. If left as None, the cookie will be restricted to the same domain that issued it, while providing a value will allow greater flexibility.
• secure=False—If set to True, this will indicate that the cookie contains sensitive information and should only be sent to the server through a secure connection, such as SSL.

  >>> response = HttpResponse()
  >>> response.set_cookie('a', '1')
  >>> response.set_cookie('b', '2', max_age=3600)
  >>> response.set_cookie('c', '3', path='/test/', secure=True)
  >>> print response.cookies
  Set-Cookie: a=1; Path=/
  Set-Cookie: b=2; Max-Age=3600; Path=/
  Set-Cookie: c=3; Path=/test/; secure

Keep in mind that this will set the cookie in the browser only after the response has made its way across the wire. That means that the cookie’s value won’t be available on the request object until the browser’s next request.

HttpResponse.delete_cookie(key, path='/', domain=None)

If a cookie has already been delivered to the Web browser and is no longer needed or has become invalid, the delete_cookie() method can be used to instruct the browser to remove it. As mentioned, the path and domain provided here must match an existing cookie in order to have it deleted properly.

It does this by setting a new cookie with max-age set to 0 and expires set to Thu, 01-Jan-1970 00:00:00 GMT. This causes the browser to overwrite any existing cookie matching the same key, path and domain, then expire it immediately.

HttpResponse.cookies

In addition to being able to explicitly set and delete cookies during the response phase, you can view the cookies that will be sent to the Web browser. The cookies attribute uses Python’s standard Cookie module,⁷ with the attribute itself being a SimpleCookie object, which behaves much like a dictionary, with each value being a Morsel object.

Using a cookie’s name as the key, you can retrieve a Morsel representing a specific cookie value, along with its associated options. This object may be used as a dictionary to reference these additional options, while its value attribute contains the value that was set for the cookie. Even deleted cookies are accessible using this dictionary, since the process involves setting a new cookie that will simply expire immediately.

>>> len(response.cookies)
3
>>> for name, cookie in response.cookies.items():
...     print '%s: %s (path: %s)' % (name, cookie.value, cookie['path'])
...
a: 1 (path: /)
b: 2 (path: /test/)
c: 3 (path: /)

HttpResponse.set_signed_cookie(key, value, salt=''[, …])

This works just like set_cookie(), except that it also cryptographically signs the value before sending it out to the browser. Because cookies are stored in the browser, this ensures that the user doesn’t modify the values in those cookies before visiting your site again. You still don’t want to store sensitive information in your cookies, but this allows you to confidently store things like a logged-in username in a cookie, without the user being able to use it as an attack vector.

This takes all the same arguments as set_cookie(), with one addition: salt. By default, Django uses your settings.SECRET_KEY to generate a signature, which is fine in most cases, where a cookie with a particular key is only likely to be used for one purpose. In other cases, the salt argument allows you to craft a signature to whatever use you currently have.

For example, if you’re serving up multiple domains with a single Django installation, you could use the domain name as the salt for your signatures, so that a user can’t reuse the signature from one domain on a different domain. The different salts ensure the signatures will be different, so that a copied signature would fail the signature test when retrieving the cookie in your view.

HttpResponse.content

This attribute provides access to the string content of the response body. This can be read or written, and is particularly useful during the response phase of middleware processing.

Specialty Response Objects

Since there are several common HTTP status codes, Django provides a set of customized HttpResponse subclasses with their status_code attribute already set accordingly. Like HttpResponse itself, these all live at django.http. Some of them take a different set of arguments than the standard HttpResponse, and those differences are also listed here.

• HttpResponseRedirect—Takes a single argument, a URL that the browser will redirect to. It also sets the status_code to 302, indicating a “Found” status, where the resource is located.
• HttpResponsePermanentRedirect—Takes a single argument, a URL that the browser will redirect to. It sets the status_code to 301, indicating the resource was permanently moved to the URL specified.
• HttpResponseNotModified—Sets the status_code to 304, indicating a “Not Modified” status, to be used in response to a conditional GET, when the response hasn’t changed from the conditions associated with the request.
• HttpResponseBadRequest—Sets the status_code to 400, indicating a “Bad Request” where the syntax used in the request couldn’t be understood by the view.
• HttpResponseForbidden—Sets the status_code to 403, “Forbidden,” where the requested resource does exist, but the requesting user doesn’t have permission to access it.
• HttpResponseNotFound—Perhaps most common of all custom classes, this sets the status_code to 404, “Not Found,” where the URL in the request didn’t map to a known resource.
• HttpResponseNotAllowed—Sets the status_code to 405, “Not Allowed,” indicating that the method used in the request isn’t valid for the resource specified by the URL.
• HttpResponseGone—Sets the status_code to 410, “Gone,” to indicate that the resource specified by the URL is no longer available and can’t be located at any other URL.
• HttpResponseServerError—Sets the status_code to 500, “Server Error,” used whenever the view encountered an unrecoverable error.

Some of these specialized responses aren’t supported by Web browsers, but they’re all quite useful for Web service applications, where a wider range of options are available. It often makes more sense to set these statuses on a site-wide basis, so individual views don’t have to worry about managing them directly. For this, Django provides HTTP middleware.

Differences in Scope

One of the most notable differences between middleware and view decorators is how much of the site is covered. Middleware is activated in a site’s settings.py, so it covers all requests that come in on any URL. This simple fact provides a few advantages:

• Many operations—such as caching or compression—should naturally happen for every request on the site; middleware makes these tasks easy to implement.
• Future additions to the site are automatically covered by existing middleware, without having to make any special allowances for the behavior they provide.
• Third-party applications don’t need any modifications in order to take advantage of middleware behavior.

Decorators, on the other hand, are applied to individual functions, which means that every view must have decorators added manually. This makes decorators a bit more time-consuming to manage, but some operations—such as access restriction or specialized cache requirements—are more appropriate for limited parts of the site, where decorators can be used to great effect.

Configuration Options

Middleware classes are referenced as strings containing the import path to the class, which doesn’t allow any direct way to configure any of their features. Most middleware that accept options do so by way of custom settings that are specific to that middleware. This does provide a way to customize how the middleware works, but like middleware themselves, these settings are sitewide, by definition. There isn’t any room for customizing them for individual views.

As shown in Chapter 2, decorators can be written to accept configuration options when they’re applied to a function, and view decorators are no different. Each view could have a separate set of options or curry could be used to create a brand-new decorator with a set of preconfigured arguments.

Using Middleware As Decorators

Given the similarities between middleware and decorators, Django provides a utility to transform an existing middleware class into a decorator. This allows code to be reused across an entire site, using the best tool for the job in any situation.

Living at django.utils.decorators, the special decorator_from_middleware() function takes, as its only argument, a middleware class that should be applied to a single view. The return value is a perfectly functional decorator, which can be applied to any number of views.

class MinimumResponseMiddleware(object):
    """
    Makes sure a response is at least a certain size
    """
 
    def __init__(self, min_length=1024):
        self.min_length = min_length

 
    def process_response(self, request, response):
        """
        Pads the response content to be at least as
        long as the length specified in __init__()
        """
        response.content = response.content.ljust(self.min_length)

Signing Outgoing Response Cookies

The middleware can start with the process_response() method, which will need to find any cookies that were set by the view and add signatures to their values.

class SignedCookiesMiddleware(object):

    def process_response(self, request, response):
        for (key, morsel) in response.cookies.items():
            response.set_signed_cookie(key, morsel.value,
                max_age=morsel['max-age'],
                expires=morsel['expires'],
                path=morsel['path'],
                domain=morsel['domain'],
                secure=morsel['secure']
            )
        return response

This approach uses all the attributes of the original cookie when setting the new one, so that it’s identical except for the method that’s used to set it. Using set_signed_cookie() will do all the appropriate things behind the scenes.

Deleted cookies show up in response.cookies as well, though, even though they don’t have a value and don’t need to be signed. These can be identified by their max-age of 0, which can be used to ignore them and only sign actual values that matter to the application.

class SignedCookiesMiddleware(object):
    def process_response(self, request, response):
        for (key, morsel) in response.cookies.items():
            if morsel['max-age'] == 0:
                # Deleted cookies don't need to be signed
                continue
            response.set_signed_cookie(key, morsel.value,
                max_age=morsel['max-age'],
                expires=morsel['expires'],
                path=morsel['path'],
                domain=morsel['domain'],
                secure=morsel['secure']
            )
        return response

Validating Incoming Request Cookies

Working with incoming requests is also fairly simple. The process_request() method is our entry point for this part of the process, and it merely has to find all the incoming cookies and use get_signed_cookie() to check the signatures and remove those signatures from the values.

class SignedCookiesMiddlewar
e(object):
    def process_request(self, request):
        for key in request.COOKIES:
            request.COOKIES[key] = request.get_signed_cookie(key)

Reading cookies is simpler than writing them, because we don’t have to deal with all the individual parameters; they’re already part of the cookies themselves. This code still has one problem, though. If any signature is missing, invalid or expired, get_signed_cookie() will raise an exception, and we’ll need to handle that in some way.

One option is to simply let the errors go through, hoping they’ll get caught in some other code, but because your views and other middleware won’t even know that this middleware is signing cookies, they aren’t likely to deal with signature exceptions. Worse, if you don’t have code that handles these exceptions, they’ll work their way all the way out to your users, typically in the form of an HTTP 500 error, which doesn’t explain the situation at all.

Instead, this middleware can handle the exceptions directly. Since only values with valid signatures can be passed to your views, an obvious approach is to simply remove any invalid cookies from the request altogether. The exceptions go away, along with the cookies that generated those exceptions. Your views will just see the valid cookies, just like they’re expecting, and any invalid cookies won’t exist in the request anymore. Users can clear their cookies at any time, so views that rely on cookies should always handle requests with missing cookies anyway, so this approach fits well with what views will already be doing.

Supporting this behavior requires nothing more than catching the relevant exceptions and removing those cookies responsible for raising them.

from django.core.signing import BadSignature, SignatureExpired
 
class SignedCookiesMiddleware(object):
    def process_request(self, request):
        for (key, signed_value) in request.COOKIES.items():
            try:
                request.COOKIES[key] = request.get_signed_cookie(key)
            except (BadSignature, SignatureExpired):
                # Invalid cookies should behave as if they were never sent
                del request.COOKIES[key]

Signing Cookies As a Decorator

So far, SignedCookiesMiddleware hasn’t used any of the signature-specific options when setting and retrieving signed cookies. The defaults are often good enough for a middleware that’s meant for use on the whole site. Since middleware can also be used as decorators, though, we also need to account for customizations to individual views. That’s where the salt and expiration settings become useful.

As shown earlier in the chapter, decorator_from_middleware() can supply arguments to the middleware’s __init__() method, so that will provide a path for customizing the salt and max_age arguments. Once accepting those arguments in __init__(), the individual hook methods can incorporate them as appropriate.

from django.core.signing import BadSignature, SignatureExpired
 
class SignedCookiesMiddleware(object):
    def __init__(self, salt='', max_age=None):
        self.salt = salt
        self.max_age = max_age
 
    def process_request(self, request):
        for (key, signed_value) in request.COOKIES.items():
            try:
                request.COOKIES[key] = request.get_signed_cookie(key,
                    salt=self.salt,
                    max_age=self.max_age)
            except (BadSignature, SignatureExpired):
                # Invalid cookies should behave as if they were never sent
                del request.COOKIES[key]
 
    def process_response(self, request, response):
        for (key, morsel) in response.cookies.items():
            if morsel['max-age'] == 0:
                # Deleted cookies don't need to be signed
                continue
            response.set_signed_cookie(key, morsel.value,
                salt=self.salt
                max_age=self.max_age or morsel['max-age'],
                expires=morsel['expires'],
                path=morsel['path'],
                domain=morsel['domain'],
                secure=morsel['secure']
            )
        return response

Now you can create a decorator using decorator_from_middleware_with_args() and supply salt and max_age arguments to customize that decorator’s behavior for each individual view.

from django.utils.decorators import decorator_from_middleware_with_args
signed_cookies = decorator_from_middleware_with_args(SignedCookiesMiddleware)
 
@signed_cookies(salt='foo')
def foo(request, ...):
    ...