Table of Contents

Chapter 1: Introduction to Django

This book is about Django, a Web development framework that saves you time and makes Web development a joy. Using Django, you can build and maintain high-quality Web applications with minimal fuss.

At its best, Web development is an exciting, creative act; at its worst, it can be a repetitive, frustrating nuisance. Django lets you focus on the fun stuff — the crux of your Web application — while easing the pain of the repetitive bits. In doing so, it provides high-level abstractions of common Web development patterns, shortcuts for frequent programming tasks, and clear conventions for how to solve problems. At the same time, Django tries to stay out of your way, letting you work outside the scope of the framework as needed.

The goal of this book is to make you a Django expert. The focus is twofold. First, we explain, in depth, what Django does and how to build Web applications with it. Second, we discuss higher-level concepts where appropriate, answering the question “How can I apply these tools effectively in my own projects?” By reading this book, you’ll learn the skills needed to develop powerful Web sites quickly, with code that is clean and easy to maintain.

In this chapter, we provide a high-level overview of Django.

What Is a Web Framework?

Django is a prominent member of a new generation of Web frameworks. So what exactly does that term mean?

To answer that question, let’s consider the design of a Web application written using the Common Gateway Interface (CGI) standard, a popular way to write Web applications circa 1998. In those days, when you wrote a CGI application, you did everything yourself — the equivalent of baking a cake from scratch. For example, here’s a simple CGI script, written in Python, that displays the ten most recently published books from a database:

#!/usr/bin/python

import MySQLdb

print "Content-Type: text/html"
print
print "<html><head><title>Books</title></head>"
print "<body>"
print "<h1>Books</h1>"
print "<ul>"

connection = MySQLdb.connect(user='me', passwd='letmein', db='my_db')
cursor = connection.cursor()
cursor.execute("SELECT name FROM books ORDER BY pub_date DESC LIMIT 10")
for row in cursor.fetchall():
    print "<li>%s</li>" % row[0]

print "</ul>"
print "</body></html>"

connection.close()

This code is straightforward. First, it prints a “Content-Type” line, followed by a blank line, as required by CGI. It prints some introductory HTML, connects to a database and executes a query that retrieves the latest ten books. Looping over those books, it generates an HTML unordered list. Finally, it prints the closing HTML and closes the database connection.

With a one-off dynamic page such as this one, the write-it-from-scratch approach isn’t necessarily bad. For one thing, this code is simple to comprehend — even a novice developer can read these 16 lines of Python and understand all it does, from start to finish. There’s nothing else to learn; no other code to read. It’s also simple to deploy: just save this code in a file called latestbooks.cgi, upload that file to a Web server, and visit that page with a browser.

But as a Web application grows beyond the trivial, this approach breaks down, and you face a number of problems:

These problems are precisely what a Web framework intends to solve. A Web framework provides a programming infrastructure for your applications, so that you can focus on writing clean, maintainable code without having to reinvent the wheel. In a nutshell, that’s what Django does.

The MVC Design Pattern

Let’s dive in with a quick example that demonstrates the difference between the previous approach and that undertaken using a Web framework. Here’s how you might write the previous CGI code using Django:

# models.py (the database tables)

from django.db import models

class Book(models.Model):
    name = models.CharField(maxlength=50)
    pub_date = models.DateField()


# views.py (the business logic)

from django.shortcuts import render_to_response
from models import Book

def latest_books(request):
    book_list = Book.objects.order_by('-pub_date')[:10]
    return render_to_response('latest_books.html', {'book_list': book_list})


# urls.py (the URL configuration)

from django.conf.urls.defaults import *
import views

urlpatterns = patterns('',
    (r'latest/$', views.latest_books),
)


# latest_books.html (the template)

<html><head><title>Books</title></head>
<body>
<h1>Books</h1>
<ul>
{% for book in book_list %}
<li>{{ book.name }}</li>
{% endfor %}
</ul>
</body></html>

Don’t worry about the particulars of how this works just yet — we just want you to get a feel for the overall design. The main thing to note here is the separation of concerns:

Taken together, these pieces loosely follow the Model-View-Controller (MVC) design pattern. Simply put, MVC defines a way of developing software so that the code for defining and accessing data (the model) is separate from request routing logic (the controller), which in turn is separate from the user interface (the view).

A key advantage of such an approach is that components are loosely coupled. That is, each distinct piece of a Django-powered Web application has a single key purpose and can be changed independently without affecting the other pieces. For example, a developer can change the URL for a given part of the application without affecting the underlying implementation. A designer can change a page’s HTML without having to touch the Python code that renders it. A database administrator can rename a database table and specify the change in a single place, rather than having to search and replace through a dozen files.

In this book, each component of this stack gets its own chapter. For example, Chapter 3 covers views, Chapter 4 covers templates, and Chapter 5 covers models. Chapter 5 also discusses Django’s MVC philosophies in depth.

Django’s History

Before we dive into more code, we should take a moment to explain Django’s history. It’s helpful to understand why the framework was created, because a knowledge of the history will put into context why Django works the way it does.

If you’ve been building Web applications for a while, you’re probably familiar with the problems in the CGI example we presented earlier. The classic Web developer’s path goes something like this:

  1. Write a Web application from scratch.
  2. Write another Web application from scratch.
  3. Realize the application from step 1 shares much in common with the application from step 2.
  4. Refactor the code so that application 1 shares code with application 2.
  5. Repeat steps 2-4 several times.
  6. Realize you’ve invented a framework.

This is precisely how Django itself was created!

Django grew organically from real-world applications written by a Web development team in Lawrence, Kansas. It was born in the fall of 2003, when the Web programmers at the Lawrence Journal-World newspaper, Adrian Holovaty and Simon Willison, began using Python to build applications. The World Online team, responsible for the production and maintenance of several local news sites, thrived in a development environment dictated by journalism deadlines. For the sites — including LJWorld.com, Lawrence.com, and KUsports.com — journalists (and management) demanded that features be added and entire applications be built on an intensely fast schedule, often with only days’ or hours’ notice. Thus, Adrian and Simon developed a time-saving Web development framework out of necessity — it was the only way they could build maintainable applications under the extreme deadlines.

In summer 2005, after having developed this framework to a point where it was efficiently powering most of World Online’s sites, the World Online team, which now included Jacob Kaplan-Moss, decided to release the framework as open source software. They released it in July 2005 and named it Django, after the jazz guitarist Django Reinhardt.

Although Django is now an open source project with contributors across the planet, the original World Online developers still provide central guidance for the framework’s growth, and World Online contributes other important aspects such as employee time, marketing materials, and hosting/bandwidth for the framework’s Web site (http://www.djangoproject.com/).

This history is relevant because it helps explain two key matters. The first is Django’s “sweet spot.” Because Django was born in a news environment, it offers several features (particularly its admin interface, covered in Chapter 6) that are particularly well suited for “content” sites — sites like eBay, craigslist.org, and washingtonpost.com that offer dynamic, database-driven information. (Don’t let that turn you off, though — although Django is particularly good for developing those sorts of sites, that doesn’t preclude it from being an effective tool for building any sort of dynamic Web site. There’s a difference between being particularly effective at something and being ineffective at other things.)

The second matter to note is how Django’s origins have shaped the culture of its open source community. Because Django was extracted from real-world code, rather than being an academic exercise or commercial product, it is acutely focused on solving Web development problems that Django’s developers themselves have faced — and continue to face. As a result, Django itself is actively improved on an almost daily basis. The framework’s developers have a keen interest in making sure Django saves developers time, produces applications that are easy to maintain, and performs well under load. If nothing else, the developers are motivated by their own selfish desires to save themselves time and enjoy their jobs. (To put it bluntly, they eat their own dog food.)

How to Read This Book

In writing this book, we tried to strike a balance between readability and reference, with a bias toward readability. Our goal with this book, as stated earlier, is to make you a Django expert, and we believe the best way to teach is through prose and plenty of examples, rather than a providing an exhaustive but bland catalog of Django features. (As someone once said, you can’t expect to teach somebody how to speak merely by teaching them the alphabet.)

With that in mind, we recommend that you read Chapters 1 through 7 in order. They form the foundation of how to use Django; once you’ve read them, you’ll be able to build Django-powered Web sites. The remaining chapters, which focus on specific Django features, can be read in any order.

The appendixes are for reference. They, along with the free documentation at http://www.djangoproject.com/, are probably what you’ll flip back to occasionally to recall syntax or find quick synopses of what certain parts of Django do.

Required Programming Knowledge

Readers of this book should understand the basics of procedural and object-oriented programming: control structures (if, while, and for), data structures (lists, hashes/dictionaries), variables, classes, and objects.

Experience in Web development is, as you may expect, very helpful, but it’s not required to read this book. Throughout the book, we try to promote best practices in Web development for readers who lack this type of experience.

Required Python Knowledge

At its core, Django is simply a collection of libraries written in the Python programming language. To develop a site using Django, you write Python code that uses these libraries. Learning Django, then, is a matter of learning how to program in Python and understanding how the Django libraries work.

If you have experience programming in Python, you should have no trouble diving in. By and large, the Django code doesn’t perform “black magic” (i.e., programming trickery whose implementation is difficult to explain or understand). For you, learning Django will be a matter of learning Django’s conventions and APIs.

If you don’t have experience programming in Python, you’re in for a treat. It’s easy to learn and a joy to use! Although this book doesn’t include a full Python tutorial, it highlights Python features and functionality where appropriate, particularly when code doesn’t immediately make sense. Still, we recommend you read the official Python tutorial, available online at http://docs.python.org/tut/. We also recommend Mark Pilgrim’s free book Dive Into Python, available at http://www.diveintopython.org/ and published in print by Apress.

New Django Features

As we noted earlier, Django is frequently improved, and it will likely have a number of useful — even essential — new features by the time this book is published. Thus, our goal as authors of this book is twofold:

  • Make sure this book is as “future-proof” as possible, so that whatever you read here will still be relevant in future Django versions
  • Actively update this book on its Web site, http://www.djangobook.com/, so you can access the latest and greatest documentation as soon as we write it

If you want to implement something with Django that isn’t explained in this book, check the latest version of this book on the aforementioned Web site, and also check the official Django documentation.

Getting Help

One of the greatest benefits of Django is its kind and helpful user community. For help with any aspect of Django — from installation, to application design, to database design, to deployment — feel free to ask questions online.

  • The django-users mailing list is where thousands of Django users hang out to ask and answer questions. Sign up for free at http://www.djangoproject.com/r/django-users.
  • The Django IRC channel is where Django users hang out to chat and help each other in real time. Join the fun by logging on to #django on the Freenode IRC network.

What’s Next

In the next chapter, we’ll get started with Django, covering installation and initial setup.

Chapter 2: Getting Started

We think it’s best to get a running start. The details and extent of the Django framework will be fleshed out in the later chapters, but for now, trust us, this chapter will be fun.

Installing Django is easy. Because Django runs anywhere Python does, Django can be configured in many ways. We cover the common scenarios for Django installations in this chapter. Chapter 20 covers deploying Django to production.

Installing Python

Django is written in 100% pure Python code, so you’ll need to install Python on your system. Django requires Python 2.3 or higher.

If you’re on Linux or Mac OS X, you probably already have Python installed. Type python at a command prompt (or in Terminal, in OS X). If you see something like this, then Python is installed:

Python 2.4.1 (#2, Mar 31 2005, 00:05:10)
[GCC 3.3 20030304 (Apple Computer, Inc. build 1666)] on darwin
Type "help", "copyright", "credits" or "license" for more information.
>>>

Otherwise, if you see an error such as "command not found", you’ll have to download and install Python. See http://www.python.org/download/ to get started. The installation is fast and easy.

Installing Django

In this section, we cover two installation options: installing an official release and installing from Subversion.

Installing an Official Release

Most people will want to install the latest official release from http://www.djangoproject.com/download/. Django uses the standard Python distutils installation method, which in Linux land looks like this:

  1. Download the tarball, which will be named something like Django-0.96.tar.gz.
  2. tar xzvf Django-*.tar.gz.
  3. cd Django-*.
  4. sudo python setup.py install.

On Windows, we recommend using 7-Zip to handle all manner of compressed files, including .tar.gz. You can download 7-Zip from http://www.djangoproject.com/r/7zip/.

Change into some other directory and start python. If everything worked, you should be able to import the module django:

>>> import django
>>> django.VERSION
(0, 96, None)

Note

The Python interactive interpreter is a command-line program that lets you write a Python program interactively. To start it, just run the command python at the command line. Throughout this book, we feature example Python code that’s printed as if it’s being entered in the interactive interpreter. The triple greater-than signs (>>>) signify a Python prompt.

Installing Django from Subversion

If you want to work on the bleeding edge, or if you want to contribute code to Django itself, you should install Django from its Subversion repository.

Subversion is a free, open source revision-control system similar to CVS, and the Django team uses it to manage changes to the Django codebase. You can use a Subversion client to grab the very latest Django source code and, at any given time, you can update your local version of the Django code, known as your local checkout, to get the latest changes and improvements made by Django developers.

The latest and greatest Django development code is referred to as the trunk. The Django team runs production sites on trunk and strives to keep it stable.

To grab the latest Django trunk, follow these steps:

  1. Make sure you have a Subversion client installed. You can get the software free from http://subversion.tigris.org/, and you can find excellent documentation at http://svnbook.red-bean.com/.
  2. Check out the trunk using the command svn co http://code.djangoproject.com/svn/django/trunk djtrunk.
  3. Create site-packages/django.pth and add the djtrunk directory to it, or update your PYTHONPATH to point to djtrunk.
  4. Place djtrunk/django/bin on your system PATH. This directory includes management utilities such as django-admin.py.

Tip:

If .pth files are new to you, you can learn more about them at http://www.djangoproject.com/r/python/site-module/.

After downloading from Subversion and following the preceding steps, there’s no need to python setup.py install—you’ve just done the work by hand!

Because the Django trunk changes often with bug fixes and feature additions, you’ll probably want to update it every once in a while — or hourly, if you’re really obsessed. To update the code, just run the command svn update from within the djtrunk directory. When you run that command, Subversion will contact http://code.djangoproject.com, determine if any code has changed, and update your local version of the code with any changes that have been made since you last updated. It’s quite slick.

Setting Up a Database

Django’s only prerequisite is a working installation of Python. However, this book focuses on one of Django’s sweet spots, which is developing database-backed Web sites, so you’ll need to install a database server of some sort, for storing your data.

If you just want to get started playing with Django, skip ahead to the “Starting a Project” section—but trust us, you’ll want to install a database eventually. All of the examples in the book assume you have a database set up.

As of the time of this writing, Django supports three database engines:

Work is in progress to support Microsoft SQL Server and Oracle. The Django Web site will always have the latest information about supported databases.

We’re quite fond of PostgreSQL ourselves, for reasons outside the scope of this book, so we mention it first. However, all the engines listed here will work equally well with Django.

SQLite deserves special notice as a development tool. It’s an extremely simple in-process database engine that doesn’t require any sort of server setup or configuration. It’s by far the easiest to set up if you just want to play around with Django, and it’s even included in the standard library of Python 2.5.

On Windows, obtaining database driver binaries is sometimes an involved process. Since you’re just getting started with Django, we recommend using Python 2.5 and its built-in support for SQLite. Compiling driver binaries is a downer.

Using Django with PostgreSQL

If you’re using PostgreSQL, you’ll need the psycopg package available from http://www.djangoproject.com/r/python-pgsql/. Take note of whether you’re using version 1 or 2; you’ll need this information later.

If you’re using PostgreSQL on Windows, you can find precompiled binaries of psycopg at http://www.djangoproject.com/r/python-pgsql/windows/.

Using Django with SQLite 3

If you’re using a Python version over 2.5, you already have SQLite. If you’re working with Python 2.4 or older, you’ll need SQLite 3— not version 2—from http://www.djangoproject.com/r/sqlite/ and the pysqlite package from http://www.djangoproject.com/r/python-sqlite/. Make sure you have pysqlite version 2.0.3 or higher.

On Windows, you can skip installing the separate SQLite binaries, since they’re statically linked into the pysqlite binaries.

Using Django with MySQL

Django requires MySQL 4.0 or above; the 3.x versions don’t support nested subqueries and some other fairly standard SQL statements. You’ll also need the MySQLdb package from http://www.djangoproject.com/r/python-mysql/.

Using Django Without a Database

As mentioned earlier, Django doesn’t actually require a database. If you just want to use it to serve dynamic pages that don’t hit a database, that’s perfectly fine.

With that said, bear in mind that some of the extra tools bundled with Django do require a database, so if you choose not to use a database, you’ll miss out on those features. (We highlight these features throughout this book.)

Starting a Project

A project is a collection of settings for an instance of Django, including database configuration, Django-specific options, and application-specific settings.

If this is your first time using Django, you’ll have to take care of some initial setup. Create a new directory to start working in, perhaps something like /home/username/djcode/, and change into that directory.

Note

django-admin.py should be on your system path if you installed Django via its setup.py utility. If you checked out from Subversion, you can find it in djtrunk/django/bin. Since you’ll be using django-admin.py often, consider adding it to your path. On Unix, you can do so by symlinking from /usr/local/bin, using a command such as sudo ln -s /path/to/django/bin/django-admin.py /usr/local/bin/django-admin.py. On Windows, you’ll need to update your PATH environment variable.

Run the command django-admin.py startproject mysite to create a mysite directory in your current directory.

Let’s look at what startproject created:

mysite/
    __init__.py
    manage.py
    settings.py
    urls.py

These files are as follows:

Where Should This Directory Live?

If your background is in PHP, you’re probably used to putting code under the Web server’s document root (in a place such as /var/www). With Django, you don’t do that. It’s not a good idea to put any of this Python code within your Web server’s document root, because in doing so you risk the possibility that people will be able to view your code over the Web. That’s not good for security.

Put your code in some directory outside of the document root.

The Development Server

Django includes a built-in, lightweight Web server you can use while developing your site. We’ve included this server so you can develop your site rapidly, without having to deal with configuring your production Web server (e.g., Apache) until you’re ready for production. This development server watches your code for changes and automatically reloads, helping you make many rapid changes to your project without needing to restart anything.

Change into the mysite directory, if you haven’t already, and run the command python manage.py runserver. You’ll see something like this:

Validating models...
0 errors found.

Django version 1.0, using settings 'mysite.settings'
Development server is running at http://127.0.0.1:8000/
Quit the server with CONTROL-C.

Although the development server is extremely nice for, well, development, resist the temptation to use this server in anything resembling a production environment. The development server can handle only a single request at a time reliably, and it has not gone through a security audit of any sort. When the time comes to launch your site, see Chapter 20 for information on how to deploy Django.

Changing the Host or the Port

By default, the runserver command starts the development server on port 8000, listening only for local connections. If you want to change the server’s port, pass it as a command-line argument:

python manage.py runserver 8080

You can also change the IP address that the server listens on. This is especially helpful if you’d like to share a development site with other developers. The following:

python manage.py runserver 0.0.0.0:8080

will make Django listen on any network interface, thus allowing other computers to connect to the development server.

Now that the server’s running, visit http://127.0.0.1:8000/ with your Web browser. You’ll see a “Welcome to Django” page shaded a pleasant pastel blue. It worked!

What’s Next?

Now that you have everything installed and the development server running, in the next chapter you’ll write some basic code that demonstrates how to serve Web pages using Django.

Chapter 3: The Basics of Dynamic Web Pages

In the previous chapter, we explained how to set up a Django project and run the Django development server. Of course, that site doesn’t actually do anything useful yet—all it does is display the “It worked!” message. Let’s change that. This chapter introduces how to create dynamic Web pages with Django.

Your First View: Dynamic Content

As our first goal, let’s create a Web page that displays the current date and time. This is a good example of a dynamic Web page, because the contents of the page are not static—rather, the contents change according to the result of a computation (in this case, a calculation of the current time). This simple example doesn’t involve a database or any sort of user input—just the output of your server’s internal clock.

To create this page, we’ll write a view function. A view function, or view for short, is simply a Python function that takes a Web request and returns a Web response. This response can be the HTML contents of a Web page, or a redirect, or a 404 error, or an XML document, or an image … or anything, really. The view itself contains whatever arbitrary logic is necessary to return that response. This code can live anywhere you want, as long as it’s on your Python path. There’s no other requirement—no “magic,” so to speak. For the sake of putting the code somewhere, let’s create a file called views.py in the mysite directory, which you created in the previous chapter.

Here’s a view that returns the current date and time, as an HTML document:

from django.http import HttpResponse
import datetime

def current_datetime(request):
    now = datetime.datetime.now()
    html = "<html><body>It is now %s.</body></html>" % now
    return HttpResponse(html)

Let’s step through this code one line at a time:

Django’s Time Zone

Django includes a TIME_ZONE setting that defaults to America/Chicago. This probably isn’t where you live, so you might want to change it in your settings.py. See Appendix E for details.

Mapping URLs to Views

So, to recap, this view function returns an HTML page that includes the current date and time. But how do we tell Django to use this code? That’s where URLconfs come in.

A URLconf is like a table of contents for your Django-powered Web site. Basically, it’s a mapping between URL patterns and the view functions that should be called for those URL patterns. It’s how you tell Django, “For this URL, call this code, and for that URL, call that code.” Remember that the view functions need to be on the Python path.

Your Python Path

Your Python path is the list of directories on your system where Python looks when you use the Python import statement.

For example, let’s say your Python path is set to ['', '/usr/lib/python2.4/site-packages', '/home/username/djcode/']. If you execute the Python code from foo import bar, Python will first check for a module called foo.py in the current directory. (The first entry in the Python path, an empty string, means “the current directory.”) If that file doesn’t exist, Python will look for the file /usr/lib/python2.4/site-packages/foo.py. If that file doesn’t exist, it will try /home/username/djcode/foo.py. Finally, if that file doesn’t exist, it will raise ImportError.

If you’re interested in seeing the value of your Python path, start the Python interactive interpreter and type import sys, followed by print sys.path.

Generally you don’t have to worry about setting your Python path—Python and Django will take care of things for you automatically behind the scenes. (If you’re curious, setting the Python path is one of the things that the manage.py file does.)

When you executed django-admin.py startproject in the previous chapter, the script created a URLconf for you automatically: the file urls.py. Let’s edit that file. By default, it looks something like this:

from django.conf.urls.defaults import *

urlpatterns = patterns('',
    # Example:
    # (r'^mysite/', include('mysite.apps.foo.urls.foo')),

    # Uncomment this for admin:
#     (r'^admin/', include('django.contrib.admin.urls')),
)

Let’s step through this code one line at a time:

The main thing to note here is the variable urlpatterns, which Django expects to find in your ROOT_URLCONF module. This variable defines the mapping between URLs and the code that handles those URLs.

By default, everything in the URLconf is commented out—your Django application is a blank slate. (As a side note, that’s how Django knew to show you the “It worked!” page in the last chapter. If your URLconf is empty, Django assumes you just started a new project and, hence, displays that message.)

Let’s edit this file to expose our current_datetime view:

from django.conf.urls.defaults import *
from mysite.views import current_datetime

urlpatterns = patterns('',
    (r'^time/$', current_datetime),
)

We made two changes here. First, we imported the current_datetime view from its module (mysite/views.py, which translates into mysite.views in Python import syntax). Next, we added the line (r'^time/$', current_datetime),. This line is referred to as a URLpattern—it’s a Python tuple in which the first element is a simple regular expression and the second element is the view function to use for that pattern.

In a nutshell, we just told Django that any request to the URL /time/ should be handled by the current_datetime view function.

A few things are worth pointing out:

To test our changes to the URLconf, start the Django development server, as you did in Chapter 2, by running the command python manage.py runserver. (If you left it running, that’s fine, too. The development server automatically detects changes to your Python code and reloads as necessary, so you don’t have to restart the server between changes.) The server is running at the address http://127.0.0.1:8000/, so open up a Web browser and go to http://127.0.0.1:8000/time/. You should see the output of your Django view.

Hooray! You’ve made your first Django-powered Web page.

Regular Expressions

Regular expressions (or regexes) are a compact way of specifying patterns in text. While Django URLconfs allow arbitrary regexes for powerful URL-matching capability, you’ll probably use only a few regex patterns in practice. Here’s a small selection of common patterns:

Symbol Matches
. (dot) Any character
\d Any digit
[A-Z] Any character, A-Z (uppercase)
[a-z] Any character, a-z (lowercase)
[A-Za-z] Any character, a-z (case insensitive)
+ One or more of the previous expression (e.g., \d+ matches one or more digit)
[^/]+ All characters except forward slash
? Zero or more of the previous expression (e.g., \d* matches zero or more digits)
{1,3} Between one and three (inclusive) of the previous expression

For more on regular expressions, see http://www.djangoproject.com/r/python/re-module/.

How Django Processes a Request

We should point out several things about what just happened. Here’s the nitty-gritty of what goes on when you run the Django development server and make requests to Web pages:

You now know the basics of how to make Django-powered pages. It’s quite simple, really—just write view functions and map them to URLs via URLconfs. You might think it would be slow to map URLs to functions using a series of regular expressions, but you’d be surprised.

How Django Processes a Request: Complete Details

In addition to the straightforward URL-to-view mapping just described, Django provides quite a bit of flexibility in processing requests.

The typical flow—URLconf resolution to a view function which returns an HttpResponse—can be short-circuited or augmented via middleware. The deep secrets of middleware will be fully covered in Chapter 15, but a quick sketch (see Figure 3-1) should aid you in conceptually fitting the pieces together.

The complete flow of a Django request and response.

Figure 3-1: The complete flow of a Django request and response.

When an HTTP request comes in from the browser, a server-specific handler constructs the HttpRequest passed to later components and handles the flow of the response processing.

The handler then calls any available Request or View middleware. These types of middleware are useful for augmenting incoming HttpRequest objects as well as providing special handling for specific types of requests. If either returns an HttpResponse, processing bypasses the view.

Bugs slip by even the best programmers, but exception middleware can help squash them. If a view function raises an exception, control passes to the Exception middleware. If this middleware does not return an HttpResponse, the exception is re-raised.

Even then, all is not lost. Django includes default views that create a friendly 404 and 500 response.

Finally, response middleware is good for post-processing an HttpResponse just before it’s sent to the browser or doing cleanup of request-specific resources.

URLconfs and Loose Coupling

Now’s a good time to highlight a key philosophy behind URLconfs and behind Django in general: the principle of loose coupling. Simply put, loose coupling is a software-development approach that values the importance of making pieces interchangeable. If two pieces of code are loosely coupled, then changes made to one of the pieces will have little or no effect on the other.

Django’s URLconfs are a good example of this principle in practice. In a Django Web application, the URL definitions and the view functions they call are loosely coupled; that is, the decision of what the URL should be for a given function, and the implementation of the function itself, reside in two separate places. This lets a developer switch out one piece without affecting the other.

In contrast, other Web development platforms couple the URL to the program. In typical PHP (http://www.php.net/) applications, for example, the URL of your application is designated by where you place the code on your filesystem. In early versions of the CherryPy Python Web framework (http://www.cherrypy.org/), the URL of your application corresponded to the name of the method in which your code lived. This may seem like a convenient shortcut in the short term, but it can get unmanageable in the long run.

For example, consider the view function we wrote earlier, which displays the current date and time. If we wanted to change the URL for the application— say, move it from /time/ to /currenttime/—we could make a quick change to the URLconf, without having to worry about the underlying implementation of the function. Similarly, if we wanted to change the view function—altering its logic somehow—we could do that without affecting the URL to which the function is bound. Furthermore, if we wanted to expose the current-date functionality at several URLs, we could easily take care of that by editing the URLconf, without having to touch the view code.

That’s loose coupling in action. We’ll continue to point out examples of this important philosophy throughout this book.

404 Errors

In our URLconf thus far, we’ve defined only a single URLpattern: the one that handles requests to the URL /time/. What happens when a different URL is requested?

To find out, try running the Django development server and hitting a page such as http://127.0.0.1:8000/hello/ or http://127.0.0.1:8000/does-not-exist/, or even http://127.0.0.1:8000/ (the site “root”). You should see a “Page not found” message (see Figure 3-2). (Pretty, isn’t it? We Django people sure do like our pastel colors.) Django displays this message because you requested a URL that’s not defined in your URLconf.

Screenshot of Django’s 404 page.

Figure 3-2. Django’s 404 page

The utility of this page goes beyond the basic 404 error message; it also tells you precisely which URLconf Django used and every pattern in that URLconf. From that information, you should be able to tell why the requested URL threw a 404.

Naturally, this is sensitive information intended only for you, the Web developer. If this were a production site deployed live on the Internet, we wouldn’t want to expose that information to the public. For that reason, this “Page not found” page is only displayed if your Django project is in debug mode. We’ll explain how to deactivate debug mode later. For now, just know that every Django project is in debug mode when you first create it, and if the project is not in debug mode, a different response is given.

Your Second View: Dynamic URLs

In our first view example, the contents of the page—the current date/time— were dynamic, but the URL (/time/) was static. In most dynamic Web applications, though, a URL contains parameters that influence the output of the page.

Let’s create a second view that displays the current date and time offset by a certain number of hours. The goal is to craft a site in such a way that the page /time/plus/1/ displays the date/time one hour into the future, the page /time/plus/2/ displays the date/time two hours into the future, the page /time/plus/3/ displays the date/time three hours into the future, and so on.

A novice might think to code a separate view function for each hour offset, which might result in a URLconf like this:

urlpatterns = patterns('',
    (r'^time/$', current_datetime),
    (r'^time/plus/1/$', one_hour_ahead),
    (r'^time/plus/2/$', two_hours_ahead),
    (r'^time/plus/3/$', three_hours_ahead),
    (r'^time/plus/4//$', four_hours_ahead),
)

Clearly, this line of thought is flawed. Not only would this result in redundant view functions, but also the application is fundamentally limited to supporting only the predefined hour ranges—one, two, three, or four hours. If, all of a sudden, we wanted to create a page that displayed the time five hours into the future, we’d have to create a separate view and URLconf line for that, furthering the duplication and insanity. We need to do some abstraction here.

A Word About Pretty URLs

If you’re experienced in another Web development platform, such as PHP or Java, you may be thinking, “Hey, let’s use a query string parameter!”, something like /time/plus?hours=3, in which the hours would be designated by the hours parameter in the URL’s query string (the part after the ?).

You can do that with Django (and we’ll tell you how later, if you really must know), but one of Django’s core philosophies is that URLs should be beautiful. The URL /time/plus/3/ is far cleaner, simpler, more readable, easier to recite to somebody aloud and … just plain prettier than its query string counterpart. Pretty URLs are a sign of a quality Web application.

Django’s URLconf system encourages pretty URLs by making it easier to use pretty URLs than not to.

Wildcard URLpatterns

Continuing with our hours_ahead example, let’s put a wildcard in the URLpattern. As we mentioned previously, a URLpattern is a regular expression; hence, we can use the regular expression pattern \d+ to match one or more digits:

from django.conf.urls.defaults import *
from mysite.views import current_datetime, hours_ahead

urlpatterns = patterns('',
    (r'^time/$', current_datetime),
    (r'^time/plus/\d+/$', hours_ahead),
)

This URLpattern will match any URL such as /time/plus/2/, /time/plus/25/, or even /time/plus/100000000000/. Come to think of it, let’s limit it so that the maximum allowed offset is 99 hours. That means we want to allow either one- or two-digit numbers—in regular expression syntax, that translates into \d{1,2}:

(r'^time/plus/\d{1,2}/$', hours_ahead),

Note

When building Web applications, it’s always important to consider the most outlandish data input possible, and decide whether or not the application should support that input. We’ve curtailed the outlandishness here by limiting the offset to 99 hours. And, by the way, The Outlandishness Curtailers would be a fantastic, if verbose, band name.

Now that we’ve designated a wildcard for the URL, we need a way of passing that data to the view function, so that we can use a single view function for any arbitrary hour offset. We do this by placing parentheses around the data in the URLpattern that we want to save. In the case of our example, we want to save whatever number was entered in the URL, so let’s put parentheses around the \d{1,2}:

(r'^time/plus/(\d{1,2})/$', hours_ahead),

If you’re familiar with regular expressions, you’ll be right at home here; we’re using parentheses to capture data from the matched text.

The final URLconf, including our previous current_datetime view, looks like this:

from django.conf.urls.defaults import *
from mysite.views import current_datetime, hours_ahead

urlpatterns = patterns('',
    (r'^time/$', current_datetime),
    (r'^time/plus/(\d{1,2})/$', hours_ahead),
)

With that taken care of, let’s write the hours_ahead view.

Coding Order

In this example, we wrote the URLpattern first and the view second, but in the previous example, we wrote the view first, then the URLpattern. Which technique is better? Well, every developer is different.

If you’re a big-picture type of person, it may make the most sense to you to write all of the URLpatterns for your application at the same time, at the start of your project, and then code up the views. This has the advantage of giving you a clear to-do list, and it essentially defines the parameter requirements for the view functions you’ll need to write.

If you’re more of a bottom-up developer, you might prefer to write the views first, and then anchor them to URLs afterward. That’s OK, too.

In the end, it comes down to which technique fits your brain the best. Both approaches are valid.

hours_ahead is very similar to the current_datetime view we wrote earlier, with a key difference: it takes an extra argument, the number of hours of offset. Add this to views.py:

def hours_ahead(request, offset):
    offset = int(offset)
    dt = datetime.datetime.now() + datetime.timedelta(hours=offset)
    html = "<html><body>In %s hour(s), it will be %s.</body></html>" % (offset, dt)
    return HttpResponse(html)

Let’s step through this code one line at a time:

  • Just as we did for our current_datetime view, we import the class django.http.HttpResponse and the datetime module.

  • The view function, hours_ahead, takes two parameters: request and offset.

    • request is an HttpRequest object, just as in current_datetime. We’ll say it again: each view always takes an HttpRequest object as its first parameter.

    • offset is the string captured by the parentheses in the URLpattern. For example, if the requested URL were /time/plus/3/, then offset would be the string '3'. If the requested URL were /time/plus/21/, then offset would be the string '21'. Note that captured strings will always be strings, not integers, even if the string is composed of only digits, such as '21'.

      We decided to call the variable offset, but you can call it whatever you’d like, as long as it’s a valid Python identifier. The variable name doesn’t matter; all that matters is that it’s the second argument to the function (after request). It’s also possible to use keyword, rather than positional, arguments in an URLconf. We cover that in Chapter 8.

  • The first thing we do within the function is call int() on offset. This converts the string value to an integer.

    Note that Python will raise a ValueError exception if you call int() on a value that cannot be converted to an integer, such as the string 'foo'. However, in this example we don’t have to worry about catching that exception, because we can be certain offset will be a string containing only digits. We know that because the regular-expression pattern in our URLconf— (\d{1,2})—captures only digits. This illustrates another nicety of URLconfs: they provide a fair level of input validation.

  • The next line of the function shows why we called int() on offset. On this line, we calculate the current time plus a time offset of offset hours, storing the result in dt. The datetime.timedelta function requires the hours parameter to be an integer.

  • Next, we construct the HTML output of this view function, just as we did in current_datetime. A small difference in this line from the previous line is that it uses Python’s format-string capability with two values, not just one. Hence, there are two %s symbols in the string and a tuple of values to insert: (offset, dt).

  • Finally, we return an HttpResponse of the HTML—again, just as we did in current_datetime.

With that view function and URLconf written, start the Django development server (if it’s not already running), and visit http://127.0.0.1:8000/time/plus/3/ to verify it works. Then try http://127.0.0.1:8000/time/plus/5/. Then http://127.0.0.1:8000/time/plus/24/. Finally, visit http://127.0.0.1:8000/time/plus/100/ to verify that the pattern in your URLconf only accepts one- or two-digit numbers; Django should display a “Page not found” error in this case, just as we saw in the “404 Errors” section earlier. The URL http://127.0.0.1:8000/time/plus/ (with no hour designation) should also throw a 404.

If you’re following along while coding at the same time, you’ll notice that the views.py file now contains two views. (We omitted the current_datetime view from the last set of examples for clarity.) Put together, views.py should look like this:

from django.http import HttpResponse
import datetime

def current_datetime(request):
    now = datetime.datetime.now()
    html = "<html><body>It is now %s.</body></html>" % now
    return HttpResponse(html)

def hours_ahead(request, offset):
    offset = int(offset)
    dt = datetime.datetime.now() + datetime.timedelta(hours=offset)
    html = "<html><body>In %s hour(s), it will be %s.</body></html>" % (offset, dt)
    return HttpResponse(html)

Django’s Pretty Error Pages

Take a moment to admire the fine Web application we’ve made so far … now let’s break it! We’ll deliberately introduce a Python error into our views.py file by commenting out the offset = int(offset) line in the hours_ahead view:

def hours_ahead(request, offset):
    #offset = int(offset)
    dt = datetime.datetime.now() + datetime.timedelta(hours=offset)
    html = "<html><body>In %s hour(s), it will be %s.</body></html>" % (offset, dt)
    return HttpResponse(html)

Load up the development server and navigate to /time/plus/3/. You’ll see an error page with a significant amount of information, including a TypeError message displayed at the very top: "unsupported type for timedelta hours component: str".

What happened? Well, the datetime.timedelta function expects the hours parameter to be an integer, and we commented out the bit of code that converted offset to an integer. That caused datetime.timedelta to raise the TypeError. It’s the typical kind of small bug that every programmer runs into at some point.

The point of this example was to demonstrate Django’s error pages. Take some time to explore the error page and get to know the various bits of information it gives you.

Here are some things to notice:

The Django error page is capable of displaying more information in certain special cases, such as the case of template syntax errors. We’ll get to those later, when we discuss the Django template system. For now, uncomment the offset = int(offset) line to get the view function working properly again.

Are you the type of programmer who likes to debug with the help of carefully placed print statements? You can use the Django error page to do so—just without the print statements. At any point in your view, temporarily insert an assert False to trigger the error page. Then, you can view the local variables and state of the program. (There’s a more advanced way to debug Django views, which we’ll explain later, but this is the quickest and easiest.)

Finally, it’s obvious that much of this information is sensitive—it exposes the innards of your Python code and Django configuration—and it would be foolish to show this information on the public Internet. A malicious person could use it to attempt to reverse-engineer your Web application and do nasty things. For that reason, the Django error page is only displayed when your Django project is in debug mode. We’ll explain how to deactivate debug mode later. For now, just know that every Django project is in debug mode automatically when you start it. (Sound familiar? The “Page not found” errors, described in the “404 Errors” section, work the same way.)

What’s next?

We’ve so far been producing views by hard-coding HTML into the Python code. Unfortunately, this is nearly always a bad idea. Luckily, Django ships with a simple yet powerful template engine that allows you to separate the design of the page from the underlying code. We’ll dive into Django’s template engine in the next chapter.

Chapter 4: The Django Template System

In the previous chapter, you may have noticed something peculiar in how we returned the text in our example views. Namely, the HTML was hard-coded directly in our Python code.

This arrangement leads to several problems:

For these reasons, it’s much cleaner and more maintainable to separate the design of the page from the Python code itself. We can do this with Django’s template system, which we discuss in this chapter.

Template System Basics

A Django template is a string of text that is intended to separate the presentation of a document from its data. A template defines placeholders and various bits of basic logic (i.e., template tags) that regulate how the document should be displayed. Usually, templates are used for producing HTML, but Django templates are equally capable of generating any text-based format.

Let’s dive in with a simple example template. This template describes an HTML page that thanks a person for placing an order with a company. Think of it as a form letter:

<html>
<head><title>Ordering notice</title></head>

<body>

<p>Dear {{ person_name }},</p>

<p>Thanks for placing an order from {{ company }}. It's scheduled to
ship on {{ ship_date|date:"F j, Y" }}.</p>

<p>Here are the items you've ordered:</p>

<ul>
{% for item in item_list %}
<li>{{ item }}</li>
{% endfor %}
</ul>

{% if ordered_warranty %}
<p>Your warranty information will be included in the packaging.</p>
{% endif %}

<p>Sincerely,<br />{{ company }}</p>

</body>
</html>

This template is basic HTML with some variables and template tags thrown in. Let’s step through it:

Each Django template has access to several built-in tags and filters, many of which are discussed in the sections that follow. Appendix F contains the full list of tags and filters, and it’s a good idea to familiarize yourself with that list so you know what’s possible. It’s also possible to create your own filters and tags, which we cover in Chapter 10.

Using the Template System

To use the template system in Python code, just follow these two steps:

  1. Create a Template object by providing the raw template code as a string. Django also offers a way to create Template objects by designating the path to a template file on the filesystem; we’ll examine that in a bit.
  2. Call the render() method of the Template object with a given set of variables (i.e., the context). This returns a fully rendered template as a string, with all of the variables and block tags evaluated according to the context.

The following sections describe each step in more detail.

Creating Template Objects

The easiest way to create a Template object is to instantiate it directly. The Template class lives in the django.template module, and the constructor takes one argument, the raw template code. Let’s dip into the Python interactive interpreter to see how this works in code.

Interactive Interpreter Examples

Throughout this book, we feature example Python interactive interpreter sessions. You can recognize these examples by the triple greater-than signs (>>>), which designate the interpreter’s prompt. If you’re copying examples from this book, don’t copy those greater-than signs.

Multiline statements in the interactive interpreter are padded with three dots (...), for example:

>>> print """This is a
... string that spans
... three lines."""
This is a
string that spans
three lines.
>>> def my_function(value):
...     print value
>>> my_function('hello')
hello

Those three dots at the start of the additional lines are inserted by the Python shell—they’re not part of our input. We include them here to be faithful to the actual output of the interpreter. If you copy our examples to follow along, don’t copy those dots.

From within the project directory created by django-admin.py startproject (as covered in Chapter 2), type python manage.py shell to start the interactive interpreter. Here’s a basic walk-through:

>>> from django.template import Template
>>> t = Template("My name is {{ name }}.")
>>> print t

If you’re following along interactively, you’ll see something like this:

<django.template.Template object at 0xb7d5f24c>

That 0xb7d5f24c will be different every time, and it doesn’t really matter; it’s simply the Python “identity” of the Template object.

Django Settings

When using Django, you need to tell Django which settings to use. Interactively, this is typically done using python manage.py shell, but you’ve got a few other options described in Appendix E.

When you create a Template object, the template system compiles the raw template code into an internal, optimized form, ready for rendering. But if your template code includes any syntax errors, the call to Template() will cause a TemplateSyntaxError exception:

>>> from django.template import Template
>>> t = Template('{% notatag %} ')
Traceback (most recent call last):
  File "<stdin>", line 1, in ?
  ...
  django.template.TemplateSyntaxError: Invalid block tag: 'notatag'

The system raises a TemplateSyntaxError exception for any of the following cases:

  • Invalid block tags
  • Invalid arguments to valid block tags
  • Invalid filters
  • Invalid arguments to valid filters
  • Invalid template syntax
  • Unclosed block tags (for block tags that require closing tags)

Rendering a Template

Once you have a Template object, you can pass it data by giving it a context. A context is simply a set of variables and their associated values. A template uses this to populate its variable tags and evaluate its block tags.

A context is represented in Django by the Context class, which lives in the django.template module. Its constructor takes one optional argument: a dictionary mapping variable names to variable values. Call the Template object’s render() method with the context to “fill” the template:

>>> from django.template import Context, Template
>>> t = Template("My name is {{ name }}.")
>>> c = Context({"name": "Stephane"})
>>> t.render(c)
'My name is Stephane.'

Dictionaries and Contexts

A Python dictionary is a mapping between known keys and variable values. A Context is similar to a dictionary, but a Context provides additional functionality, as covered in Chapter 10.

Variable names must begin with a letter (A-Z or a-z) and may contain digits, underscores, and dots. (Dots are a special case we’ll get to in a moment.) Variable names are case sensitive.

Here’s an example of template compilation and rendering, using the sample template from the beginning of this chapter:

>>> from django.template import Template, Context
>>> raw_template = """<p>Dear {{ person_name }},</p>
...
... <p>Thanks for ordering {{ product }} from {{ company }}. It's scheduled
... to ship on {{ ship_date|date:"F j, Y" }}.</p>
...
... {% if ordered_warranty %}
... <p>Your warranty information will be included in the packaging.</p>
... {% endif %}
...
... <p>Sincerely,<br />{{ company }}</p>"""
>>> t = Template(raw_template)
>>> import datetime
>>> c = Context({'person_name': 'John Smith',
...     'product': 'Super Lawn Mower',
...     'company': 'Outdoor Equipment',
...     'ship_date': datetime.date(2009, 4, 2),
...     'ordered_warranty': True})
>>> t.render(c)
"<p>Dear John Smith,</p>\n\n<p>Thanks for ordering Super Lawn Mower from
Outdoor Equipment. It's scheduled \nto ship on April 2, 2009.</p>\n\n\n
<p>Your warranty information will be included in the packaging.</p>\n\n\n
<p>Sincerely,<br />Outdoor Equipment</p>"

Let’s step through this code one statement at a time:

  • First, we import the classes Template and Context, which both live in the module django.template.

  • We save the raw text of our template into the variable raw_template. Note that we use triple quote marks to designate the string, because it wraps over multiple lines; in Python codde, strings designated with single quote marks cannot be wrapped over multiple lines.

  • Next, we create a template object, t, by passing raw_template to the Template class constructor.

  • We import the datetime module from Python’s standard library, because we’ll need it in the following statement.

  • Then, we create a Context object, c. The Context constructor takes a Python dictionary, which maps variable names to values. Here, for example, we specify that the person_name is 'John Smith', product is 'Super Lawn Mower', and so forth.

  • Finally, we call the render() method on our template object, passing it the context. This returns the rendered template—that is, it replaces template variables with the actual values of the variables, and it executes any block tags.

    Note that the warranty paragraph was displayed because the ordered_warranty variable evaluated to True. Also note the date, April 2, 2009, which is displayed according to the format string 'F j, Y'. (We explain format strings for the date filter shortly.)

    If you’re new to Python, you may wonder why this output includes newline characters ('\n') rather than displaying the line breaks. That’s happening because of a subtlety in the Python interactive interpreter: the call to t.render(c) returns a string, and by default the interactive interpreter displays the representation of the string, rather than the printed value of the string. If you want to see the string with line breaks displayed as true line breaks rather than '\n' characters, use the print statement: print t.render(c).

Those are the fundamentals of using the Django template system: just write a template, create a Template object, create a Context, and call the render() method.

Multiple Contexts, Same Template

Once you have a Template object, you can render multiple contexts through it, for example:

>>> from django.template import Template, Context
>>> t = Template('Hello, {{ name }}')
>>> print t.render(Context({'name': 'John'}))
Hello, John
>>> print t.render(Context({'name': 'Julie'}))
Hello, Julie
>>> print t.render(Context({'name': 'Pat'}))
Hello, Pat

Whenever you’re using the same template source to render multiple contexts like this, it’s more efficient to create the Template object once, and then call render() on it multiple times:

# Bad
for name in ('John', 'Julie', 'Pat'):
    t = Template('Hello, {{ name }}')
    print t.render(Context({'name': name}))

# Good
t = Template('Hello, {{ name }}')
for name in ('John', 'Julie', 'Pat'):
    print t.render(Context({'name': name}))

Django’s template parsing is quite fast. Behind the scenes, most of the parsing happens via a single call to a short regular expression. This is in stark contrast to XML-based template engines, which incur the overhead of an XML parser and tend to be orders of magnitude slower than Django’s template rendering engine.

Context Variable Lookup

In the examples so far, we’ve passed simple values in the contexts—mostly strings, plus a datetime.date example. However, the template system elegantly handles more complex data structures, such as lists, dictionaries, and custom objects.

The key to traversing complex data structures in Django templates is the dot character (.). Use a dot to access dictionary keys, attributes, indices, or methods of an object.

This is best illustrated with a few examples. For instance, suppose you’re passing a Python dictionary to a template. To access the values of that dictionary by dictionary key, use a dot:

>>> from django.template import Template, Context
>>> person = {'name': 'Sally', 'age': '43'}
>>> t = Template('{{ person.name }} is {{ person.age }} years old.')
>>> c = Context({'person': person})
>>> t.render(c)
'Sally is 43 years old.'

Similarly, dots also allow access of object attributes. For example, a Python datetime.date object has year, month, and day attributes, and you can use a dot to access those attributes in a Django template:

>>> from django.template import Template, Context
>>> import datetime
>>> d = datetime.date(1993, 5, 2)
>>&