This last part of the book deals with exceptions, which are events that can modify the flow of control through a program. In Python, exceptions are triggered automatically on errors, and can be triggered and intercepted by your code. They are processed by four statements we’ll study in this part, the first of which has two variations (listed separately here), and the last of which is an optional extension until Python 2.6:
try
/except
Catch and recover from exceptions raised by Python, or by you.
try
/finally
Perform cleanup actions, whether exceptions occur or not.
raise
Trigger an exception manually in your code.
assert
Conditionally trigger an exception in your code.
with
/as
Implement context managers in Python 2.6 and later (optional in 2.5).
This topic was saved for this last part of the book because you need to know about classes to code exceptions of your own. With a few exceptions (pun intended), though, you’ll find that exception handling is simple in Python because it’s integrated into the language itself as another high-level tool.
One procedural note up front: the exception story has changed in two major ways since this book was first written—the finally
clause can now appear in the same try
statement as except
and else
clauses, and user-defined exceptions should now be coded as class instances not strings. I will describe both the old and new ways of doing things in this edition, because you are still very likely to see the original techniques in code for some time to come. Along the way, I’ll point out how things have evolved in this domain. I’ll also document the new with
statement, even though its official appearance is still one release in the future.
In a nutshell, exceptions let us jump out of arbitrarily large chunks of a program. Consider the pizza-making robot we talked about earlier in the book. Suppose we took the idea seriously, and actually built such a machine. To make a pizza, our culinary automaton would need to execute a plan, which we would implement as a Python program: it would take an order, prepare the dough, add toppings, bake the pie, and so on.
Now, suppose that something goes very wrong during the “bake the pie” step. Perhaps the oven is broken, or perhaps our robot miscalculates its reach, and spontaneously bursts into flames. Clearly, we want to be able to jump to code that handles such states quickly. Also, as we have no hope of finishing the pizza task in such unusual cases, we might as well abandon the entire plan.
That’s exactly what exceptions let you do: you can jump to an exception handler in a single step, abandoning all suspended function calls. An exception is a sort of structured “super-goto."'[73] An exception handler (try
statement) leaves a marker and executes some code. Somewhere further ahead in the program, an exception is raised that makes Python jump back to the marker immediately, abandoning any active functions that were called after the marker was left. Code in the exception handler can respond to the raised exception as appropriate (by calling the fire department, for instance). Moreover, because Python jumps to the handler statement immediately, there is usually no need to check status codes after every call to a function that could possibly fail.
In Python programs, exceptions are typically used for a variety of purposes. Here are some of their most common roles:
Python raises exceptions whenever it detects errors in programs at runtime. You can catch and respond to the errors in your code, or ignore the exceptions that are raised. If an error is ignored, Python’s default exception-handling behavior kicks in—it stops the program, and prints an error message. If you don’t want this default behavior, code a try
statement to catch and recover from the exception—Python will jump to your try
handler when the error is detected, and your program will resume execution after the try
.
Exceptions can also be used to signal valid conditions without you having to pass result flags around a program or test them explicitly. For instance, a search routine might raise an exception on failure, rather than returning an integer result code (and hoping that the code will never be a valid result).
Sometimes a condition may occur so rarely that it’s hard to justify convoluting your code to handle it. You can often eliminate special-case code by handling unusual cases in exception handlers instead.
As you’ll see, the try
/finally
statement allows you to guarantee that required closing-time operations will be performed, regardless of the presence or absence of exceptions in your programs.
And, finally, because exceptions are a sort of high-level “goto,” you can use them as the basis for implementing exotic control flows. For instance, although backtracking is not part of the language itself, it can be implemented in Python with exceptions, and a bit of support logic to unwind assignments.[74]
We’ll see such typical uses in action later in this part of the book. For now, let’s get started with a look at Python’s exception-processing tools.
Compared to some other core language topics we’ve met in this book, exceptions are a fairly lightweight tool in Python. Because they are so simple, let’s jump right into an initial example. Suppose we code the following function:
>>>def fetcher(obj, index):
...return obj[index]
...
There’s not much to this function—it simply indexes an object on a passed-in index. In normal operation, it returns the result of a legal index:
>>>x = 'spam'
>>>fetcher(x, 3)
# Like x[3] 'm'
However, if we ask this function to index off the end of the string, an exception will be triggered when the function tries to run obj[index]
. Python detects out-of-bounds indexing for sequences, and reports it by raising (triggering) the built-in IndexError
exception:
>>> fetcher(x, 4)
Traceback (most recent call last):
File "<stdin>", line 1, in ?
File "<stdin>", line 2, in fetcher
IndexError: string index out of range
Because our code does not explicitly catch this exception, it filters back up to the top level of the program, and invokes the default exception handler—which simply prints the standard error message. By this point in the book, you’ve probably seen your share of standard error messages. They include the exception that was raised, along with a stack trace—a list of the lines and functions active when the exception occurred. When coding interactively, the file is just “stdin” (standard input stream) or “pyshell” (in IDLE), so the file line numbers are not very meaningful here.
In a more realistic program launched outside the interactive prompt, the default handler at the top also terminates the program immediately. That course of action makes sense for simple scripts; errors often should be fatal, and the best you can do when they occur is inspect the standard error message. Sometimes, this isn’t what you want, though. Server programs, for instance, typically need to remain active even after internal errors. If you don’t want the default exception behavior, wrap the call in a try
statement to catch exceptions yourself:
>>>try:
...fetcher(x, 4)
...except IndexError:
...print 'got exception'
... got exception >>>
Now, Python jumps to your handler (the block under the except
clause that names the exception raised) automatically when an exception is triggered while the try
block is running. When working interactively like this, after the except
clause runs, we wind up back at the Python prompt. In a more realistic program, try
statements not only catch exceptions, but also recover from them:
>>>def catcher( ):
...try:
...fetcher(x, 4)
...except IndexError:
...print 'got exception'
...print 'continuing'
... >>>catcher( )
got exception continuing >>>
This time, after the exception is caught and handled, the program resumes execution after the entire try
statement that caught it—which is why we get the “continuing” message here. We don’t see the standard error message, and the program continues on its way normally.
Exceptions can be raised by Python or by your program, and can be caught or not. To trigger an exception manually, simply run a raise
statement (or an assert
, which is a conditional raise
). User-defined exceptions are caught the same way as built-ins:
>>>bad = 'bad'
>>>try:
...raise bad
...except bad:
...print 'got bad'
... got bad
If they’re not caught, user-defined exceptions are propagated up to the top-level default exception handler, and terminate the program with a standard error message. In this case, the standard message includes the text of the string used to identify the exception:
>>> raise bad
Traceback (most recent call last):
File "<pyshell#18>", line 1, in ?
raise bad
bad
In other cases, the error message may include text provided by classes used to identify exceptions. As we’ll see in the next chapter, user-defined exceptions may be defined with strings or classes, but class-based exceptions allow scripts to build exception categories, inherit behavior, and have attached state information. Class-based exceptions are preferred over strings today, and will be required as of Python 3.0:
>>>class Bad(Exception): pass
... >>>def doomed( ): raise Bad( )
... >>>try:
...doomed( )
...except Bad:
...print 'got Bad'
... got Bad >>>
Finally, try
statements can include finally
blocks. The try
/finally
combination specifies termination actions that always execute “on the way out,” regardless of whether an exception occurs in the try
block:
>>>try:
...fetcher(x, 3)
...finally:
...print 'after fetch'
... 'm' after fetch
Here, if the try
block finishes without an exception, the finally
block will run, and the program will resume after the entire try
. In this case, this statement seems a bit silly—we might as well have simply typed the print
right after a call to the function, and skipped the try
altogether:
fetcher(x, 3) print 'after fetch'
There is a problem with coding this way, though: if the function call raises an exception, the print
will never be reached. The try
/finally
combination avoids this pitfall—when an exception does occur in a try
block, finally
blocks are executed while the program is being unwound:
>>>def after( ):
...try:
...fetcher(x, 4)
...finally:
...print 'after fetch'
...print 'after try?'
... >>>after( )
after fetch Traceback (most recent call last): File "<stdin>", line 1, in ? File "<stdin>", line 3, in after File "<stdin>", line 2, in fetcher IndexError: string index out of range
Here, we don’t get the “after try?” message because control does not resume after the try
/finally
block when an exception occurs. Instead, Python jumps back to run the finally
action, but then propagates the exception up to a prior handler (in this case, to the default handler at the top). If we change the call inside this function so as not to trigger an exception, the finally
code still runs, but the program continues after the try
:
>>>def after( ):
...try:
...fetcher(x, 3)
...finally:
...print 'after fetch'
...print 'after try?'
... >>>after( )
after fetch after try? >>>
In practice, try
/except
combinations are useful for catching and recovering from exceptions, and try
/finally
combinations come in handy to guarantee that termination actions will fire regardless of any exceptions that may occur in the try
block’s code. For instance, you might use try
/except
to catch errors raised by code that you import from a third-party library, and try
/finally
to ensure that calls to close files or terminate server connections are always run. We’ll see some such practical examples later in this part of the book.
Although they serve conceptually distinct purposes, as of Python 2.5, we can now mix except
and finally
clauses in the same try
statement—the finally
is run on the way out regardless of whether an exception was raised, and regardless of whether the exception was caught by an except
clause.
That is the majority of the exception story; exceptions really are a simple tool. In the rest of this part, we’ll fill in some of the details about the statements involved, examine the other sorts of clauses that can appear under a try
, and discuss string- and class-based exception objects.
Python exceptions are a high-level control flow device. They may be raised by Python, or by your own programs; in both cases, they may be ignored (to trigger the default error message), or caught by try
statements (to be processed by your code). The try
statement comes in two logical formats that, as of Python 2.5, can be combined—one that handles exceptions, and one that executes finalization code whether exceptions occur or not. Python’s raise
and assert
statements trigger exceptions on demand. With that overview in mind, let’s take a deeper look at these statements’ general forms.
In the following discussion, I’ll first present try
/except
/else
and try
/finally
as separate statements because they serve distinct roles and cannot be combined in versions of Python prior to 2.5. As you’ve seen, in Python 2.5 except
and finally
can be mixed in a single try
statement; I’ll explain the implications of this change after we’ve explored the two original forms in isolation.
try
is a compound statement; its most complete form is sketched below. It starts with a try
header line, followed by a block of (usually) indented statements, then one or more except
clauses that identify exceptions to be caught, and an optional else
clause at the end. The words try
, except
, and else
are associated by indenting them to the same level (i.e., lining them up vertically). For reference, here’s the general format:
try: <statements> # Run this action first except <name1>: <statements> # Run if name1 is raised during try block except <name2>, <data>: <statements> # Run if name2 is raised, and get extra data except (name3, name4): <statements> # Run if any of these exceptions occur except: <statements> # Run for all (other) exceptions raised else: <statements> # Run if no exception was raised during try block
In this statement, the block under the try
header represents the main action of the statement—the code you’re trying to run. The except
clauses define handlers for exceptions raised during the try
block, and the else
clause (if coded) provides a handler to be run if no exceptions occur. The <data>
entry here has to do with a feature of raise
statements, which we will discuss later in this chapter.
Here’s how try
statements work. When a try
statement is started, Python marks the current program context so it can return to it if an exception occurs. The statements nested under the try
header are run first. What happens next depends on whether exceptions are raised while the try
block’s statements are running:
If an exception does occur while the try
block’s statements are running, Python jumps back to the try
, and runs the statements under the first except
clause that matches the raised exception. Control resumes below the entire try
statement after the except
block runs (unless the except
block raises another exception).
If an exception happens in the try
block, and no except
clause matches, the exception is propagated up to a try
that was entered earlier in the program, or to the top level of the process (which makes Python kill the program and print a default error message).
If no exception occurs while the statements under the try
header run, Python runs the statements under the else
line (if present), and control then resumes below the entire try
statement.
In other words, except
clauses catch any exceptions that happen while the try
block is running, and the else
clause runs only if no exceptions happen while the try
block runs.
except
clauses are focused exception handlers—they catch exceptions that occur only within the statements in the associated try
block. However, as the try
block’s statements can call functions coded elsewhere in a program, the source of an exception may be outside the try
statement itself. I’ll have more to say about this when we explore try
nesting in Chapter 29.
When you write try
statements, a variety of clauses can appear after the try
statement block. Table 27-1 summarizes all the possible forms—you must use at least one. We’ve already met some of these: as you know, except
clauses catch exceptions, finally
clauses run on the way out, and else
clauses run if no exceptions are encountered. Syntactically, there may be any number of except
clauses, but there should be only one else
. Through Python 2.4, the finally
clause must appear alone (without else
or except
); it’s really a different statement. As of Python 2.5, however, a finally
can appear in the same statement as except
and else
.
Interpretation | |
|
Catch all (other) exception types. |
|
Catch a specific exception only. |
|
Catch the listed exception and its extra data (or instance). |
|
Catch any of the listed exceptions. |
|
Catch any of the listed exceptions, and get its extra data. |
|
Run if no exceptions are raised. |
|
Always perform this block. |
We’ll explore the entries with the extra value
part when we meet the raise
statement. The first and fourth entries in Table 27-1 are new here:
except
clauses that list no exception name (except:
) catch all exceptions not previously listed in the try
statement.
except
clauses that list a set of exceptions in parentheses (except (e1, e2, e3):
) catch any of the listed exceptions.
Because Python looks for a match within a given try
by inspecting except
clauses from top to bottom, the parenthesized version is like listing each exception in its own except
clause, but the statement body needs to be coded only once. Here’s an example of multiple except clauses at work, which demonstrates just how specific your handlers can be:
try: action( ) except NameError: ... except IndexError ... except KeyError: ... except (AttributeError, TypeError, SyntaxError): ... else: ...
In this example, if an exception is raised while the call to the action
function is running, Python returns to the try
, and searches for the first except
that names the exception raised. It inspects except
clauses from top to bottom and left to right, and runs the statements under the first one that matches. If none match, the exception is propagated past this try
. Note that the else
runs only when no exception occurs in action
—it does not run when an exception without a matching except
is raised.
If you really want a general “catch-all” clause, an empty except
does the trick:
try: action( ) except NameError: ... # Handle NameError except IndexError: ... # Handle IndexError except: ... # Handle all other exceptions else: ... # Handle the no-exception case
The empty except
clause is a sort of wildcard feature—because it catches everything, it allows your handlers to be as general or specific as you like. In some scenarios, this form may be more convenient than listing all possible exceptions in a try
. For example, the following catches everything without listing anything:
try:
action( )
except:
... # Catch all possible exceptions
Empty except
s also raise some design issues, though. Although convenient, they may catch unexpected system exceptions unrelated to your code, and may inadvertently intercept exceptions meant for another handler. For example, even system exit calls in Python trigger exceptions, and you usually want these to pass. We’ll revisit this as a gotcha at the end of this part of the book. For now, I’ll just say: use with care.
In Python 3.0, the third row of Table 27-1 is scheduled to change: except name, value:
will instead be coded as except name as value:
. This change is being made to remove syntax confusion when a tuple of alternative exceptions is coded—the fourth row in Table 27-1 will no longer require enclosing parentheses in 3.0. This change will also modify the scoping rules: with the new as
syntax, the value
variable at the end of the except
block will be deleted.
Also in 3.0, the raise
statement form raise E, V
will need to be coded as raise E(V)
to explicitly generate a class instance to be raised. The prior form had been retained just for backward compatibility with string exceptions in Python 2.x. (See later in this chapter for more on raise
, and the next chapter for a discussion of class-based exceptions.)
Although you can’t use the as
form of except
in Python 2.x to future-proof your code, the “2to3” conversion tool to be shipped with 3.0 will automate the except
and raise
translations for existing 2.x code.
At first glance, the purpose of the else
clause is not always obvious to Python newcomers. Without it, though, there is no way to tell (without setting and checking Boolean flags) whether the flow of control has proceeded past a try
statement because no exception was raised, or because an exception occurred and was handled:
try:...run code...
except IndexError:...handle exception...
# Did we get here because the try failed or not?
Much like the way else
clauses in loops make the exit cause more apparent, the else
clause provides syntax in a try
that makes what has happened obvious and unambiguous:
try:...run code...
except IndexError:...handle exception...
else:...no exception occurred...
You can almost emulate an else
clause by moving its code into the try
block:
try:...run code... ...no exception occurred...
except IndexError:...handle exception...
This can lead to incorrect exception classifications, though. If the “no exception occurred” action triggers an IndexError
, it will register as a failure of the try
block, and hence erroneously trigger the exception handler below the try
(subtle, but true!). By using an explicit else
clause instead, you make the logic more obvious, and guarantee that except
handlers will run only for real failures in the code you’re wrapping in a try
, not for failures in the else
case’s action.
Because the control flow through a program is easier to capture in Python than in English, let’s run some examples that further illustrate exception basics. I’ve mentioned that exceptions not caught by try
statements percolate up to the top level of the Python process, and run Python’s default exception-handling logic (i.e., Python terminates the running program, and prints a standard error message). Let’s look at an example. Running the following module, bad.py, generates a divide-by-zero exception:
def gobad(x, y): return x / y def gosouth(x): print gobad(x, 0) gosouth(1)
Because the program ignores the exception it triggers, Python kills the program, and prints a message:[75]
% python bad.py
Traceback (most recent call last):
File "bad.py", line 7, in <module>
gosouth(1)
File "bad.py", line 5, in gosouth
print gobad(x, 0)
File "bad.py", line 2, in gobad
return x / y
ZeroDivisionError: integer division or modulo by zero
The message consists of a stack trace and the name of (and any extra data about) the exception that was raised. The stack trace lists all lines active when the exception occurred, from oldest to newest. Note that because we’re not working at the interactive prompt, in this case, the file and line number information is useful. For example, here we can see that the bad divide happens at the last entry in the trace—line 2 of the file bad.py, a return
statement.
Because Python detects and reports all errors at runtime by raising exceptions, exceptions are intimately bound up with the ideas of error handling and debugging in general. If you’ve worked through this book’s examples, you’ve undoubtedly seen an exception or two along the way—even typos usually generate a SyntaxError
or other exception when a file is imported or executed (that’s when the compiler is run). By default, you get a useful error display like the one above, which helps you track down the problem.
Often, this standard error message is all you need to resolve problems in your code. For more heavy-duty debugging jobs, you can catch exceptions with try
statements, or use debugging tools I’ll introduce in Chapter 29, such as the pdb
standard library module.
Python’s default exception handling is often exactly what you want—especially for code in a top-level script file, an error generally should terminate your program immediately. For many programs, there is no need to be more specific about errors in your code.
Sometimes, though, you’ll want to catch errors and recover from them instead. If you don’t want your program terminated when Python raises an exception, simply catch it by wrapping the program logic in a try
. This is an important capability for programs such as network servers, which must keep running persistently. For example, the following code catches and recovers from the TypeError
Python raises immediately when you try to concatenate a list and a string (the +
operator wants the same sequence type on both sides):
def kaboom(x, y): print x + y # Trigger TypeError try: kaboom([0,1,2], "spam") except TypeError: # Catch and recover here print 'Hello world!' print 'resuming here' # Continue here if exception or not
When the exception occurs in the function kaboom
, control jumps to the try
statement’s except
clause, which prints a message. Since an exception is “dead” after it’s been caught like this, the program continues executing below the try
, rather than being terminated by Python. In effect, the code processes and clears the error.
Notice that once you’ve caught an error, control resumes at the place where you caught it (i.e., after the try
); there is no direct way to go back to the place where the exception occurred (here, in the function kaboom
). In a sense, this makes exceptions more like simple jumps than function calls—there is no way to return to the code that triggered the error.
The other flavor of the try
statement is a specialization that has to do with finalization actions. If a finally
clause is included in a try
, Python will always run its block of statements “on the way out” of the try
statement, whether an exception occurred while the try
block was running or not. Its general form is:
try: <statements> # Run this action first finally: <statements> # Always run this code on the way out
With this variant, Python begins by running the statement block associated with the try
header line. What happens next depends on whether an exception occurs during the try
block:
If no exception occurs while the try
block is running, Python jumps back to run the finally
block, and then continues execution past the entire try
statement.
If an exception does occur during the try
block’s run, Python still comes back and runs the finally
block, but then propagates the exception up to a higher try
or the top-level default handler; the program does not resume execution below the try
statement. That is, the finally
block is run even if an exception is raised, but unlike an except
, the finally
does not terminate the exception—it continues being raised after the finally
block runs.
The try
/finally
form is useful when you want to be completely sure that an action will happen after some code runs, regardless of the exception behavior of the program. In practice, it allows you to specify cleanup actions that always must occur, such as file closes, and server disconnections.
Note that the finally
clause cannot be used in the same try
statement as except
and else
in Python 2.4 and earlier, so the try
/finally
is best thought of as a distinct statement form if you are using an older release. In Python 2.5, however, finally
can appear in the same statement as except
and else
, so today, there is really a single try
statement with many optional clauses (more about this shortly). Whichever version you use, though, the finally
clause still serves the same purpose—to specify “cleanup” actions that must always be run, regardless of any exceptions.
As we’ll see later in this chapter, in Python 2.6, the with
statement and its context managers provide an object-based way to do similar work for exit actions; however, this statement also supports entry actions.
We saw some simple try
/finally
examples earlier. Here’s a more realistic example that illustrates a typical role for this statement:
class MyError(Exception): pass def stuff(file): raise MyError( ) file = open('data', 'w') # Open an output file try: stuff(file) # Raises exception finally: file.close( ) # Always close file to flush output buffers ... # Continue here only if no exception
In this code, we’ve wrapped a call to a file-processing function in a try
with a finally
clause to make sure that the file is always closed, and thus finalized, whether the function triggers an exception or not. This way, later code can be sure that the file’s output buffer’s content has been flush from memory to disk. A similar code structure can guarantee that server connections are closed, and so on.
This particular example’s function isn’t all that useful (it just raises an exception), but wrapping calls in try
/finally
statements is a good way to ensure that your closing-time (i.e., termination) activities always run. Again, Python always runs the code in your finally
blocks, regardless of whether an exception happens in the try
block.[76]
When the function here raises its exception, the control flow jumps back, and runs the finally
block to close the file. The exception is then propagated on to either another try
or the default top-level handler, which prints the standard error message and shuts down the program; the statement after this try
is never reached. If the function here did not raise an exception, the program would still execute the finally
block to close your file, but it would then continue below the entire try
statement.
Also, notice that the user-defined exception here is again defined with a class—as we’ll see in the next chapter, exceptions today should all be class instances.
In all versions of Python prior to Release 2.5 (for its first 15 years of life, more or less), the try
statement came in two flavors, and was really two separate statements—we could either use a finally
to ensure that cleanup code was always run, or write except
blocks to catch and recover from specific exceptions and optionally specify an else
clause to be run if no exceptions occurred.
That is, the finally
clause could not be mixed with except
and else
. This was partly because of implementation issues, and partly because the meaning of mixing the two seemed obscure—catching and recovering from exceptions, seemed a disjoint concept from performing cleanup actions.
In Python 2.5, though (the version of Python used in this edition of this book), the two statements have merged. Today, we can mix finally
, except
, and else
clauses in the same statement. That is, we can now write a statement of this form:
try:main-action
except Exception1:handler1
except Exception2:handler2
... else:else-block
finally:finally-block
The code in this statement’s main-action
block is executed first, as usual. If that code raises an exception, all the except
blocks are tested, one after another, looking for a match to the exception raised. If the exception raised is Exception1
, the handler1
block is executed; if it’s Exception2
, handler2
is run, and so on. If no exception is raised, the else-block
is executed.
No matter what’s happened previously, the finally-block
is executed once the main action block is complete, and any raised exceptions have been handled. In fact, the code in the finally-block
will be run even if there is an error in an exception handler, or the else-block
and a new exception is raised.
As always, the finally
clause does not end the exception—if an exception is active when the finally-block
is executed, it continues to be propagated after the finally-block
runs, and control jumps somewhere else in the program (to another try
, or to the default top-level handler). If no exception is active when the finally
is run, control resumes after the entire try
statement.
The net effect is that the finally
is always run, regardless of whether:
An exception occurred in the main action and was handled.
An exception occurred in main action and was not handled.
No exceptions occurred in the main action.
A new exception was triggered in one of the handlers.
Again, the finally
serves to specify cleanup actions that must always occur on the way out of the try
, regardless of what exceptions have been raised or handled.
Prior to Python 2.5, it is actually possible to combine finally
and except
clauses in a try
by syntactically nesting a try
/except
in the try
block of a try
/finally
statement (we’ll explore this technique more fully in Chapter 29). In fact, the following has the same effect as the new merged form shown in the previous section:
try: try:main-action
except Exception1:handler1
except Exception2:handler2
... else:no-error
finally:clean-up
Again, the finally
block is always run on the way out, regardless of what happened in the main action, and regardless of any exception handlers run in the nested try
(trace through the four cases listed previously to see how this works the same). However, this equivalent is more obscure, and requires more code than the new merged form. Mixing finally
into the same statement is easier to write and read, and so is the preferred technique today.
Here’s a demonstration of the merged try
statement form at work. The following codes four common scenarios, with print
statements that describe the meaning of each:
print '-' * 30, ' EXCEPTION RAISED AND CAUGHT' try: x = 'spam'[99] except IndexError: print 'except run' finally: print 'finally run' print 'after run' print '-' * 30, ' NO EXCEPTION RAISED' try: x = 'spam'[3] except IndexError: print 'except run' finally: print 'finally run' print 'after run' print '-' * 30, ' NO EXCEPTION RAISED, ELSE RUN' try: x = 'spam'[3] except IndexError: print 'except run' else: print 'else run' finally: print 'finally run' print 'after run' print '-' * 30, ' EXCEPTION RAISED BUT NOT CAUGHT' try: x = 1 / 0 except IndexError: print 'except run' finally: print 'finally run' print 'after run'
When this code is run, the following output is produced. Trace through the code to see how exception handling produces each of the test’s outputs here:
------------------------------ EXCEPTION RAISED AND CAUGHT except run finally run after run ------------------------------ NO EXCEPTION RAISED finally run after run ------------------------------ NO EXCEPTION RAISED, ELSE RUN else run finally run after run ------------------------------ EXCEPTION RAISED BUT NOT CAUGHT finally run Traceback (most recent call last): File "C:/Python25/mergedexc.py", line 32, in <module> x = 1 / 0 ZeroDivisionError: integer division or modulo by zero
This example uses built-in operations in the main action to trigger exceptions or not, and relies on the fact that Python always checks for errors as our code is running. The next section shows how to raise exceptions manually instead.
To trigger exceptions explicitly, you can code raise
statements. Their general form is simple—a raise
statement consists of the word raise
, optionally followed by the name of the exception to be raised, and an optional extra data item to pass with the exception:
raise <name> # Manually trigger an exception raise <name>, <data> # Pass extra data to catcher too raise # Re-raise the most recent exception
The second form allows you to pass an extra data item along with the exception to provide details for the handler. In the raise
statement, the data is listed after the exception name; back in the try
statement, the data is obtained by including a variable to receive it. For instance, if the try
includes an except name, X:
statement, the variable X
will be assigned the extra data item listed in the raise
. The third raise
form simply reraises the current exception; it’s handy if you want to propagate an exception you’ve caught to another handler.
So, what’s an exception name? It might be the name of a built-in exception from the built-in scope (e.g., IndexError
), or the name of an arbitrary string object you’ve assigned in your program. It can also reference a user-defined class, or class instance—a possibility that further generalizes raise
statement formats. I’ll postpone the details of this generalization until after we’ve had a chance to study class exceptions in the next chapter.
Regardless of how you name exceptions, they are always identified by normal objects, and at most one is active at any given time. Once caught by an except
clause anywhere in the program, an exception dies (i.e., won’t propagate to another try
), unless it’s reraised by another raise
statement or error.
Python programs can trigger built-in and user-defined exceptions using the raise
statement. User-defined exceptions should be class instance objects today, like the one that calling MyBad
creates in the following code:
class MyBad: pass def stuff( ): raise MyBad( ) # Trigger exception manually try: stuff( ) # Raises exception except MyBad: print 'got it' # Handle exception here ... # Resume execution here
This time the raise
occurs inside a function, but this makes no real difference—control jumps back to the except
block immediately. Notice that user-defined exceptions are caught with try
statements, just like built-in exceptions.
As stated earlier, a raise
statement can pass an extra data item along with the exception for use in a handler. In general, the extra data allows you to send context information about the exception to a handler. If you’re writing a data file parser, for example, you might raise a syntax error exception on errors and also pass along an object that gives line and file information to the handler (we’ll meet such an example in Chapter 28).
This is useful because when an exception is raised, it may cross arbitrary file boundaries—the raise
statement that triggers an exception, and the try
statement that catches it may be in completely different files. It is not generally feasible to store extra details in global variables because the try
statement might not know which file the globals reside in. Passing extra data along with the exception itself enables the try
statement to access it more reliably. Strictly speaking, every exception has this extra data: as with function return values, it defaults to the special None
object if nothing is passed explicitly. The following code, raisedata.py, illustrates this concept at work with simple string-based exceptions:
myException = 'Error' # String object def raiser1( ): raise myException, "hello" # Raise, pass data def raiser2( ): raise myException # Raise, None implied def tryer(func): try: func( ) except myException, extraInfo: # Run func; catch exception + data print 'got this:', extraInfo %python
>>>from raisedata import *
>>>tryer(raiser1)
# Explicitly passed extra data got this: hello >>>tryer(raiser2)
# Extra data is None by default got this: None
Here, the tryer
function always requests the extra data object; it comes back as an explicit string from raiser1
, but defaults to None
in raiser2
’s raise
statement.
In the next chapter, we’ll see that the same hook can also be used to access instances raised in conjunction with class-based exceptions—the variable in the except
is then assigned to the raised instance, which gives access to attached state information, as well as callable class methods.
A raise
statement that does not include an exception name or extra data value simply reraises the current exception. This form is typically used if you need to catch and handle an exception, but don’t want the exception to die in your code:
>>>try:
...raise IndexError, 'spam'
...except IndexError:
...print 'propagating'
...raise
... propagating Traceback (most recent call last): File "<stdin>", line 2, in ? IndexError: spam
Running a raise
this way reraises the exception, and propagates it to a higher handler, or the default handler at the top, which stops the program with a standard error message.
As a somewhat special case, Python includes the assert
statement. It is mostly syntactic shorthand for a common raise
usage pattern, and can be thought of as a conditional raise
statement. A statement of the form:
assert <test>, <data> # The <data> part is optional
works like the following code:
if _ _debug_ _: if not <test>: raise AssertionError, <data>
In other words, if the test evaluates to false, Python raises an exception: the data item (if it’s provided) as the exception’s extra data. Like all exceptions, the AssertionError
exception raised will kill your program if it’s not caught with a try
.
As an added feature, assert
statements may be removed from a compiled program’s byte code if the -O
Python command-line flag is used, thereby optimizing the program. AssertionError
is a built-in exception, and the _ _debug_ _
flag is a built-in name that is automatically set to 1
(true) unless the -O
flag is used.
Assertions are typically used to verify program conditions during development. When displayed, their error message text automatically includes source code line information, and the value listed in the assert
statement. Consider the file asserter.py:
def f(x): assert x < 0, 'x must be negative' return x ** 2 %python
>>>import asserter
>>>asserter.f(1)
Traceback (most recent call last): File "<stdin>", line 1, in ? File "asserter.py", line 2, in f assert x < 0, 'x must be negative' AssertionError: x must be negative
It’s important to keep in mind that assert
is mostly intended for trapping user-defined constraints, not for catching genuine programming errors. Because Python traps programming errors itself, there is usually no need to code assert
s to catch things like out-of-bounds indexes, type mismatches, and zero divides:
def reciprocal(x): assert x != 0 # A useless assert! return 1 / x # Python checks for zero automatically
Such assert
s are generally superfluous—because Python raises exceptions on errors automatically, you might as well let it do the job for you.[77] For another example of common assert
usage, see the abstract superclass example in Chapter 24; there, we used assert
to make calls to undefined methods fail with a message.
Python 2.6 (still in the future as this edition is being written) will introduce a new exception-related statement—the with
, and its optional as
clause. This statement is designed to work with context-manager objects, which support a new method-based protocol.
In short, the with
/as
statement is designed to be an alternative to a common try
/finally
usage idiom; like that statement, it is intended for specifying termination-time or “cleanup” activities that must run regardless of whether an exception occurs in a processing step. Unlike try
/finally
, though, the with
statement supports a richer object-based protocol for specifying both entry and exit actions around a block of code.
Python enhances some built-in tools with context managers, such as files that automatically close themselves, and thread locks that automatically lock and unlock, but programmers can code context managers of their own with classes, too.
This feature will not become an official part of Python until version 2.6. In Python 2.5, it is not yet available by default; it must be enabled with the special future import statement form we met in the modules part of this book (because of the two new reserved words with
and as
, this feature is being introduced gradually, as usual):
from _ _future_ _ import with_statement
When you run this import
statement in 2.5, you enable the new with
statement, and its two reserved words. The basic format of the with
statement looks like this:
withexpression
[asvariable
]:with-block
The expression
here is assumed to return an object that supports the context management protocol (more on this protocol in a moment). This object may also return a value that will be assigned to the name variable
if the optional as
clause is present.
Note that the variable
is not assigned the result of the expression
; the result of the expression
is the object that supports the context protocol, and the variable
may be assigned something else. The object returned by the expression
may then run startup code before the with-block
is started, as well as termination code after the block is done, whether the block raised an exception or not.
Some built-in Python objects have been augmented to support the context management protocol, and so can be used with the with
statement. For example, file objects have a context manager that automatically closes the file after the with
block, regardless of whether an exception is raised:
with open(r'C:pythonscripts') as myfile:
for line in myfile:
print line
line = line.replace('spam', 'SPAM')
...more code here...
Here, the call to open
returns a simple file object that is assigned to the name myfile
. We can use myfile
with the usual file tools—in this case, the file iterator reads line by line in the for
loop.
However, this object also supports the context management protocol used by the with
statement. After this with
statement has run, the context management machinery guarantees that the file object referenced by myfile
is automatically closed, even if the for
loop raised an exception while processing the file.
We won’t cover Python’s multithreading modules in this book (for more on that topic, see follow-up application-level texts such as Programming Python), but the lock and condition variable synchronization tools they define also support the with
statement by supporting the context management protocol:
lock = threading.Lock( )
with lock:
# critical section of code
...access shared resources...
Here, the context management machinery guarantees that the lock is automatically acquired before the block is executed and released once the block is complete.
The decimal
module (see Chapter 5 for more on decimals) also uses context managers to simplify saving and restoring the current decimal context, which specifies the precision and rounding characteristics for calculations.
The interface expected of objects used in with
statements is somewhat complex, and many programmers only need to know how to use existing context managers. For tool builders who might want to write new ones, though, let’s take a quick look at what is involved. Here’s how the with
statement actually works:
The expression is evaluated, and results in an object known as a context manager, which must have _ _enter_ _
and _ _exit_ _
methods.
The context manager’s _ _enter_ _
method is called. The value it returns is assigned to a variable if the as
clause is present, or simply discarded otherwise.
The code in the nested with
block is executed.
If the with
block raises an exception, the _ _exit_ _(
type
,
value
,
traceback
)
method is called with the exception details. Note that these are the same values returned by sys.exc_info
, described in Python manuals and later in this part of the book. If this method returns a false value, the exception is reraised; otherwise, the exception is terminated. The exception should normally be reraised so that it is propagated outside the with
statement.
If the with
block does not raise an exception, the _ _exit_ _
method is still called, but its type
, value
, and traceback
arguments are all passed in as None
.
Let’s look at a quick demo of the protocol in action. The following defines a context manager object that traces the entry and exit of the with
block in any with
statement it is used for:
from _ _future_ _ import with_statement # Required in Python 2.5
class TraceBlock:
def message(self, arg):
print 'running', arg
def _ _enter_ _(self):
print 'starting with block'
return self
def _ _exit_ _(self, exc_type, exc_value, exc_tb):
if exc_type is None:
print 'exited normally
'
else:
print 'raise an exception!', exc_type
return False # propagate
with TraceBlock( ) as action:
action.message('test 1')
print 'reached'
with TraceBlock( ) as action:
action.message('test 2')
raise TypeError
print 'not reached'
Notice that this class’ _ _exit_ _
method returns False
to propagate the exception; deleting the return
statement there would have the same effect, as the default None
return value of functions is False
by definition. Also, notice how the _ _enter_ _
method returns self
as the object to assign to the as
variable; in other use cases, this might return a completely different object instead.
When run, the context manager traces the entry and exit of the with
statement block with its _ _enter_ _
and _ _exit_ _
methods:
% python withas.py
starting with block
running test 1
reached
exited normally
starting with block
running test 2
raise an exception! <type 'exceptions.TypeError'>
Traceback (most recent call last):
File "C:/Python25/withas.py", line 22, in <module>
raise TypeError
TypeError
Context managers are somewhat advanced devices that are not yet officially part of Python, so we’ll skip additional details here (see Python’s standard manuals for the full story—for example, a new contextlib
standard module provides additional tools for coding context managers). For simpler purposes, the try
/finally
statement provides sufficient support for termination-time activities.
In this chapter, we began our look at exception processing by exploring the statements related to exceptions in Python: try
to catch them, raise
to trigger them, assert
to raise them conditionally, and with
to wrap code blocks in context managers that specify entry and exit actions.
So far, exceptions probably seem like a fairly lightweight tool, and in fact, they are; the only substantially complex thing about them is how they are identified. The next chapter continues our exploration by describing how to implement exception objects of your own; as you’ll see, classes allow you to code more useful exceptions than simple strings today. Before we move ahead, though, work though the following short quiz on the basics covered here.
[73] * If you’ve used C, you may be interested to know that Python exceptions are roughly similar to C’s setjmp
/longjmp
standard function pair: the try
statement acts much like a setjmp
, and raise
works like a longjmp
. But, in Python, exceptions are based on objects and are a standard part of the execution model.
[74] * True backtracking is an advanced topic that is not part of the Python language (even with the addition of generator functions in 2.2), so I won’t say more about it here. Roughly, backtracking undoes all computations before it jumps; Python exceptions do not (i.e., variables assigned between the time a try
statement is entered, and the time an exception is raised, are not reset to their prior values). See a book on artificial intelligence, or the Prolog or Icon programming languages if you’re curious.
[75] * The text of error messages and stack traces tends to vary slightly over time. Don’t be alarmed if your error messages don’t exactly match mine. When I ran this example in Python 2.5’s IDLE, for instance, its error message text showed the full directory paths in filenames.
[76] * Unless Python crashes completely, of course. It does a good job of avoiding this, though, by checking all possible errors as a program runs. When a program does crash hard, it is often due to a bug in linked-in C extension code, outside of Python’s scope.
[77] * In most cases, at least. As suggested in Part IV, if a function has to perform long-running or unrecoverable actions before it reaches the place where an exception will be triggered, you still might want to test for errors. Even in this case, though, be careful not to make your tests overly specific or restrictive, or you will limit your code’s utility.