Lines and Indentation

A Python program is a sequence of logical lines, each made up of one or more physical lines. Each physical line may end with a comment. A hash sign # that is not inside a string literal starts a comment. All characters after the #, up to but excluding the line end, are the comment: Python ignores them. A line containing only whitespace, possibly with a comment, is a blank line: Python ignores it.¹

In Python, the end of a physical line marks the end of most statements. Unlike in other languages, you don’t normally terminate Python statements with a delimiter, such as a semicolon (;). When a statement is too long to fit on a physical line, you can join two adjacent physical lines into a logical line by ensuring that the first physical line has no comment and ends with a backslash (). However, Python also automatically joins adjacent physical lines into one logical line if an open parenthesis ((), bracket ([), or brace ({) has not yet been closed: take advantage of this mechanism to produce more readable code than you’d get with backslashes at line ends. Triple-quoted string literals can also span physical lines. Physical lines after the first one in a logical line are known as continuation lines. Indentation issues apply to the first physical line of each logical line, not to continuation lines.

Python uses indentation to express the block structure of a program. Unlike other languages, Python does not use braces, or other begin/end delimiters, around blocks of statements; indentation is the only way to denote blocks. Each logical line in a Python program is indented by the whitespace on its left. A block is a contiguous sequence of logical lines, all indented by the same amount; a logical line with less indentation ends the block. All statements in a block must have the same indentation, as must all clauses in a compound statement. The first statement in a source file must have no indentation (i.e., must not begin with any whitespace). Statements that you type at the interactive interpreter primary prompt >>> (covered in “Interactive Sessions”) must also have no indentation.

v2 logically replaces each tab by up to eight spaces, so that the next character after the tab falls into logical column 9, 17, 25, and so on. Standard Python style is to use four spaces (never tabs) per indentation level.

Don’t mix spaces and tabs for indentation, since different tools (e.g., editors, email systems, printers) treat tabs differently. The -t and -tt options to the v2 Python interpreter (covered in “Command-Line Syntax and Options”) ensure against inconsistent tab and space usage in Python source code. In v3, Python does not allow mixing tabs and spaces for indentation.

Character Sets

A v3 source file can use any Unicode character, encoded as UTF-8. (Characters with codes between 0 and 127, AKA ASCII characters, encode in UTF-8 into the respective single bytes, so an ASCII text file is a fine v3 Python source file, too.)

A v2 source file is usually made up of characters from the ASCII set (character codes between 0 and 127).

In both v2 and v3, you may choose to tell Python that a certain source file is written in a different encoding. In this case, Python uses that encoding to read the file (in v2, you can use non-ASCII characters only in comments and string literals).

To let Python know that a source file is written with a nonstandard encoding, start your source file with a comment whose form must be, for example:

# coding: iso-8859-1

After coding:, write the name of a codec known to Python and ASCII-compatible, such as utf-8 or iso-8859-1. Note that this coding directive comment (also known as an encoding declaration) is taken as such only if it is at the start of a source file (possibly after the “shebang line,” covered in “Running Python Programs”). The only effect of a coding directive in v2 is to let you use non-ASCII characters in string literals and comments. Best practice is to use utf-8 for all of your text files, including Python source files.

Tokens

Python breaks each logical line into a sequence of elementary lexical components known as tokens. Each token corresponds to a substring of the logical line. The normal token types are identifiers, keywords, operators, delimiters, and literals, which we cover in the following sections. You may freely use whitespace between tokens to separate them. Some whitespace separation is necessary between logically adjacent identifiers or keywords; otherwise, Python would parse them as a single, longer identifier. For example, ifx is a single identifier; to write the keyword if followed by the identifier x, you need to insert some whitespace (e.g., if x).

42                       # Integer literal
3.14                     # Floating-point literal
1.0j                     # Imaginary literal
'hello'                  # String literal
"world"                  # Another string literal
"""Good
night"""                 # Triple-quoted string literal

[42, 3.14, 'hello']    # List
[]                     # Empty list
100, 200, 300          # Tuple
()                     # Empty tuple
{'x':42, 'y':3.14}     # Dictionary
{}                     # Empty dictionary
{1, 2, 4, 8, 'string'} # Set
# There is no literal to denote an empty set; use set() instead

Statements

You can look at a Python source file as a sequence of simple and compound statements. Unlike some other languages, Python has no “declarations” or other top-level syntax elements: just statements.

Numbers

The built-in numeric types in Python include integers (int and long, in v2; in v3, there’s no distinction between kinds of integers), floating-point numbers, and complex numbers. The standard library also offers decimal floating-point numbers, covered in “The decimal Module”, and fractions, covered in “The fractions Module”. All numbers in Python are immutable objects; therefore, when you perform an operation on a number object, you produce a new number object. We cover operations on numbers, also known as arithmetic operations, in “Numeric Operations”.

Numeric literals do not include a sign: a leading + or -, if present, is a separate operator, as discussed in “Arithmetic Operations”.

1, 23, 3493                  # Decimal integer literals
0b010101, 0b110010           # Binary integer literals
0o1, 0o27, 0o6645            # Octal integer literals
0x1, 0x17, 0xDA5             # Hexadecimal integer literals

0., 0.0, .0, 1., 1.0, 1e0, 1.e0, 1.0e0  # Floating-point literals

0j, 0.j, 0.0j, .0j, 1j, 1.j, 1.0j, 1e0j, 1.e0j, 1.0e0j

>>> 100_000.000_0001, 0x_FF_FF, 0o7_777, 0b_1010_1010
(100000.0000001, 65535, 4095, 170)

Sequences

A sequence is an ordered container of items, indexed by integers. Python has built-in sequence types known as strings (bytes and Unicode), tuples, and lists. Library and extension modules provide other sequence types, and you can write yet others yourself (as discussed in “Sequences”). You can manipulate sequences in a variety of ways, as discussed in “Sequence Operations”.

'This is a literal string'
"This is another string"

'I'm a Python fanatic'        # a quote can be escaped
"I'm a Python fanatic"         # this way is more readable

'A not very long string 
that spans two lines'      # comment not allowed on previous line

'A not very long string

that prints on two lines'  # comment not allowed on previous line

"""An even bigger
string that spans
three lines"""           # comments not allowed on previous lines

the_text = """
First line
Second line
"""  # like 'First line
Second line
' but more readable

marypop = ('supercalifragilistic' # Open paren->logical line continues
           'expialidocious')      # Indentation ignored in continuation

(100, 200, 300)            # Tuple with three items
(3.14,)                    # Tuple with 1 item needs trailing comma
()                         # Empty tuple (parentheses NOT optional)

tuple('wow')

('w', 'o', 'w')

[42, 3.14, 'hello']        # List with three items
[100]                      # List with one item
[]                         # Empty list

list('wow')

['w', 'o', 'w']

Sets

Python has two built-in set types, set and frozenset, to represent arbitrarily ordered collections of unique items. Items in a set may be of different types, but they must be hashable (see hash in Table 7-2). Instances of type set are mutable, and thus, not hashable; instances of type frozenset are immutable and hashable. You can’t have a set whose items are sets, but you can have a set (or frozenset) whose items are frozensets. Sets and frozensets are not ordered.

To create a set, you can call the built-in type set with no argument (this means an empty set) or one argument that is iterable (this means a set whose items are those of the iterable). You can similarly build a frozenset by calling frozenset.

Alternatively, to denote a (nonfrozen, nonempty) set, use a series of expressions (the items of the set) separated by commas (,) and within braces ({}); if every item is a literal, the whole assembly is a set literal. You may optionally place a redundant comma after the last item. Some example sets (two literals, one not):

{42, 3.14, 'hello'}     # Literal for a set with three items
{100}                   # Literal for a set with one item
set()                   # Empty set (can't use {}—empty dict!)

You can also build nonfrozen sets with set comprehensions, as discussed in “List comprehensions”.

Dictionaries

A mapping is an arbitrary collection of objects indexed by nearly³ arbitrary values called keys. Mappings are mutable and, like sets but unlike sequences, are not (necessarily) ordered.

Python provides a single built-in mapping type: the dictionary type. Library and extension modules provide other mapping types, and you can write others yourself (as discussed in “Mappings”). Keys in a dictionary may be of different types, but they must be hashable (see hash in Table 7-2). Values in a dictionary are arbitrary objects and may be of any type. An item in a dictionary is a key/value pair. You can think of a dictionary as an associative array (known in some other languages as an “unordered map,” “hash table,” or “hash”).

To denote a dictionary, you can use a series of colon-separated pairs of expressions (the pairs are the items of the dictionary) separated by commas (,) within braces ({}); if every expression is a literal, the whole assembly is a dict literal. You may optionally place a redundant comma after the last item. Each item in a dictionary is written as key:value, where key is an expression giving the item’s key and value is an expression giving the item’s value. If a key’s value appears more than once in a dictionary expression, only an arbitrary one of the items with that key is kept in the resulting dictionary object—dictionaries do not allow duplicate keys. To denote an empty dictionary, use an empty pair of braces.

Here are some dictionary literals:

{'x':42, 'y':3.14, 'z':7}    # Dictionary with three items, str keys
{1:2, 3:4}                   # Dictionary with two items, int keys
{1:'za', 'br':23}            # Dictionary with mixed key types
{}                           # Empty dictionary

You can also call the built-in type dict to create a dictionary in a way that, while usually less concise, can sometimes be more readable. For example, the dictionaries in the preceding snippet can equivalently be written as:

dict(x=42, y=3.14, z=7)      # Dictionary with three items, str keys
dict([(1, 2), (3, 4)])       # Dictionary with two items, int keys
dict([(1,'za'), ('br',23)])  # Dictionary with mixed key types
dict()                       # Empty dictionary

dict() without arguments creates and returns an empty dictionary, like {}. When the argument x to dict is a mapping, dict returns a new dictionary object with the same keys and values as x. When x is iterable, the items in x must be pairs, and dict(x) returns a dictionary whose items (key/value pairs) are the same as the items in x. If a key value appears more than once in x, only the last item from x with that key value is kept in the resulting dictionary.

When you call dict, in addition to, or instead of, the positional argument x, you may pass named arguments, each with the syntax name=value, where name is an identifier to use as an item’s key and value is an expression giving the item’s value. When you call dict and pass both a positional argument and one or more named arguments, if a key appears both in the positional argument and as a named argument, Python associates to that key the value given with the named argument (i.e., the named argument “wins”).

You can also create a dictionary by calling dict.fromkeys. The first argument is an iterable whose items become the keys of the dictionary; the second argument is the value that corresponds to each and every key (all keys initially map to the same value). If you omit the second argument, it defaults to None. For example:

dict.fromkeys('hello', 2)   # same as {'h':2, 'e':2, 'l':2, 'o':2}
dict.fromkeys([1, 2, 3])    # same as {1:None, 2:None, 3:None}

You can also build dicts with dict comprehensions, as discussed in “List comprehensions”.

None

The built-in None denotes a null object. None has no methods or other attributes. You can use None as a placeholder when you need a reference but you don’t care what object you refer to, or when you need to indicate that no object is there. Functions return None as their result unless they have specific return statements coded to return other values.

Callables

In Python, callable types are those whose instances support the function call operation (see “Calling Functions”). Functions are callable. Python provides several built-in functions (see “Built-in Functions”) and supports user-defined functions (see “The def Statement”). Generators are also callable (see “Generators”).

Types are also callable, as we already saw for the dict, list, set, and tuple built-in types. (See “Built-in Types” for a complete list of built-in types.) As we discuss in “Python Classes”, class objects (user-defined types) are also callable. Calling a type normally creates and returns a new instance of that type.

Other callables are methods, which are functions bound to class attributes, and instances of classes that supply a special method named __call__.

Boolean Values

Any data value in Python can be used as a truth value: true or false. Any nonzero number or nonempty container (e.g., string, tuple, list, set, or dictionary) is true. 0 (of any numeric type), None, and empty containers are false.

The built-in type bool is a subclass of int. The only two values of type bool are Trueand False, which have string representations of 'True' and 'False', but also numerical values of 1 and 0, respectively. Several built-in functions return bool results, as do comparison operators.

You can call bool(x) with any x as the argument. The result is True when x is true and False when x is false. Good Python style is not to use such calls when they are redundant, as they most often are: always write if x:, never any of if bool(x):, if x is True, if x==True:, if bool(x)==True. However, you can use bool(x) to count the number of true items in a sequence. For example:

def count_trues(seq): return sum(bool(x) for x in seq)

In this example, the bool call ensures each item of seq is counted as 0 (if false) or 1 (if true), so count_trues is more general than sum(seq) would be.

When we write “expression is true,” we mean that bool(expression) would return True.

Variables

In Python, there are no “declarations.” The existence of a variable begins with a statement that binds the variable (in other words, sets a name to hold a reference to some object). You can also unbind a variable, resetting the name so it no longer holds a reference. Assignment statements are the most common way to bind variables and other references. The del statement unbinds references.

Binding a reference that was already bound is also known as rebinding it. Whenever we mention binding, we implicitly include rebinding (except where we explicitly exclude it). Rebinding or unbinding a reference has no effect on the object to which the reference was bound, except that an object goes away when nothing refers to it. The cleanup of objects with no references is known as garbage collection.

You can name a variable with any identifier except the 30-plus reserved as Python’s keywords (see “Keywords”). A variable can be global or local. A global variable is an attribute of a module object (see Chapter 6). A local variable lives in a function’s local namespace (see “Namespaces”).

Assignment Statements

Assignment statements can be plain or augmented. Plain assignment to a variable (e.g., name=value) is how you create a new variable or rebind an existing variable to a new value. Plain assignment to an object attribute (e.g., x.attr=value) is a request to object x to create or rebind attribute 'attr'. Plain assignment to an item in a container (e.g., x[k]=value) is a request to container x to create or rebind the item with index or key k.

Augmented assignment (e.g., name+=value) cannot, per se, create new references. Augmented assignment can rebind a variable, ask an object to rebind one of its existing attributes or items, or request the target object to modify itself. When you make a request to an object, it is up to the object to decide whether and how to honor the request, and whether to raise an exception.

target = expression

a = b = c = 0

a, b, c = x

a, b = b, a

first, *middle, last = x

first, middle, last = x[0], x[1:-1], x[-1]

del Statements

Despite its name, a del statement unbinds references—it does not, per se, delete objects. Object deletion may automatically follow as a consequence, by garbage collection, when no more references to an object exist.

A del statement consists of the keyword del, followed by one or more target references separated by commas (,). Each target can be a variable, attribute reference, indexing, or slicing, just like for assignment statements, and must be bound at the time del executes. When a del target is an identifier, the del statement means to unbind the variable. If the identifier was bound, unbinding it is never disallowed; when requested, it takes place.

In all other cases, the del statement specifies a request to an object to unbind one or more of its attributes or items. An object may refuse to unbind some (or all) attributes or items, raising an exception if you attempt a disallowed unbinding (see also __delattr__ in “General-Purpose Special Methods” and __delitem__ in “Container methods”). Unbinding a slicing normally has the same effect as assigning an empty sequence to that slicing, but it is up to the container object to implement this equivalence.

Containers are also allowed to have del cause side effects. For example, assuming del C[2] succeeds, when C is a dict, this makes future references to C[2] invalid (raising KeyError) until and unless you assign to C[2] again; but when C is a list, del C[2] implies that every following item of C “shifts left by one”—so, if C is long enough, future references to C[2] are still valid, but denote a distinct item than they did before the del.

Comparison Chaining

You can chain comparisons, implying a logical and. For example:

a < b <= c < d

has the same meaning as:

a < b and b <= c and c < d

The chained form is more readable, and evaluates each subexpression at most once.⁴

Short-Circuiting Operators

The and and or operators short-circuit their operands’ evaluation: the righthand operand evaluates only when its value is necessary to get the truth value of the entire and or or operation.

In other words, x and y first evaluates x. When x is false, the result is x; otherwise, the result is y. Similarly, x or y first evaluates x. When x is true, the result is x; otherwise, the result is y.

and and or don’t force their results to be True or False, but rather return one or the other of their operands. This lets you use these operators more generally, not just in Boolean contexts. and and or, because of their short-circuiting semantics, differ from other operators, which fully evaluate all operands before performing the operation. and and or let the left operand act as a guard for the right operand.

whentrue if condition else whenfalse

Numeric Conversions

You can perform arithmetic operations and comparisons between any two numbers of Python built-in types. If the operands’ types differ, coercion applies: Python converts the operand with the “smaller” type to the “larger” type. The types, in order from smallest to largest, are integers, floating-point numbers, and complex numbers. You can request an explicit conversion by passing a noncomplex numeric argument to any of the built-in number types: int, float, and complex. int drops its argument’s fractional part, if any (e.g., int(9.8) is 9). You can also call complex with two numeric arguments, giving real and imaginary parts. You cannot convert a complex to another numeric type in this way, because there is no single unambiguous way to convert a complex number into, for example, a float.

You can also call each built-in numeric type with a string argument with the syntax of an appropriate numeric literal, with small extensions: the argument string may have leading and/or trailing whitespace, may start with a sign, and—for complex numbers—may sum or subtract a real part and an imaginary one. int can also be called with two arguments: the first one a string to convert, and the second the radix, an integer between 2 and 36 to use as the base for the conversion (e.g., int('101', 2) returns 5, the value of '101' in base 2). For radices larger than 10, the appropriate subset of letters from the start of the alphabet (in either lower- or uppercase) are the extra needed “digits.”

Arithmetic Operations

Python arithmetic operations behave in rather obvious ways, with the possible exception of division and exponentiation.

from __future__ import division

Comparisons

All objects, including numbers, can be compared for equality (==) and inequality (!=). Comparisons requiring order (<, <=, >, >=) may be used between any two numbers, unless either operand is complex, in which case they raise runtime exceptions. All these operators return Boolean values (True or False). Beware comparing floating-point numbers for equality, as the online tutorial explains.

Bitwise Operations on Integers

Integers can be interpreted as strings of bits and used with the bitwise operations shown in Table 3-2. Bitwise operators have lower priority than arithmetic operators. Positive integers are conceptually extended by an unbounded string of 0 bits on the left. Negative integers, as they’re held in two’s complement representation, are conceptually extended by an unbounded string of 1 bits on the left.

Sequences in General

Sequences are ordered containers with items that are accessible by indexing and slicing. The built-in len function takes any container as an argument and returns the number of items in the container. The built-in min and max functions take one argument, a nonempty iterable whose items are comparable, and return the smallest and largest items, respectively. You can also call min and max with multiple arguments, in which case they return the smallest and largest arguments, respectively. The built-in sum function takes one argument, an iterable whose items are numbers, and returns the sum of the numbers.

x = [1, 2, 3, 4]
x[1]                  # 2
x[-1]                 # 4

x = [1, 2, 3, 4]
x[1:3]                 # [2, 3]
x[1:]                  # [2, 3, 4]
x[:2]                  # [1, 2]

y = list(range(10))
y[-5:]           #  last five items
[5, 6, 7, 8, 9]
y[::2]           #  every other item
[0, 2, 4, 6, 8]
y[10:0:-2]       #  every other item, in reverse order
[9, 7, 5, 3, 1]
y[:0:-2]         #  every other item, in reverse order (simpler)
[9, 7, 5, 3, 1]
y[::-2]          #  every other item, in reverse order (best)
[9, 7, 5, 3, 1]

Strings

String objects (byte strings, as well as text, AKA Unicode, ones) are immutable: attempting to rebind or delete an item or slice of a string raises an exception. The items of a string object (corresponding to each of the characters in the string) are themselves strings of the same kind, each of length 1—Python has no special data type for “single characters” (except for the items of a bytes object in v3: in that case, indexing produces an int). All slices of a string are strings of the same kind. String objects have many methods, covered in “Methods of String and Bytes Objects”.

Tuples

Tuple objects are immutable: therefore, attempting to rebind or delete an item or slice of a tuple raises an exception. The items of a tuple are arbitrary objects and may be of different types; tuple items may be mutable, but we don’t recommend this practice, as it can be confusing. The slices of a tuple are also tuples. Tuples have no normal (nonspecial) methods, except count and index, with the same meanings as for lists; they do have some of the special methods covered in “Special Methods”.

Lists

List objects are mutable: you may rebind or delete items and slices of a list. Items of a list are arbitrary objects and may be of different types. Slices of a list are lists.

x = [1, 2, 3, 4]
x[1] = 42                # x is now [1, 42, 3, 4]

x = [1, 2, 3, 4]
x[1:3] = [22, 33, 44]     # x is now [1, 22, 33, 44, 4]
x[1:4] = [8, 9]           # x is now [1, 8, 9, 4]

x = [1, 2, 3, 4, 5]
del x[1]                 # x is now [1, 3, 4, 5]
del x[::2]               # x is now [3, 5]

mylist = ['alpha', 'Beta', 'GAMMA']
mylist.sort()                  #  ['Beta', 'GAMMA', 'alpha']
mylist.sort(key=str.lower)     #  ['alpha', 'Beta', 'GAMMA']

Set Membership

The k in S operator checks whether object k is one of the items of set S. It returns True when it is, False when it isn’t. k not in S is like not (k in S).

Set Methods

Set objects provide several methods, as shown in Table 3-4. Nonmutating methods return a result without altering the object to which they apply, and can also be called on instances of frozenset; mutating methods may alter the object to which they apply, and can be called only on instances of set. In Table 3-4, S denotes any set object, S1 any iterable with hashable items (often but not necessarily a set or frozenset), x any hashable object.

All mutating methods of set objects, except pop, return None.

The pop method can be used for destructive iteration on a set, consuming little extra memory. The memory savings make pop usable for a loop on a huge set, when what you want is to “consume” the set in the course of the loop. A potential advantage of a destructive loop such as

while S:
    item = S.pop()
    ...handle item...

in comparison to a nondestructive loop such as

for item in S:
    ...handle item...

is that, in the body of the destructive loop, you’re allowed to modify S (adding and/or removing items), which is not allowed in the nondestructive loop.

Sets also have mutating methods named difference_update, intersection_update, symmetric_difference_update, and update (corresponding to non-mutating method union). Each such mutating method performs the same operation as the corresponding nonmutating method, but it performs the operation in place, altering the set on which you call it, and returns None. The four corresponding non-mutating methods are also accessible with operator syntax: where S2 is a set or frozenset, respectively, S-S2, S&S2, S^S2, and S|S2; the mutating methods are accessible with augmented assignment syntax: respectively, S-=S2, S&=S2, S^=S2, and S|=S2. Note that, when you use operator or augmented assignment syntax, both operands must be sets or frozensets; however, when you call named methods, argument S1 can be any iterable with hashable items, and it works just as if the argument you passed was set(S1).

Dictionary Membership

The k in D operator checks whether object k is a key in dictionary D. It returns True when it is, False when it isn’t. k not in D is like not (k in D).

Indexing a Dictionary

To denote the value in a dictionary D currently associated with key k, use an indexing: D[k]. Indexing with a key that is not present in the dictionary raises an exception. For example:

d = {'x':42, 'y':3.14, 'z':7}
d['x']                           # 42
d['z']                           # 7
d['a']                           # raises KeyError exception

Plain assignment to a dictionary indexed with a key that is not yet in the dictionary (e.g., D[newkey]=value) is a valid operation and adds the key and value as a new item in the dictionary. For instance:

d = {'x':42, 'y':3.14}
d['a'] = 16                    # d is now {'x':42, 'y':3.14, 'a':16}

The del statement, in the form del D[k], removes from the dictionary the item whose key is k. When k is not a key in dictionary D, del D[k] raises a KeyError exception.

Dictionary Methods

Dictionary objects provide several methods, as shown in Table 3-5. Nonmutating methods return a result without altering the object to which they apply, while mutating methods may alter the object to which they apply. In Table 3-5, D and D1 indicate any dictionary object, k any hashable object, and x any object.

The items, keys, and values methods return their resulting lists (in v2; dict_... iterable instances in v3) in arbitrary order.⁷ If you call more than one of these methods without any intervening change to the dictionary, the order of the results is the same for all. The iteritems, iterkeys, and itervalues methods (v2 only) return iterators equivalent to these lists (we cover iterators in “Iterators”).

Iterating on the lists returned by items, keys, or values in v2 carries no such constraint (in v3, for the same purpose, you can explicitly make lists, such as list(D.keys())). Iterating directly on a dictionary D is exactly like iterating on D.iterkeys() in v2, on D.keys() in v3.

v3’s dict_... types are all iterable. dict_items and dict_keys also implement set nonmutating methods and behave much like frozensets; dict_values doesn’t, since, differently from the others (and from sets), it may contain some duplicate items.

The popitem method can be used for destructive iteration on a dictionary. Both items and popitem return dictionary items as key/value pairs. popitem is usable for a loop on a huge dictionary, when what you want is to “consume” the dictionary in the course of the loop.

D.setdefault(k, x) returns the same result as D.get(k, x), but, when k is not a key in D, setdefault also has the side effect of binding D[k] to the value x. (In modern Python, setdefault is rarely used, since type collections.defaultdict, covered in “defaultdict”, offers similar functionality with more elegance and speed.)

The pop method returns the same result as get, but, when k is a key in D, pop also has the side effect of removing D[k] (when x is not specified, and k is not a key in D, get returns None, but pop raises an exception).

The update method can also accept an iterable of key/value pairs, as an alternative argument instead of a mapping, and can accept named arguments instead of—or in addition to—its positional argument; the semantics are the same as for passing such arguments when calling the built-in dict type, as covered in “Dictionaries”.

The if Statement

Often, you need to execute some statements only when some condition holds, or choose statements to execute depending on mutually exclusive conditions. The compound statement if —comprising if, elif, and else clauses—lets you conditionally execute blocks of statements. The syntax for the if statement is:

if expression:
    statement(s)
elif expression:
    statement(s)
elif expression:
    statement(s)
...
else:
    statement(s)

The elif and else clauses are optional. Note that, unlike some languages, Python does not have a “switch” statement. Use if, elif, and else for all conditional processing.

Here’s a typical if statement with all three kinds of clauses (not in optimal style):

if x < 0: print('x is negative')
elif x % 2: print('x is positive and odd')
else: print('x is even and non-negative')

When there are multiple statements in a clause (i.e., the clause controls a block of statements), place the statements on separate logical lines after the line containing the clause’s keyword (known as the header line of the clause), indented rightward from the header line. The block terminates when the indentation returns to that of the clause header (or further left from there). When there is just a single simple statement, as here, it can follow the : on the same logical line as the header, but it can also be on a separate logical line, immediately after the header line and indented rightward from it. Most Python programmers prefer the separate-line style, with four-space indents for the guarded statements. Such a style is more general and more readable, and recommended by PEP 8. So, a generally preferred style is:

if x < 0:
    print('x is negative')
elif x % 2:
    print('x is positive and odd')
else:
    print('x is even and non-negative')

You can use any Python expression as the condition in an if or elif clause. Using an expression this way is known as using it in a Boolean context. In a Boolean context, any value is taken as either true or false. As mentioned earlier, any nonzero number or nonempty container (string, tuple, list, dictionary, set) evaluates as true; zero (of any numeric type), None, and empty containers evaluate as false. To test a value x in a Boolean context, use the following coding style:

if x:

This is the clearest and most Pythonic form. Do not use any of the following:

if x is True:
if x == True:
if bool(x):

There is a crucial difference between saying that an expression returns True (meaning the expression returns the value 1 with the bool type) and saying that an expression evaluates as true (meaning the expression returns any result that is true in a Boolean context). When testing an expression, for example in an if clause, you only care about what it evaluates as, not what, precisely, it returns.

When the if clause’s condition evaluates as true, the statements after the if clause execute, then the entire if statement ends. Otherwise, Python evaluates each elif clause’s condition, in order. The statements after the first elif clause whose condition evaluates as true, if any, execute, and the entire if statement ends. Otherwise, when an else clause exists it is executed, and then the statements after the entire if construct execute.

The while Statement

The while statement repeats execution of a statement or block of statements, controlled by a conditional expression. Here’s the syntax of the while statement:

while expression:
    statement(s)

A while statement can also include an else clause, covered in “The else Clause on Loop Statements”, and break and continue statements, covered in “The break Statement” and “The continue Statement”.

Here’s a typical while statement:

count = 0
while x > 0:
    x //= 2            # truncating division
    count += 1
print('The approximate log2 is', count)

First, Python evaluates expression, which is known as the loop condition. When the condition is false, the while statement ends. When the loop condition is true, the statement or statements that make up the loop body execute. Once the loop body finishes executing, Python evaluates the loop condition again to check whether another iteration should execute. This process continues until the loop condition is false, at which point the while statement ends.

The loop body should contain code that eventually makes the loop condition false; otherwise, the loop never ends (unless the body raises an exception or executes a break statement). A loop within a function’s body also ends if the loop body executes a return statement, since in this case the whole function ends.

The for Statement

The for statement repeats execution of a statement, or block of statements, controlled by an iterable expression. Here’s the syntax of the for statement:

for target in iterable:
    statement(s)

The in keyword is part of the syntax of the for statement; its purpose here is distinct from the in operator, which tests membership.

Here’s a typical for statement:

for letter in 'ciao':
    print('give me a', letter, '...')

A for statement can also include an else clause, covered in “The else Clause on Loop Statements”, and break and continue statements, covered in “The break Statement” and “The continue Statement”. iterable may be any Python expression suitable as an argument to built-in function iter, which returns an iterator object (explained in detail in the next section). In particular, any sequence is iterable. target is normally an identifier that names the control variable of the loop; the for statement successively rebinds this variable to each item of the iterator, in order. The statement or statements that make up the loop body execute once for each item in iterable (unless the loop ends because of an exception or a break or return statement). Note that, since the loop body may contain a break statement to terminate the loop, this is one case in which you may use an unbounded iterable—one that, per se, would never cease yielding items.

You can also have a target with multiple identifiers, like in an unpacking assignment. In this case, the iterator’s items must be iterables, each with exactly as many items as there are identifiers in the target (in v3 only, precisely one of the identifiers can be preceded by a star, in which case the starred item is bound to a list of all items, if any, that were not assigned to other targets). For example, when d is a dictionary, this is a typical way to loop on the items (key/value pairs) in d:

for key, value in d.items():
    if key and value:             # print only true keys and values
        print(key, value)

The items method returns a list (in v2; another kind of iterable, in v3) of key/value pairs, so we use a for loop with two identifiers in the target to unpack each item into key and value. Although components of a target are commonly identifiers, values can be bound to any acceptable LHS expression as covered in “Assignment Statements”:

prototype = [1, 'placemarker', 3]
for prototype[1] in 'xyz': print(prototype)
# prints [1, 'x', 3], then [1, 'y', 3], then [1, 'z', 3]

The control target variable(s) may be rebound in the loop body, but get rebound again to the next item in the iterator at the next iteration of the loop. The loop body does not execute at all if the iterator yields no items. In this case, the control variable is not bound or rebound in any way by the for statement. However, if the iterator yields at least one item, then, when the loop statement terminates, the control variable remains bound to the last value to which the loop statement has bound it. The following code is therefore correct only when someseq is not empty:

for x in someseq:
    process(x)
print('Last item processed was', x)

for x in c:
    statement(s)

_temporary_iterator = iter(c)
while True:
    try: x = next(_temporary_iterator)
    except StopIteration: break
    statement(s)

for i in range(n):
    statement(s)

>>> print(range(1, 5))
[1, 2, 3, 4]
>>> print(xrange(1, 5))
xrange(1, 5)

[ expression for target in iterable lc-clauses ]

for target in iterable
if expression

result1 = [x+1 for x in some_sequence]

result2 = []
for x in some_sequence:
    result2.append(x+1)

result3 = [x+1 for x in some_sequence if x>23]

result4 = []
for x in some_sequence:
    if x>23:
        result4.append(x+1)

result5 = [x for sublist in listoflists for x in sublist]

result6 = []
for sublist in listoflists:
    for x in sublist:
        result6.append(x)

s = {n//2 for n in range(10)}
print(sorted(s))  # prints: [0, 1, 2, 3, 4]

d = {n:n//2 for n in range(5)}
print(d)  # prints: {0:0, 1:0, 2:1, 3:1, 4:2] or other order

The break Statement

The break statement is allowed only within a loop body. When break executes, the loop terminates. If a loop nests inside other loops, a break terminates only the innermost nested loop. In practice, a break is usually within a clause of an if in the loop body, so that break executes conditionally.

One common use of break is in the implementation of a loop that decides whether it should keep looping only in the middle of each loop iteration (what Knuth called the “loop and a half” structure in his seminal 1974 paper, in which he also first proposed using “devices like indentation, rather than delimiters” to express program structure—just as Python does). For example:

while True:           # this loop can never terminate naturally
    x = get_next()
    y = preprocess(x)
    if not keep_looping(x, y): break
    process(x, y)

The continue Statement

The continue statement can exist only within a loop body. When continue executes, the current iteration of the loop body terminates, and execution continues with the next iteration of the loop. In practice, a continue is usually within a clause of an if in the loop body, so that continue executes conditionally.

Sometimes, a continue statement can take the place of nested if statements within a loop. For example:

for x in some_container:
    if not seems_ok(x): continue
    lowbound, highbound = bounds_to_test()
    if x<lowbound or x>=highbound: continue
    pre_process(x)
    if final_check(x):
        do_processing(x)

This equivalent code does conditional processing without continue:

for x in some_container:
    if seems_ok(x):
        lowbound, highbound = bounds_to_test()
        if lowbound <= x < highbound:
            pre_process(x)
            if final_check(x):
                do_processing(x)

The else Clause on Loop Statements

while and for statements may optionally have a trailing else clause. The statement or block under that else executes when the loop terminates naturally (at the end of the for iterator, or when the while loop condition becomes false), but not when the loop terminates prematurely (via break, return, or an exception). When a loop contains one or more break statements, you often need to check whether the loop terminates naturally or prematurely. You can use an else clause on the loop for this purpose:

for x in some_container:
    if is_ok(x): break  # item x is satisfactory, terminate loop
else:
    print('Beware: no satisfactory item was found in container')
    x = None

The pass Statement

The body of a Python compound statement cannot be empty; it must always contain at least one statement. You can use a pass statement, which performs no action, as an explicit placeholder when a statement is syntactically required but you have nothing to do. Here’s an example of using pass in a conditional statement as a part of somewhat convoluted logic to test mutually exclusive conditions:

if condition1(x):
    process1(x)
elif x>23 or condition2(x) and x<5:
    pass  # nothing to be done in this case
elif condition3(x):
    process3(x)
else:
    process_default(x)

The try and raise Statements

Python supports exception handling with the try statement, which includes try, except, finally, and else clauses. Your code can explicitly raise an exception with the raise statement. As we discuss in detail in “Exception Propagation”, when code raises an exception, normal control flow of the program stops, and Python looks for a suitable exception handler.

The with Statement

A with statement can be a much more readable and useful alternative to the try/finally statement. We discuss it in detail in “The with Statement and Context Managers”.

The def Statement

The def statement is the most common way to define a function. def is a single-clause compound statement with the following syntax:

def function_name(parameters):
    statement(s)

function_name is an identifier. It is a variable that gets bound (or rebound) to the function object when def executes.

parameters is an optional list of identifiers that get bound to the values supplied when you call the function, which we refer to as arguments to distinguish between definitions (parameters) and calls (arguments). In the simplest case a function doesn’t have any parameters, which means the function won’t accept any arguments when you call it. In this case, the function definition has empty parentheses after function_name, as must all calls to the function.

When a function does accept arguments, parameters contains one or more identifiers, separated by commas (,). In this case, each call to the function supplies values, known as arguments, corresponding to the parameters listed in the function definition. The parameters are local variables of the function (as we’ll discuss later in this section), and each call to the function binds these local variables in the function namespace to the corresponding values that the caller, or the function definition, supplies as arguments.

The nonempty block of statements, known as the function body, does not execute when the def statement executes. Rather, the function body executes later, each time you call the function. The function body can contain zero or more occurrences of the return statement, as we’ll discuss shortly.

Here’s an example of a simple function that returns a value that is twice the value passed to it each time it’s called:

def twice(x):
    return x*2

Note that the argument can be anything that can be multiplied by two, so the function could be called with a number, string, list, or tuple as an argument, and it would return, in each case, a new value of the same type as the argument.

Parameters

Parameters (more pedantically, formal parameters) name, and sometimes specify a default value for, the values passed into a function call. Each of the names is bound inside a new local namespace every time the function is called; this namespace is destroyed when the function returns or otherwise exits (values returned by the function may be kept alive by the caller binding the function result). Parameters that are just identifiers indicate positional parameters (also known as mandatory parameters). Each call to the function must supply a corresponding value (argument) for each mandatory (positional) parameter.

In the comma-separated list of parameters, zero or more positional parameters may be followed by zero or more named parameters (also known as optional parameters, and sometimes—confusingly, since they do not involve any keyword—as keyword parameters), each with the syntax:

identifier=expression

The def statement evaluates each such expression and saves a reference to the expression’s value, known as the default value for the parameter, among the attributes of the function object. When a function call does not supply an argument corresponding to a named parameter, the call binds the parameter’s identifier to its default value for that execution of the function.

Note that Python computes each default value precisely once, when the def statement evaluates, not each time the resulting function gets called. In particular, this means that exactly the same object, the default value, gets bound to the named parameter whenever the caller does not supply a corresponding argument.

Things can be tricky when the default value is a mutable object and the function body alters the parameter. For example:

def f(x, y=[]):
    y.append(x)
    return y, id(y)
print(f(23))                # prints: ([23], 4302354376)
print(f(42))                # prints: ([23, 42], 4302354376)

The second print prints [23, 42] because the first call to f altered the default value of y, originally an empty list [], by appending 23 to it. The id values confirm that both calls return the same object. If you want y to be bound to a new empty list object each time you call f with a single argument, use the following idiom instead:

def f(x, y=None):
    if y is None: y = []
    y.append(x)
    return y, id(y)
print(f(23))               # prints: ([23], 4302354376)
prinf(f(42))               # prints: ([42], 4302180040)

Of course, there are cases in which you explicitly want to alter a parameter’s default value, most often for caching purposes, as in the following example:

def cached_compute(x, _cache={}):
    if x not in _cache:
        _cache[x] = costly_computation(x)
    return _cache[x]

Such caching behavior (also known as memoization), however, is usually best obtained by decorating the underlying function with functools.lru_cache, covered in Table 7-4.

At the end of the parameters, you may optionally use either or both of the special forms *args and **kwds.⁸ There is nothing special about the names—you can use any identifier you want in each special form; args and kwds or kwargs, as well as a and k, are just popular identifiers to use in these roles. If both forms are present, the form with two asterisks must come second.

*args specifies that any extra positional arguments to a call (i.e., those positional arguments not explicitly specified in the signature, defined in “Function signatures” below) get collected into a (possibly empty) tuple, and bound in the call namespace to the name args. Similarly **kwds specifies that any extra named arguments (i.e., those named arguments not explicitly specified in the signature, defined in “Function signatures” below) get collected into a (possibly empty) dictionary⁹ whose items are the names and values of those arguments, and bound to the name kwds in the function call namespace (we cover positional and named arguments in “Calling Functions”).

For example, here’s a function that accepts any number of positional arguments and returns their sum (also showing the use of an identifier other than *args):

def sum_args(*numbers):
    return sum(numbers)
print(sum_args(23, 42))          # prints: 65