Remember those set functions we wrote in Chapters 16 and 18? Here’s what they look like brought back to life as a Python class. The following example (the file setwrapper.py) implements a new set object type by moving some of the set functions to methods and adding some basic operator overloading. For the most part, this class just wraps a Python list with extra set operations. But because it’s a class, it also supports multiple instances and customization by inheritance in subclasses. Unlike our earlier functions, using classes here allows us to make multiple self-contained set objects with preset data and behavior, rather than passing lists into functions manually:

class Set:
   def __init__(self, value = []):    # Constructor
       self.data = []                 # Manages a list
       self.concat(value)

   def intersect(self, other):        # other is any sequence
       res = []                       # self is the subject
       for x in self.data:
           if x in other:             # Pick common items
               res.append(x)
       return Set(res)                # Return a new Set

   def union(self, other):            # other is any sequence
       res = self.data[:]             # Copy of my list
       for x in other:                # Add items in other
           if not x in res:
               res.append(x)
       return Set(res)

   def concat(self, value):           # value: list, Set...
       for x in value:                # Removes duplicates
          if not x in self.data:
               self.data.append(x)

   def __len__(self):          return len(self.data)            # len(self)
   def __getitem__(self, key): return self.data[key]            # self[i]
   def __and__(self, other):   return self.intersect(other)     # self & other
   def __or__(self, other):    return self.union(other)         # self | other
   def __repr__(self):         return 'Set:' + repr(self.data)  # print()

To use this class, we make instances, call methods, and run defined operators as usual:

x = Set([1, 3, 5, 7])
print(x.union(Set([1, 4, 7])))       # prints Set:[1, 3, 5, 7, 4]
print(x | Set([1, 4, 6]))            # prints Set:[1, 3, 5, 7, 4, 6]

Overloading operations such as indexing enables instances of our Set class to masquerade as real lists. Because you will interact with and extend this class in an exercise at the end of this chapter, I won’t say much more about this code until Appendix B.

Beginning with Python 2.2, all the built-in types in the language can now be subclassed directly. Type-conversion functions such as list, str, dict, and tuple have become built-in type names—although transparent to your script, a type-conversion call (e.g., list('spam')) is now really an invocation of a type’s object constructor.

This change allows you to customize or extend the behavior of built-in types with user-defined class statements: simply subclass the new type names to customize them. Instances of your type subclasses can be used anywhere that the original built-in type can appear. For example, suppose you have trouble getting used to the fact that Python list offsets begin at 0 instead of 1. Not to worry—you can always code your own subclass that customizes this core behavior of lists. The file typesubclass.py shows how:

# Subclass built-in list type/class
# Map 1..N to 0..N-1; call back to built-in version.

class MyList(list):
    def __getitem__(self, offset):
        print('(indexing %s at %s)' % (self, offset))
        return list.__getitem__(self, offset - 1)

if __name__ == '__main__':
    print(list('abc'))
    x = MyList('abc')               # __init__ inherited from list
    print(x)                        # __repr__ inherited from list

    print(x[1])                     # MyList.__getitem__
    print(x[3])                     # Customizes list superclass method

    x.append('spam'), print(x)      # Attributes from list superclass
    x.reverse();      print(x)

In this file, the MyList subclass extends the built-in list’s __getitem__ indexing method only to map indexes 1 to N back to the required 0 to N−1. All it really does is decrement the submitted index and call back to the superclass’s version of indexing, but it’s enough to do the trick:

% python typesubclass.py
['a', 'b', 'c']
['a', 'b', 'c']
(indexing ['a', 'b', 'c'] at 1)
a
(indexing ['a', 'b', 'c'] at 3)
c
['a', 'b', 'c', 'spam']
['spam', 'c', 'b', 'a']

This output also includes tracing text the class prints on indexing. Of course, whether changing indexing this way is a good idea in general is another issue—users of your MyList class may very well be confused by such a core departure from Python sequence behavior. The ability to customize built-in types this way can be a powerful asset, though.

For instance, this coding pattern gives rise to an alternative way to code a set—as a subclass of the built-in list type, rather than a standalone class that manages an embedded list object, as shown earlier in this section. As we learned in Chapter 5, Python today comes with a powerful built-in set object, along with literal and comprehension syntax for making new sets. Coding one yourself, though, is still a great way to learn about type subclassing in general.

The following class, coded in the file setsubclass.py, customizes lists to add just methods and operators related to set processing. Because all other behavior is inherited from the built-in list superclass, this makes for a shorter and simpler alternative:

class Set(list):
    def __init__(self, value = []):      # Constructor
        list.__init__([])                # Customizes list
        self.concat(value)               # Copies mutable defaults

    def intersect(self, other):          # other is any sequence
        res = []                         # self is the subject
        for x in self:
            if x in other:               # Pick common items
                res.append(x)
        return Set(res)                  # Return a new Set

    def union(self, other):              # other is any sequence
        res = Set(self)                  # Copy me and my list
        res.concat(other)
        return res

    def concat(self, value):             # value: list, Set . . .
        for x in value:                  # Removes duplicates
            if not x in self:
                self.append(x)

    def __and__(self, other): return self.intersect(other)
    def __or__(self, other):  return self.union(other)
    def __repr__(self):       return 'Set:' + list.__repr__(self)

if __name__ == '__main__':
    x = Set([1,3,5,7])
    y = Set([2,1,4,5,6])
    print(x, y, len(x))
    print(x.intersect(y), y.union(x))
    print(x & y, x | y)
    x.reverse(); print(x)

Here is the output of the self-test code at the end of this file. Because subclassing core types is an advanced feature, I’ll omit further details here, but I invite you to trace through these results in the code to study its behavior:

% python setsubclass.py
Set:[1, 3, 5, 7] Set:[2, 1, 4, 5, 6] 4
Set:[1, 5] Set:[2, 1, 4, 5, 6, 3, 7]
Set:[1, 5] Set:[1, 3, 5, 7, 2, 4, 6]
Set:[7, 5, 3, 1]

There are more efficient ways to implement sets with dictionaries in Python, which replace the linear scans in the set implementations shown here with dictionary index operations (hashing) and so run much quicker. (For more details, see Programming Python.) If you’re interested in sets, also take another look at the set object type we explored in Chapter 5; this type provides extensive set operations as built-in tools. Set implementations are fun to experiment with, but they are no longer strictly required in Python today.

For another type subclassing example, see the implementation of the bool type in Python 2.3 and later. As mentioned earlier in the book, bool is a subclass of int with two instances (True and False) that behave like the integers 1 and 0 but inherit custom string-representation methods that display their names.

In new-style classes, the distinction between type and class has vanished entirely. Classes themselves are types: the type object generates classes as its instances, and classes generate instances of their type. If fact, there is no real difference between built-in types like lists and strings and user-defined types coded as classes. This is why we can subclass built-in types, as shown earlier in this chapter—because subclassing a built-in type such as list qualifies a class as new-style, it becomes a user-defined type.

Besides allowing us to subclass built-in types, one of the contexts where this becomes most obvious is when we do explicit type testing. With Python 2.6’s classic classes, the type of a class instance is a generic “instance,” but the types of built-in objects are more specific:

C:misc> c:python26python
>>> class C: pass                       # Classic classes in 2.6
...
>>> I = C()
>>> type(I)                             # Instances are made from classes
<type 'instance'>
>>> I.__class__
<class __main__.C at 0x025085A0>

>>> type(C)                             # But classes are not the same as types
<type 'classobj'>
>>> C.__class__
AttributeError: class C has no attribute '__class__'

>>> type([1, 2, 3])
<type 'list'>
>>> type(list)
<type 'type'>
>>> list.__class__
<type 'type'>

But with new-style classes in 2.6, the type of a class instance is the class it’s created from, since classes are simply user-defined types—the type of an instance is its class, and the type of a user-defined class is the same as the type of a built-in object type. Classes have a __class__ attribute now, too, because they are instances of type:

C:misc> c:python26python
>>> class C(object): pass               # New-style classes in 2.6
...
>>> I = C()
>>> type(I)                             # Type of instance is class it's made from
<class '__main__.C'>
>>> I.__class__
<class '__main__.C'>

>>> type(C)                             # Classes are user-defined types
<type 'type'>
>>> C.__class__
<type 'type'>

>>> type([1, 2, 3])                     # Built-in types work the same way
<type 'list'>
>>> type(list)
<type 'type'>
>>> list.__class__
<type 'type'>

The same is true for all classes in Python 3.0, since all classes are automatically new-style, even if they have no explicit superclasses. In fact, the distinction between built-in types and user-defined class types melts away altogether in 3.0:

C:misc> c:python30python
>>> class C: pass                       # All classes are new-style in 3.0
...
>>> I = C()
>>> type(I)                             # Type of instance is class it's made from
<class '__main__.C'>
>>> I.__class__
<class '__main__.C'>

>>> type(C)                             # Class is a type, and type is a class
<class 'type'>
>>> C.__class__
<class 'type'>

>>> type([1, 2, 3])                     # Classes and built-in types work the same
<class 'list'>
>>> type(list)
<class 'type'>
>>> list.__class__
<class 'type'>

As you can see, in 3.0 classes are types, but types are also classes. Technically, each class is generated by a metaclass—a class that is normally either type itself, or a subclass of it customized to augment or manage generated classes. Besides impacting code that does type testing, this turns out to be an important hook for tool developers. We’ll talk more about metaclasses later in this chapter, and again in more detail in Chapter 39.

C:misc> c:python30python
>>> class C: pass
...
>>> class D: pass
...
>>> c = C()
>>> d = D()
>>> type(c) == type(d)                 # 3.0: compares the instances' classes
False

>>> type(c), type(d)
(<class '__main__.C'>, <class '__main__.D'>)
>>> c.__class__, d.__class__
(<class '__main__.C'>, <class '__main__.D'>)

>>> c1, c2 = C(), C()
>>> type(c1) == type(c2)
True

C:misc> c:python26python
>>> class C: pass
...
>>> class D: pass
...
>>> c = C()
>>> d = D()
>>> type(c) == type(d)                 # 2.6: all instances are same type
True
>>> c.__class__ == d.__class__         # Must compare classes explicitly
False

>>> type(c), type(d)
(<type 'instance'>, <type 'instance'>)
>>> c.__class__, d.__class__
(<class __main__.C at 0x024585A0>, <class __main__.D at 0x024588D0>)

C:misc> c:python26python
>>> class C(object): pass
...
>>> class D(object): pass
...
>>> c = C()
>>> d = D()
>>> type(c) == type(d)                 # 2.6 new-style: same as all in 3.0
False

>>> type(c), type(d)
(<class '__main__.C'>, <class '__main__.D'>)
>>> c.__class__, d.__class__
(<class '__main__.C'>, <class '__main__.D'>)

>>> class C: pass
...
>>> X = C()

>>> type(X)                           # Type is now class instance was created from
<class '__main__.C'>
>>> type(C)
<class 'type'>

>>> isinstance(X, object)
True
>>> isinstance(C, object)             # Classes always inherit from object
True

>>> type('spam')
<class 'str'>
>>> type(str)
<class 'type'>

>>> isinstance('spam', object)        # Same for  built-in types (classes)
True
>>> isinstance(str, object)
True

>>> type(type)                        # All classes are types, and vice versa
<class 'type'>
>>> type(object)
<class 'type'>

>>> isinstance(type, object)          # All classes derive from object, even type
True
>>> isinstance(object, type)          # Types make classes, and type is a class
True
>>> type is object
False

One of the most visible changes in new-style classes is their slightly different inheritance search procedures for the so-called diamond pattern of multiple inheritance trees, where more than one superclass leads to the same higher superclass further above. The diamond pattern is an advanced design concept, is coded only rarely in Python practice, and has not been discussed in this book, so we won’t dwell on this topic in depth.

In short, though, with classic classes, the inheritance search procedure is strictly depth first, and then left to right—Python climbs all the way to the top, hugging the left side of the tree, before it backs up and begins to look further to the right. In new-style classes, the search is more breadth-first in such cases—Python first looks in any superclasses to the right of the first one searched before ascending all the way to the common superclass at the top. In other words, the search proceeds across by levels before moving up. The search algorithm is a bit more complex than this, but this is as much as most programmers need to know.

Because of this change, lower superclasses can overload attributes of higher superclasses, regardless of the sort of multiple inheritance trees they are mixed into. Moreover, the new-style search rule avoids visiting the same superclass more than once when it is accessible from multiple subclasses.

>>> class A:
        attr = 1         # Classic (Python 2.6)

>>> class B(A):          # B and C both lead to A
        pass

>>> class C(A):
        attr = 2

>>> class D(B, C):
        pass             # Tries A before C

>>> x = D()
>>> x.attr               # Searches x, D, B, A
1

>>> class A(object):
        attr = 1         # New-style ("object" not required in 3.0)

>>> class B(A):
        pass

>>> class C(A):
        attr = 2

>>> class D(B, C):
        pass             # Tries C before A

>>> x = D()
>>> x.attr               # Searches x, D, B, C
2

>>> class A:
        attr = 1         # Classic

>>> class B(A):
        pass

>>> class C(A):
        attr = 2

>>> class D(B, C):
        attr = C.attr    # Choose C, to the right

>>> x = D()
>>> x.attr               # Works like new-style (all 3.0)
2

>>> class A(object):
        attr = 1         # New-style

>>> class B(A):
        pass

>>> class C(A):
        attr = 2

>>> class D(B, C):
        attr = B.attr    # Choose A.attr, above

>>> x = D()
>>> x.attr               # Works like classic (default 2.6)
1

>>> class A:
        def meth(s): print('A.meth')

>>> class C(A):
        def meth(s): print('C.meth')

>>> class B(A):
        pass

>>> class D(B, C): pass            # Use default search order
>>> x = D()                        # Will vary per class type
>>> x.meth()                       # Defaults to classic order in 2.6
A.meth

>>> class D(B, C): meth = C.meth   # Pick C's method: new-style (and 3.0)
>>> x = D()
>>> x.meth()
C.meth

>>> class D(B, C): meth = B.meth   # Pick B's method: classic
>>> x = D()
>>> x.meth()
A.meth

class D(B, C):
    def meth(self):                # Redefine lower
        ...
        C.meth(self)               # Pick C's method by calling

By assigning a sequence of string attribute names to a special __slots__ class attribute, it is possible for a new-style class to both limit the set of legal attributes that instances of the class will have and optimize memory and speed performance.

This special attribute is typically set by assigning a sequence of string names to the variable __slots__ at the top level of a class statement: only those names in the __slots__ list can be assigned as instance attributes. However, like all names in Python, instance attribute names must still be assigned before they can be referenced, even if they’re listed in __slots__. For example:

>>> class limiter(object):
...     __slots__ = ['age', 'name', 'job']
...
>>> x = limiter()
>>> x.age                                           # Must assign before use
AttributeError: age

>>> x.age = 40
>>> x.age
40
>>> x.ape = 1000                                    # Illegal: not in __slots__
AttributeError: 'limiter' object has no attribute 'ape'

Slots are something of a break with Python’s dynamic nature, which dictates that any name may be created by assignment. However, this feature is envisioned as both a way to catch “typo” errors like this (assignments to illegal attribute names not in __slots__ are detected), as well as an optimization mechanism. Allocating a namespace dictionary for every instance object can become expensive in terms of memory if many instances are created and only a few attributes are required. To save space and speed execution (to a degree that can vary per program), instead of allocating a dictionary for each instance, slot attributes are stored sequentially for quicker lookup.

>>> class C:
...     __slots__ = ['a', 'b']           # __slots__ means no __dict__ by default
...
>>> X = C()
>>> X.a = 1
>>> X.a
1
>>> X.__dict__
AttributeError: 'C' object has no attribute '__dict__'
>>> getattr(X, 'a')
1
>>> setattr(X, 'b', 2)                   # But getattr() and setattr() still work
>>> X.b
2
>>> 'a' in dir(X)                        # And dir() finds slot attributes too
True
>>> 'b' in dir(X)
True

>>> class D:
...     __slots__ = ['a', 'b']
...     def __init__(self): self.d = 4   # Cannot add new names if no __dict__
...
>>> X = D()
AttributeError: 'D' object has no attribute 'd'

>>> class D:
...     __slots__ = ['a', 'b', '__dict__']    # List __dict__ to include one too
...     c = 3                                 # Class attrs work normally
...     def __init__(self): self.d = 4        # d put in __dict__, a in __slots__
...
>>> X = D()
>>> X.d
4
>>> X.__dict__                   # Some objects have both __dict__ and __slots__
{'d': 4}                         # getattr() can fetch either type of attr
>>> X.__slots__
['a', 'b', '__dict__']
>>> X.c
3
>>> X.a                          # All instance attrs undefined until assigned
AttributeError: a
>>> X.a = 1
>>> getattr(X, 'a',), getattr(X, 'c'), getattr(X, 'd')
(1, 3, 4)

>>> for attr in list(X.__dict__) + X.__slots__:
...     print(attr, '=>', getattr(X, attr))

d => 4
a => 1
b => 2
__dict__ => {'d': 4}

>>> for attr in list(getattr(X, '__dict__', [])) + getattr(X, '__slots__', []):
...     print(attr, '=>', getattr(X, attr))


d => 4
a => 1
b => 2
__dict__ => {'d': 4}

>>> class E:
...     __slots__ = ['c', 'd']            # Superclass has slots
...
>>> class D(E):
...     __slots__ = ['a', '__dict__']     # So does its subclass
...
>>> X = D()
>>> X.a = 1; X.b = 2; X.c = 3             # The instance is the union
>>> X.a, X.c
(1, 3)

>>> E.__slots__                           # But slots are not concatenated
['c', 'd']
>>> D.__slots__
['a', '__dict__']
>>> X.__slots__                           # Instance inherits *lowest* __slots__
['a', '__dict__']
>>> X.__dict__                            # And has its own an attr dict
{'b': 2}

>>> for attr in list(getattr(X, '__dict__', [])) + getattr(X, '__slots__', []):
...     print(attr, '=>', getattr(X, attr))
...
b => 2                                    # Superclass slots missed!
a => 1
__dict__ => {'b': 2}

>>> dir(X)                                # dir() includes all slot names
[...many names omitted... 'a', 'b', 'c', 'd']

A mechanism known as properties provides another way for new-style classes to define automatically called methods for access or assignment to instance attributes. At least for specific attributes, this feature is an alternative to many current uses of the __getattr__ and __setattr__ overloading methods we studied in Chapter 29. Properties have a similar effect to these two methods, but they incur an extra method call only for accesses to names that require dynamic computation. Properties (and slots) are based on a new notion of attribute descriptors, which is too advanced for us to cover here.

In short, a property is a type of object assigned to a class attribute name. A property is generated by calling the property built-in with three methods (handlers for get, set, and delete operations), as well as a docstring; if any argument is passed as None or omitted, that operation is not supported. Properties are typically assigned at the top level of a class statement [e.g., name = property(...)]. When thus assigned, accesses to the class attribute itself (e.g., obj.name) are automatically routed to one of the accessor methods passed into the property. For example, the __getattr__ method allows classes to intercept undefined attribute references:

>>> class classic:
...     def __getattr__(self, name):
...         if name == 'age':
...             return 40
...         else:
...             raise AttributeError
...
>>> x = classic()
>>> x.age                                         # Runs __getattr__
40
>>> x.name                                        # Runs __getattr__
AttributeError

Here is the same example, coded with properties instead (note that properties are available for all classes but require the new-style object derivation in 2.6 to work properly for intercepting attribute assignments):

>>> class newprops(object):
...     def getage(self):
...         return 40
...     age = property(getage, None, None, None)  # get, set, del, docs
...
>>> x = newprops()
>>> x.age                                         # Runs getage
40
>>> x.name                                        # Normal fetch
AttributeError: newprops instance has no attribute 'name'

For some coding tasks, properties can be less complex and quicker to run than the traditional techniques. For example, when we add attribute assignment support, properties become more attractive—there’s less code to type, and no extra method calls are incurred for assignments to attributes we don’t wish to compute dynamically:

>>> class newprops(object):
...     def getage(self):
...         return 40
...     def setage(self, value):
...         print('set age:', value)
...         self._age = value
...     age = property(getage, setage, None, None)
...
>>> x = newprops()
>>> x.age                                         # Runs getage
40
>>> x.age = 42                                    # Runs setage
set age: 42
>>> x._age                                        # Normal fetch; no getage call
42
>>> x.job = 'trainer'                             # Normal assign; no setage call
>>> x.job                                         # Normal fetch; no getage call
'trainer'

The equivalent classic class incurs extra method calls for assignments to attributes not being managed and needs to route attribute assignments through the attribute dictionary (or, for new-style classes, to the object superclass’s __setattr__) to avoid loops:

>>> class classic:
...     def __getattr__(self, name):              # On undefined reference
...         if name == 'age':
...             return 40
...         else:
...             raise AttributeError
...     def __setattr__(self, name, value):       # On all assignments
...         print('set:', name, value)
...         if name == 'age':
...             self.__dict__['_age'] = value
...         else:
...             self.__dict__[name] = value
...
>>> x = classic()
>>> x.age                                         # Runs __getattr__
40
>>> x.age = 41                                    # Runs __setattr__
set: age 41
>>> x._age                                        # Defined: no __getattr__ call
41
>>> x.job = 'trainer'                             # Runs __setattr__ again
set: job trainer
>>> x.job                                         # Defined: no __getattr__ call
'trainer'

Properties seem like a win for this simple example. However, some applications of __getattr__ and __setattr__ may still require more dynamic or generic interfaces than properties directly provide. For example, in many cases, the set of attributes to be supported cannot be determined when the class is coded, and may not even exist in any tangible form (e.g., when delegating arbitrary method references to a wrapped/embedded object generically). In such cases, a generic __getattr__ or a __setattr__ attribute handler with a passed-in attribute name may be preferable. Because such generic handlers can also handle simpler cases, properties are often an optional extension.

For more details on both options, stay tuned for Chapter 37 in the final part of this book. As we’ll see there, it’s also possible to code properties using function decorator syntax, a topic introduced later in this chapter.

The __getattribute__ method, available for new-style classes only, allows a class to intercept all attribute references, not just undefined references, like __getattr__. It is also somewhat trickier to use than __getattr__: it is prone to loops, much like __setattr__, but in different ways.

In addition to properties and operator overloading methods, Python supports the notion of attribute descriptors—classes with __get__ and __set__ methods, assigned to class attributes and inherited by instances, that intercept read and write accesses to specific attributes. Descriptors are in a sense a more general form of properties; in fact, properties are a simplified way to define a specific type of descriptor, one that runs functions on access. Descriptors are also used to implement the slots feature we met earlier.

Because properties, __getattribute__, and descriptors are somewhat advanced topics, we’ll defer the rest of their coverage, as well as more on properties, to Chapter 37 in the final part of this book.

Most of the changes and feature additions of new-style classes integrate with the notion of subclassable types mentioned earlier in this chapter, because subclassable types and new-style classes were introduced in conjunction with a merging of the type/class dichotomy in Python 2.2 and beyond. As we’ve seen, in 3.0, this merging is complete: classes are now types, and types are classes.

Along with these changes, Python also grew a more coherent protocol for coding metaclasses, which are classes that subclass the type object and intercept class creation calls. As such, they provide a well-defined hook for management and augmentation of class objects. They are also an advanced topic that is optional for most Python programmers, so we’ll postpone further details here. We’ll meet metaclasses briefly later in this chapter in conjunction with class decorators, and we’ll explore them in full detail in Chapter 39, in the final part of this book.

As we’ve learned, a class method is normally passed an instance object in its first argument, to serve as the implied subject of the method call. Today, though, there are two ways to modify this model. Before I explain what they are, I should explain why this might matter to you.

Sometimes, programs need to process data associated with classes instead of instances. Consider keeping track of the number of instances created from a class, or maintaining a list of all of a class’s instances that are currently in memory. This type of information and its processing are associated with the class rather than its instances. That is, the information is usually stored on the class itself and processed in the absence of any instance.

For such tasks, simple functions coded outside a class can often suffice—because they can access class attributes through the class name, they have access to class data and never require access to an instance. However, to better associate such code with a class, and to allow such processing to be customized with inheritance as usual, it would be better to code these types of functions inside the class itself. To make this work, we need methods in a class that are not passed, and do not expect, a self instance argument.

Python supports such goals with the notion of static methods—simple functions with no self argument that are nested in a class and are designed to work on class attributes instead of instance attributes. Static methods never receive an automatic self argument, whether called through a class or an instance. They usually keep track of information that spans all instances, rather than providing behavior for instances.

Although less commonly used, Python also supports the notion of class methods—methods of a class that are passed a class object in their first argument instead of an instance, regardless of whether they are called through an instance or a class. Such methods can access class data through their self class argument even if called through an instance. Normal methods (now known in formal circles as instance methods) still receive a subject instance when called; static and class methods do not.

The concept of static methods is the same in both Python 2.6 and 3.0, but its implementation requirements have evolved somewhat in Python 3.0. Since this book covers both versions, I need to explain the differences in the two underlying models before we get to the code.

Really, we already began this story in the preceding chapter, when we explored the notion of unbound methods. Recall that both Python 2.6 and 3.0 always pass an instance to a method that is called through an instance. However, Python 3.0 treats methods fetched directly from a class differently than 2.6:

In other words, Python 2.6 class methods always require an instance to be passed in, whether they are called through an instance or a class. By contrast, in Python 3.0 we are required to pass an instance to a method only if the method expects one—methods without a self instance argument can be called through the class without passing an instance. That is, 3.0 allows simple functions in a class, as long as they do not expect and are not passed an instance argument. The net effect is that:

To illustrate, suppose we want to use class attributes to count how many instances are generated from a class. The following file, spam.py, makes a first attempt—its class has a counter stored as a class attribute, a constructor that bumps up the counter by one each time a new instance is created, and a method that displays the counter’s value. Remember, class attributes are shared by all instances. Therefore, storing the counter in the class object itself ensures that it effectively spans all instances:

class Spam:
    numInstances = 0
    def __init__(self):
        Spam.numInstances = Spam.numInstances + 1
    def printNumInstances():
        print("Number of instances created: ", Spam.numInstances)

The printNumInstances method is designed to process class data, not instance data—it’s about all the instances, not any one in particular. Because of that, we want to be able to call it without having to pass an instance. Indeed, we don’t want to make an instance to fetch the number of instances, because this would change the number of instances we’re trying to fetch! In other words, we want a self-less “static” method.

Whether this code works or not, though, depends on which Python you use, and which way you call the method—through the class or through an instance. In 2.6 (and 2.X in general), calls to a self-less method function through both the class and instances fail (I’ve omitted some error text here for space):

C:misc> c:python26python
>>> from spam import Spam
>>> a = Spam()                   # Cannot call unbound class methods in 2.6
>>> b = Spam()                   # Methods expect a self object by default
>>> c = Spam()

>>> Spam.printNumInstances()
TypeError: unbound method printNumInstances() must be called with Spam instance
as first argument (got nothing instead)
>>> a.printNumInstances()
TypeError: printNumInstances() takes no arguments (1 given)

The problem here is that unbound instance methods aren’t exactly the same as simple functions in 2.6. Even though there are no arguments in the def header, the method still expects an instance to be passed in when it’s called, because the function is associated with a class. In Python 3.0 (and later 3.X releases), calls to self-less methods made through classes work, but calls from instances fail:

C:misc> c:python30python
>>> from spam import Spam
>>> a = Spam()                       # Can call functions in class in 3.0
>>> b = Spam()                       # Calls through instances still pass a self
>>> c = Spam()

>>> Spam.printNumInstances()         # Differs in 3.0
Number of instances created:  3
>>> a.printNumInstances()
TypeError: printNumInstances() takes no arguments (1 given)

That is, calls to instance-less methods like printNumInstances made through the class fail in Python 2.6 but work in Python 3.0. On the other hand, calls made through an instance fail in both Pythons, because an instance is automatically passed to a method that does not have an argument to receive it:

Spam.printNumInstances()             # Fails in 2.6, works in 3.0
instance.printNumInstances()         # Fails in both 2.6 and 3.0

If you’re able to use 3.0 and stick with calling self-less methods through classes only, you already have a static method feature. However, to allow self-less methods to be called through classes in 2.6 and through instances in both 2.6 and 3.0, you need to either adopt other designs or be able to somehow mark such methods as special. Let’s look at both options in turn.

Short of marking a self-less method as special, there are a few different coding structures that can be tried. If you want to call functions that access class members without an instance, perhaps the simplest idea is to just make them simple functions outside the class, not class methods. This way, an instance isn’t expected in the call. For example, the following mutation of spam.py works the same in Python 3.0 and 2.6 (albeit displaying extra parentheses in 2.6 for its print statement):

def printNumInstances():
    print("Number of instances created: ", Spam.numInstances)

class Spam:
    numInstances = 0
    def __init__(self):
        Spam.numInstances = Spam.numInstances + 1

>>> import spam
>>> a = spam.Spam()
>>> b = spam.Spam()
>>> c = spam.Spam()
>>> spam.printNumInstances()           # But function may be too far removed
Number of instances created:  3        # And cannot be changed via inheritance
>>> spam.Spam.numInstances
3

Because the class name is accessible to the simple function as a global variable, this works fine. Also, note that the name of the function becomes global, but only to this single module; it will not clash with names in other files of the program.

Prior to static methods in Python, this structure was the general prescription. Because Python already provides modules as a namespace-partitioning tool, one could argue that there’s not typically any need to package functions in classes unless they implement object behavior. Simple functions within modules like the one here do much of what instance-less class methods could, and are already associated with the class because they live in the same module.

Unfortunately, this approach is still less than ideal. For one thing, it adds to this file’s scope an extra name that is used only for processing a single class. For another, the function is much less directly associated with the class; in fact, its definition could be hundreds of lines away. Perhaps worse, simple functions like this cannot be customized by inheritance, since they live outside a class’s namespace: subclasses cannot directly replace or extend such a function by redefining it.

We might try to make this example work in a version-neutral way by using a normal method and always calling it through (or with) an instance, as usual:

class Spam:
    numInstances = 0
    def __init__(self):
        Spam.numInstances = Spam.numInstances + 1
    def printNumInstances(self):
        print("Number of instances created: ", Spam.numInstances)

>>> from spam import Spam
>>> a, b, c = Spam(), Spam(), Spam()
>>> a.printNumInstances()
Number of instances created:  3
>>> Spam.printNumInstances(a)
Number of instances created:  3
>>> Spam().printNumInstances()         # But fetching counter changes counter!
Number of instances created:  4

Unfortunately, as mentioned earlier, such an approach is completely unworkable if we don’t have an instance available, and making an instance changes the class data, as illustrated in the last line here. A better solution would be to somehow mark a method inside a class as never requiring an instance. The next section shows how.

Today, there is another option for coding simple functions associated with a class that may be called through either the class or its instances. As of Python 2.2, we can code classes with static and class methods, neither of which requires an instance argument to be passed in when invoked. To designate such methods, classes call the built-in functions staticmethod and classmethod, as hinted in the earlier discussion of new-style classes. Both mark a function object as special—i.e., as requiring no instance if static and requiring a class argument if a class method. For example:

class Methods:
    def imeth(self, x):            # Normal instance method: passed a self
        print(self, x)

    def smeth(x):                  # Static: no instance passed
        print(x)

    def cmeth(cls, x):             # Class: gets class, not instance
        print(cls, x)

    smeth = staticmethod(smeth)    # Make smeth a static method
    cmeth = classmethod(cmeth)     # Make cmeth a class method

Notice how the last two assignments in this code simply reassign the method names smeth and cmeth. Attributes are created and changed by any assignment in a class statement, so these final assignments simply overwrite the assignments made earlier by the defs.

Technically, Python now supports three kinds of class-related methods: instance, static, and class. Moreover, Python 3.0 extends this model by also allowing simple functions in a class to serve the role of static methods without extra protocol, when called through a class.

Instance methods are the normal (and default) case that we’ve seen in this book. An instance method must always be called with an instance object. When you call it through an instance, Python passes the instance to the first (leftmost) argument automatically; when you call it through a class, you must pass along the instance manually (for simplicity, I’ve omitted some class imports in interactive sessions like this one):

>>> obj = Methods()                # Make an instance

>>> obj.imeth(1)                   # Normal method, call through instance
<__main__.Methods object...> 1     # Becomes imeth(obj, 1)

>>> Methods.imeth(obj, 2)          # Normal method, call through class
<__main__.Methods object...> 2     # Instance passed explicitly

By contrast, static methods are called without an instance argument. Unlike simple functions outside a class, their names are local to the scopes of the classes in which they are defined, and they may be looked up by inheritance. Instance-less functions can be called through a class normally in Python 3.0, but never by default in 2.6. Using the staticmethod built-in allows such methods to also be called through an instance in 3.0 and through both a class and an instance in Python 2.6 (the first of these works in 3.0 without staticmethod, but the second does not):

>>> Methods.smeth(3)               # Static method, call through class
3                                  # No instance passed or expected

>>> obj.smeth(4)                   # Static method, call through instance
4                                  # Instance not passed

Class methods are similar, but Python automatically passes the class (not an instance) in to a class method’s first (leftmost) argument, whether it is called through a class or an instance:

>>> Methods.cmeth(5)               # Class method, call through class
<class '__main__.Methods'> 5       # Becomes cmeth(Methods, 5)

>>> obj.cmeth(6)                   # Class method, call through instance
<class '__main__.Methods'> 6       # Becomes cmeth(Methods, 6)

Now, given these built-ins, here is the static method equivalent of this section’s instance-counting example—it marks the method as special, so it will never be passed an instance automatically:

class Spam:
    numInstances = 0                         # Use static method for class data
    def __init__(self):
        Spam.numInstances += 1
    def printNumInstances():
        print("Number of instances:", Spam.numInstances)
    printNumInstances = staticmethod(printNumInstances)

Using the static method built-in, our code now allows the self-less method to be called through the class or any instance of it, in both Python 2.6 and 3.0:

>>> a = Spam()
>>> b = Spam()
>>> c = Spam()
>>> Spam.printNumInstances()                 # Call as simple function
Number of instances: 3
>>> a.printNumInstances()                    # Instance argument not passed
Number of instances: 3

Compared to simply moving printNumInstances outside the class, as prescribed earlier, this version requires an extra staticmethod call; however, it localizes the function name in the class scope (so it won’t clash with other names in the module), moves the function code closer to where it is used (inside the class statement), and allows subclasses to customize the static method with inheritance—a more convenient approach than importing functions from the files in which superclasses are coded. The following subclass and new testing session illustrate:

class Sub(Spam):
    def printNumInstances():                 # Override a static method
        print("Extra stuff...")              # But call back to original
        Spam.printNumInstances()
    printNumInstances = staticmethod(printNumInstances)

>>> a = Sub()
>>> b = Sub()
>>> a.printNumInstances()                    # Call from subclass instance
Extra stuff...
Number of instances: 2
>>> Sub.printNumInstances()                  # Call from subclass itself
Extra stuff...
Number of instances: 2
>>> Spam.printNumInstances()
Number of instances: 2

Moreover, classes can inherit the static method without redefining it—it is run without an instance, regardless of where it is defined in a class tree:

>>> class Other(Spam): pass                  # Inherit static method verbatim

>>> c = Other()
>>> c.printNumInstances()
Number of instances: 3

Interestingly, a class method can do similar work here—the following has the same behavior as the static method version listed earlier, but it uses a class method that receives the instance’s class in its first argument. Rather than hardcoding the class name, the class method uses the automatically passed class object generically:

class Spam:
    numInstances = 0                         # Use class method instead of static
    def __init__(self):
        Spam.numInstances += 1
    def printNumInstances(cls):
        print("Number of instances:", cls.numInstances)
    printNumInstances = classmethod(printNumInstances)

This class is used in the same way as the prior versions, but its printNumInstances method receives the class, not the instance, when called from both the class and an instance:

>>> a, b = Spam(), Spam()
>>> a.printNumInstances()                    # Passes class to first argument
Number of instances: 2
>>> Spam.printNumInstances()                 # Also passes class to first argument
Number of instances: 2

When using class methods, though, keep in mind that they receive the most specific (i.e., lowest) class of the call’s subject. This has some subtle implications when trying to update class data through the passed-in class. For example, if in module test.py we subclass to customize as before, augment Spam.printNumInstances to also display its cls argument, and start a new testing session:

class Spam:
    numInstances = 0                         # Trace class passed in
    def __init__(self):
        Spam.numInstances += 1
    def printNumInstances(cls):
        print("Number of instances:", cls.numInstances, cls)
    printNumInstances = classmethod(printNumInstances)

class Sub(Spam):
    def printNumInstances(cls):              # Override a class method
        print("Extra stuff...", cls)         # But call back to original
        Spam.printNumInstances()
    printNumInstances = classmethod(printNumInstances)

class Other(Spam): pass                      # Inherit class method verbatim

the lowest class is passed in whenever a class method is run, even for subclasses that have no class methods of their own:

>>> x, y = Sub(), Spam()
>>> x.printNumInstances()                    # Call from subclass instance
Extra stuff... <class 'test.Sub'>
Number of instances: 2 <class 'test.Spam'>
>>> Sub.printNumInstances()                  # Call from subclass itself
Extra stuff... <class 'test.Sub'>
Number of instances: 2 <class 'test.Spam'>
>>> y.printNumInstances()
Number of instances: 2 <class 'test.Spam'>

In the first call here, a class method call is made through an instance of the Sub subclass, and Python passes the lowest class, Sub, to the class method. All is well in this case—since Sub’s redefinition of the method calls the Spam superclass’s version explicitly, the superclass method in Spam receives itself in its first argument. But watch what happens for an object that simply inherits the class method:

>>> z = Other()
>>> z.printNumInstances()
Number of instances: 3 <class 'test.Other'>

This last call here passes Other to Spam’s class method. This works in this example because fetching the counter finds it in Spam by inheritance. If this method tried to assign to the passed class’s data, though, it would update Other, not Spam! In this specific case, Spam is probably better off hardcoding its own class name to update its data, rather than relying on the passed-in class argument.

class Spam:
    numInstances = 0
    def count(cls):                    # Per-class instance counters
        cls.numInstances += 1          # cls is lowest class above instance
    def __init__(self):
        self.count()                   # Passes self.__class__ to count
    count = classmethod(count)

class Sub(Spam):
    numInstances = 0
    def __init__(self):                # Redefines __init__
        Spam.__init__(self)

class Other(Spam):                     # Inherits __init__
    numInstances = 0

>>> x = Spam()
>>> y1, y2 = Sub(), Sub()
>>> z1, z2, z3 = Other(), Other(), Other()
>>> x.numInstances, y1.numInstances, z1.numInstances
(1, 2, 3)
>>> Spam.numInstances, Sub.numInstances, Other.numInstances
(1, 2, 3)

Syntactically, a function decorator is a sort of runtime declaration about the function that follows. A function decorator is coded on a line by itself just before the def statement that defines a function or method. It consists of the @ symbol, followed by what we call a metafunction—a function (or other callable object) that manages another function. Static methods today, for example, may be coded with decorator syntax like this:

class C:
   @staticmethod                                 # Decoration syntax
   def meth():
       ...

Internally, this syntax has the same effect as the following (passing the function through the decorator and assigning the result back to the original name):

class C:
   def meth():
       ...
   meth = staticmethod(meth)                     # Rebind name

Decoration rebinds the method name to the decorator’s result. The net effect is that calling the method function’s name later actually triggers the result of its staticmethod decorator first. Because a decorator can return any sort of object, this allows the decorator to insert a layer of logic to be run on every call. The decorator function is free to return either the original function itself, or a new object that saves the original function passed to the decorator to be invoked indirectly after the extra logic layer runs.

With this addition, here’s a better way to code our static method example from the prior section in either Python 2.6 or 3.0 (the classmethod decorator is used the same way):

class Spam:
    numInstances = 0
    def __init__(self):
        Spam.numInstances = Spam.numInstances + 1

    @staticmethod
    def printNumInstances():
        print("Number of instances created: ", Spam.numInstances)

a = Spam()
b = Spam()
c = Spam()
Spam.printNumInstances()      # Calls from both classes and instances work now!
a.printNumInstances()         # Both print "Number of instances created:  3"

Keep in mind that staticmethod is still a built-in function; it may be used in decoration syntax, just because it takes a function as argument and returns a callable. In fact, any such function can be used in this way—even user-defined functions we code ourselves, as the next section explains.

Although Python provides a handful of built-in functions that can be used as decorators, we can also write custom decorators of our own. Because of their wide utility, we’re going to devote an entire chapter to coding decorators in the final part of this book. As a quick example, though, let’s look at a simple user-defined decorator at work.

Recall from Chapter 29 that the __call__ operator overloading method implements a function-call interface for class instances. The following code uses this to define a class that saves the decorated function in the instance and catches calls to the original name. Because this is a class, it also has state information (a counter of calls made):

class tracer:
    def __init__(self, func):
        self.calls = 0
        self.func  = func
    def __call__(self, *args):
        self.calls += 1
        print('call %s to %s' % (self.calls, self.func.__name__))
        self.func(*args)

@tracer                       # Same as spam = tracer(spam)
def spam(a, b, c):            # Wrap spam in a decorator object
    print(a, b, c)

spam(1, 2, 3)                 # Really calls the tracer wrapper object
spam('a', 'b', 'c')           # Invokes __call__ in class
spam(4, 5, 6)                 # __call__ adds logic and runs original object

Because the spam function is run through the tracer decorator, when the original spam name is called it actually triggers the __call__ method in the class. This method counts and logs the call, and then dispatches it to the original wrapped function. Note how the *name argument syntax is used to pack and unpack the passed-in arguments; because of this, this decorator can be used to wrap any function with any number of positional arguments.

The net effect, again, is to add a layer of logic to the original spam function. Here is the script’s output—the first line comes from the tracer class, and the second comes from the spam function:

call 1 to spam
1 2 3
call 2 to spam
a b c
call 3 to spam
4 5 6

Trace through this example’s code for more insight. As it is, this decorator works for any function that takes positional arguments, but it does not return the decorated function’s result, doesn’t handle keyword arguments, and cannot decorate class method functions (in short, for methods its __call__ would be passed a tracer instance only). As we’ll see in Part VIII, there are a variety of ways to code function decorators, including nested def statements; some of the alternatives are better suited to methods than the version shown here.

Function decorators turned out to be so useful that Python 2.6 and 3.0 expanded the model, allowing decorators to be applied to classes as well as functions. In short, class decorators are similar to function decorators, but they are run at the end of a class statement to rebind a class name to a callable. As such, they can be used to either manage classes just after they are created, or insert a layer of wrapper logic to manage instances when they are later created. Symbolically, the code structure:

def decorator(aClass): ...

@decorator
class C: ...

is mapped to the following equivalent:

def decorator(aClass): ...

class C: ...
C = decorator(C)

The class decorator is free to augment the class itself, or return an object that intercepts later instance construction calls. For instance, in the example in the section Counting instances per class with class methods, we could use this hook to automatically augment the classes with instance counters and any other data required:

def count(aClass):
    aClass.numInstances = 0
    return aClass                  # Return class itself, instead of a wrapper

@count
class Spam: ...                    # Same as Spam = count(Spam)

@count
class Sub(Spam): ...               # numInstances = 0 not needed here

@count
class Other(Spam): ...

Metaclasses are a similarly advanced class-based tool whose roles often intersect with those of class decorators. They provide an alternate model, which routes the creation of a class object to a subclass of the top-level type class, at the conclusion of a class statement:

class Meta(type):
    def __new__(meta, classname, supers, classdict): ...

class C(metaclass=Meta): ...

In Python 2.6, the effect is the same, but the coding differs—use a class attribute instead of a keyword argument in the class header:

class C:
    __metaclass__ = Meta
    ...

The metaclass generally redefines the __new__ or __init__ method of the type class, in order to assume control of the creation or initialization of a new class object. The net effect, as with class decorators, is to define code to be run automatically at class creation time. Both schemes are free to augment a class or return an arbitrary object to replace it—a protocol with almost limitless class-based possibilities.

Naturally, there’s much more to the decorator and metaclass stories than I’ve shown here. Although they are a general mechanism, decorators and metaclasses are advanced features of interest primarily to tool writers, not application programmers, so we’ll defer additional coverage until the final part of this book:

Although these chapters cover advanced topics, they’ll also provide us with a chance to see Python at work in more substantial examples than much of the rest of the book was able to provide.

Theoretically speaking, classes (and class instances) are mutable objects. Like built-in lists and dictionaries, they can be changed in-place by assigning to their attributes—and as with lists and dictionaries, this means that changing a class or instance object may impact multiple references to it.

That’s usually what we want (and is how objects change their state in general), but awareness of this issue becomes especially critical when changing class attributes. Because all instances generated from a class share the class’s namespace, any changes at the class level are reflected in all instances, unless they have their own versions of the changed class attributes.

Because classes, modules, and instances are all just objects with attribute namespaces, you can normally change their attributes at runtime by assignments. Consider the following class. Inside the class body, the assignment to the name a generates an attribute X.a, which lives in the class object at runtime and will be inherited by all of X’s instances:

>>> class X:
...     a = 1       # Class attribute
...
>>> I = X()
>>> I.a             # Inherited by instance
1
>>> X.a
1

So far, so good—this is the normal case. But notice what happens when we change the class attribute dynamically outside the class statement: it also changes the attribute in every object that inherits from the class. Moreover, new instances created from the class during this session or program run also get the dynamically set value, regardless of what the class’s source code says:

>>> X.a = 2         # May change more than X
>>> I.a             # I changes too
2
>>> J = X()         # J inherits from X's runtime values
>>> J.a             # (but assigning to J.a changes a in J, not X or I)
2

Is this a useful feature or a dangerous trap? You be the judge. As we learned in Chapter 26, you can actually get work done by changing class attributes without ever making a single instance; this technique can simulate the use of “records” or “structs” in other languages. As a refresher, consider the following unusual but legal Python program:

class X: pass                       # Make a few attribute namespaces
class Y: pass

X.a = 1                             # Use class attributes as variables
X.b = 2                             # No instances anywhere to be found
X.c = 3
Y.a = X.a + X.b + X.c

for X.i in range(Y.a): print(X.i)   # Prints 0..5

Here, the classes X and Y work like “fileless” modules—namespaces for storing variables we don’t want to clash. This is a perfectly legal Python programming trick, but it’s less appropriate when applied to classes written by others; you can’t always be sure that class attributes you change aren’t critical to the class’s internal behavior. If you’re out to simulate a C struct, you may be better off changing instances than classes, as that way only one object is affected:

class Record: pass
X = Record()
X.name = 'bob'
X.job  = 'Pizza maker'

This gotcha is really an extension of the prior. Because class attributes are shared by all instances, if a class attribute references a mutable object, changing that object in-place from any instance impacts all instances at once:

>>> class C:
...     shared = []                 # Class attribute
...     def __init__(self):
...         self.perobj = []        # Instance attribute
...
>>> x = C()                         # Two instances
>>> y = C()                         # Implicitly share class attrs
>>> y.shared, y.perobj
([], [])

>>> x.shared.append('spam')         # Impacts y's view too!
>>> x.perobj.append('spam')         # Impacts x's data only
>>> x.shared, x.perobj
(['spam'], ['spam'])

>>> y.shared, y.perobj              # y sees change made through x
(['spam'], [])
>>> C.shared                        # Stored on class and shared
['spam']

This effect is no different than many we’ve seen in this book already: mutable objects are shared by simple variables, globals are shared by functions, module-level objects are shared by multiple importers, and mutable function arguments are shared by the caller and the callee. All of these are cases of general behavior—multiple references to a mutable object—and all are impacted if the shared object is changed in-place from any reference. Here, this occurs in class attributes shared by all instances via inheritance, but it’s the same phenomenon at work. It may be made more subtle by the different behavior of assignments to instance attributes themselves:

x.shared.append('spam')    # Changes shared object attached to class in-place
x.shared = 'spam'          # Changed or creates instance attribute attached to x

but again, this is not a problem, it’s just something to be aware of; shared mutable class attributes can have many valid uses in Python programs.

This may be obvious by now, but it’s worth underscoring: if you use multiple inheritance, the order in which superclasses are listed in the class statement header can be critical. Python always searches superclasses from left to right, according to their order in the header line.

For instance, in the multiple inheritance example we studied in Chapter 30, suppose that the Super class implemented a __str__ method, too:

class ListTree:
    def __str__(self): ...

class Super:
    def __str__(self): ...

class Sub(ListTree, Super):    # Get ListTree's __str__ by listing it first

x = Sub()                      # Inheritance searches ListTree before Super

Which class would we inherit it from—ListTree or Super? As inheritance searches proceed from left to right, we would get the method from whichever class is listed first (leftmost) in Sub’s class header. Presumably, we would list ListTree first because its whole purpose is its custom __str__ (indeed, we had to do this in Chapter 30 when mixing this class with a tkinter.Button that had a __str__ of its own).

But now suppose Super and ListTree have their own versions of other same-named attributes, too. If we want one name from Super and another from ListTree, the order in which we list them in the class header won’t help—we will have to override inheritance by manually assigning to the attribute name in the Sub class:

class ListTree:
    def __str__(self): ...
    def other(self): ...

class Super:
    def __str__(self): ...
    def other(self): ...

class Sub(ListTree, Super):    # Get ListTree's __str__ by listing it first
    other = Super.other        # But explicitly pick Super's version of other
    def __init__(self):
        ...

x = Sub()                      # Inheritance searches Sub before ListTree/Super

Here, the assignment to other within the Sub class creates Sub.other—a reference back to the Super.other object. Because it is lower in the tree, Sub.other effectively hides ListTree.other, the attribute that the inheritance search would normally find. Similarly, if we listed Super first in the class header to pick up its other, we would need to select ListTree’s method explicitly:

class Sub(Super, ListTree):               # Get Super's other by order
    __str__ = Lister.__str__              # Explicitly pick Lister.__str__

Multiple inheritance is an advanced tool. Even if you understood the last paragraph, it’s still a good idea to use it sparingly and carefully. Otherwise, the meaning of a name may come to depend on the order in which classes are mixed in an arbitrarily far-removed subclass. (For another example of the technique shown here in action, see the discussion of explicit conflict resolution in The “New-Style” Class Model.)

As a rule of thumb, multiple inheritance works best when your mix-in classes are as self-contained as possible—because they may be used in a variety of contexts, they should not make assumptions about names related to other classes in a tree. The pseudoprivate __X attributes feature we studied in Chapter 30 can help by localizing names that a class relies on owning and limiting the names that your mix-in classes add to the mix. In this example, for instance, if ListTree only means to export its custom __str__, it can name its other method __other to avoid clashing with like-named classes in the tree.

This gotcha went away in Python 2.2 with the introduction of nested function scopes, but I’ve retained it here for historical perspective, for readers working with older Python releases, and because it demonstrates what happens to the new nested function scope rules when one layer of the nesting is a class.

Classes introduce local scopes, just as functions do, so the same sorts of scope behavior can happen in a class statement body. Moreover, methods are further nested functions, so the same issues apply. Confusion seems to be especially common when classes are nested.

In the following example (the file nester.py), the generate function returns an instance of the nested Spam class. Within its code, the class name Spam is assigned in the generate function’s local scope. However, in versions of Python prior to 2.2, within the class’s method function the class name Spam is not visible—method has access only to its own local scope, the module surrounding generate, and built-in names:

def generate():                  # Fails prior to Python 2.2, works later
    class Spam:
        count = 1
        def method(self):        # Name Spam not visible:
            print(Spam.count)    # not local (def), global (module), built-in
    return Spam()

generate().method()

C:pythonexamples> python nester.py
...error text omitted...
    print(Spam.count)            # Not local (def), global (module), built-in
NameError: Spam

This example works in Python 2.2 and later because the local scopes of all enclosing function defs are automatically visible to nested defs (including nested method defs, as in this example). However, it doesn’t work before 2.2 (we’ll look at some possible solutions momentarily).

Note that even in 2.2 and later, method defs cannot see the local scope of the enclosing class; they can only see the local scopes of enclosing defs. That’s why methods must go through the self instance or the class name to reference methods and other attributes defined in the enclosing class statement. For example, code in the method must use self.count or Spam.count, not just count.

If you’re using a release prior to 2.2, there are a variety of ways to get the preceding example to work. One of the simplest is to move the name Spam out to the enclosing module’s scope with a global declaration. Because method sees global names in the enclosing module, references to Spam will work:

def generate():
    global Spam                 # Force Spam to module scope
    class Spam:
        count = 1
        def method(self):
            print(Spam.count)   # Works: in global (enclosing module)
    return Spam()

generate().method()             # Prints 1

A better alternative would be to restructure the code such that the class Spam is defined at the top level of the module by virtue of its nesting level, rather than using global declarations. The nested method function and the top-level generate will then find Spam in their global scopes:

def generate():
    return Spam()

class Spam:                    # Define at top level of module
    count = 1
    def method(self):
        print(Spam.count)      # Works: in global (enclosing module)

generate().method()

In fact, this approach is recommended for all Python releases—code tends to be simpler in general if you avoid nesting classes and functions.

If you want to get complicated and tricky, you can also get rid of the Spam reference in method altogether by using the special __class__ attribute, which returns an instance’s class object:

def generate():
    class Spam:
        count = 1
        def method(self):
            print(self.__class__.count)      # Works: qualify to get class
    return Spam()

generate().method()

We met this issue briefly in our class tutorial in Chapter 27 and our delegation coverage in Chapter 30: classes that use the __getattr__ operator overloading method to delegate attribute fetches to wrapped objects will fail in Python 3.0 unless operator overloading methods are redefined in the wrapper class. In Python 3.0 (and 2.6, when new-style classes are used), the names of operator overloading methods implicitly fetched by built-in operations are not routed through generic attribute-interception methods. The __str__ method used by printing, for example, never invokes __getattr__. Instead, Python 3.0 looks up such names in classes and skips the normal runtime instance lookup mechanism entirely. To work around this, such methods must be redefined in wrapper classes, either by hand, with tools, or by definition in superclasses. We’ll revisit this gotcha in Chapters 37 and 38.

When used well, the code reuse features of OOP make it excel at cutting development time. Sometimes, though, OOP’s abstraction potential can be abused to the point of making code difficult to understand. If classes are layered too deeply, code can become obscure; you may have to search through many classes to discover what an operation does.

For example, I once worked in a C++ shop with thousands of classes (some machine-generated), and up to 15 levels of inheritance. Deciphering method calls in such a complex system was often a monumental task: multiple classes had to be consulted for even the most basic of operations. In fact, the logic of the system was so deeply wrapped that understanding a piece of code in some cases required days of wading through related files.

The most general rule of thumb of Python programming applies here, too: don’t make things complicated unless they truly must be. Wrapping your code in multiple layers of classes to the point of incomprehensibility is always a bad idea. Abstraction is the basis of polymorphism and encapsulation, and it can be a very effective tool when used well. However, you’ll simplify debugging and aid maintainability if you make your class interfaces intuitive, avoid making your code overly abstract, and keep your class hierarchies short and flat unless there is a good reason to do otherwise.