Chapter 30. Designing with Classes

So far in this part of the book, we’ve concentrated on using Python’s OOP tool, the class. But OOP is also about design issues—i.e., how to use classes to model useful objects. This chapter will touch on a few core OOP ideas and present some additional examples that are more realistic than those shown so far.

Along the way, we’ll code some common OOP design patterns in Python, such as inheritance, composition, delegation, and factories. We’ll also investigate some design-focused class concepts, such as pseudoprivate attributes, multiple inheritance, and bound methods. Many of the design terms mentioned here require more explanation than I can provide in this book; if this material sparks your curiosity, I suggest exploring a text on OOP design or design patterns as a next step.

Python and OOP

Let’s begin with a review—Python’s implementation of OOP can be summarized by three ideas:

Inheritance

Inheritance is based on attribute lookup in Python (in X.name expressions).

Polymorphism

In X.method, the meaning of method depends on the type (class) of X.

Encapsulation

Methods and operators implement behavior; data hiding is a convention by default.

By now, you should have a good feel for what inheritance is all about in Python. We’ve also talked about Python’s polymorphism a few times already; it flows from Python’s lack of type declarations. Because attributes are always resolved at runtime, objects that implement the same interfaces are interchangeable; clients don’t need to know what sorts of objects are implementing the methods they call.

Encapsulation means packaging in Python—that is, hiding implementation details behind an object’s interface. It does not mean enforced privacy, though that can be implemented with code, as we’ll see in Chapter 38. Encapsulation allows the implementation of an object’s interface to be changed without impacting the users of that object.

Overloading by Call Signatures (or Not)

Some OOP languages also define polymorphism to mean overloading functions based on the type signatures of their arguments. But because there are no type declarations in Python, this concept doesn’t really apply; polymorphism in Python is based on object interfaces, not types.

You can try to overload methods by their argument lists, like this:

class C:
    def meth(self, x):
        ...
    def meth(self, x, y, z):
        ...

This code will run, but because the def simply assigns an object to a name in the class’s scope, the last definition of the method function is the only one that will be retained (it’s just as if you say X = 1 and then X = 2; X will be 2).

Type-based selections can always be coded using the type-testing ideas we met in Chapters 4 and 9, or the argument list tools introduced in Chapter 18:

class C:
    def meth(self, *args):
        if len(args) == 1:
            ...
        elif type(arg[0]) == int:
            ...

You normally shouldn’t do this, though—as described in Chapter 16, you should write your code to expect an object interface, not a specific data type. That way, it will be useful for a broader category of types and applications, both now and in the future:

class C:
    def meth(self, x):
        x.operation()                   # Assume x does the right thing

It’s also generally considered better to use distinct method names for distinct operations, rather than relying on call signatures (no matter what language you code in).

Although Python’s object model is straightforward, much of the art in OOP is in the way we combine classes to achieve a program’s goals. The next section begins a tour of some of the ways larger programs use classes to their advantage.

OOP and Inheritance: “Is-a” Relationships

We’ve explored the mechanics of inheritance in depth already, but I’d like to show you an example of how it can be used to model real-world relationships. From a programmer’s point of view, inheritance is kicked off by attribute qualifications, which trigger searches for names in instances, their classes, and then any superclasses. From a designer’s point of view, inheritance is a way to specify set membership: a class defines a set of properties that may be inherited and customized by more specific sets (i.e., subclasses).

To illustrate, let’s put that pizza-making robot we talked about at the start of this part of the book to work. Suppose we’ve decided to explore alternative career paths and open a pizza restaurant. One of the first things we’ll need to do is hire employees to serve customers, prepare the food, and so on. Being engineers at heart, we’ve decided to build a robot to make the pizzas; but being politically and cybernetically correct, we’ve also decided to make our robot a full-fledged employee with a salary.

Our pizza shop team can be defined by the four classes in the example file, employees.py. The most general class, Employee, provides common behavior such as bumping up salaries (giveRaise) and printing (__repr__). There are two kinds of employees, and so two subclasses of Employee: Chef and Server. Both override the inherited work method to print more specific messages. Finally, our pizza robot is modeled by an even more specific class: PizzaRobot is a kind of Chef, which is a kind of Employee. In OOP terms, we call these relationships “is-a” links: a robot is a chef, which is a(n) employee. Here’s the employees.py file:

class Employee:
    def __init__(self, name, salary=0):
        self.name   = name
        self.salary = salary
    def giveRaise(self, percent):
        self.salary = self.salary + (self.salary * percent)
    def work(self):
        print(self.name, "does stuff")
    def __repr__(self):
        return "<Employee: name=%s, salary=%s>" % (self.name, self.salary)

class Chef(Employee):
    def __init__(self, name):
        Employee.__init__(self, name, 50000)
    def work(self):
        print(self.name, "makes food")

class Server(Employee):
    def __init__(self, name):
        Employee.__init__(self, name, 40000)
    def work(self):
        print(self.name, "interfaces with customer")

class PizzaRobot(Chef):
    def __init__(self, name):
        Chef.__init__(self, name)
    def work(self):
        print(self.name, "makes pizza")

if __name__ == "__main__":
    bob = PizzaRobot('bob')       # Make a robot named bob
    print(bob)                    # Run inherited __repr__
    bob.work()                    # Run type-specific action
    bob.giveRaise(0.20)           # Give bob a 20% raise
    print(bob); print()

    for klass in Employee, Chef, Server, PizzaRobot:
        obj = klass(klass.__name__)
        obj.work()

When we run the self-test code included in this module, we create a pizza-making robot named bob, which inherits names from three classes: PizzaRobot, Chef, and Employee. For instance, printing bob runs the Employee.__repr__ method, and giving bob a raise invokes Employee.giveRaise because that’s where the inheritance search finds that method:

C:pythonexamples> python employees.py
<Employee: name=bob, salary=50000>
bob makes pizza
<Employee: name=bob, salary=60000.0>

Employee does stuff
Chef makes food
Server interfaces with customer
PizzaRobot makes pizza

In a class hierarchy like this, you can usually make instances of any of the classes, not just the ones at the bottom. For instance, the for loop in this module’s self-test code creates instances of all four classes; each responds differently when asked to work because the work method is different in each. Really, these classes just simulate real-world objects; work prints a message for the time being, but it could be expanded to do real work later.

OOP and Composition: “Has-a” Relationships

The notion of composition was introduced in Chapter 25. From a programmer’s point of view, composition involves embedding other objects in a container object, and activating them to implement container methods. To a designer, composition is another way to represent relationships in a problem domain. But, rather than set membership, composition has to do with components—parts of a whole.

Composition also reflects the relationships between parts, called a “has-a” relationships. Some OOP design texts refer to composition as aggregation (or distinguish between the two terms by using aggregation to describe a weaker dependency between container and contained); in this text, a “composition” simply refers to a collection of embedded objects. The composite class generally provides an interface all its own and implements it by directing the embedded objects.

Now that we’ve implemented our employees, let’s put them in the pizza shop and let them get busy. Our pizza shop is a composite object: it has an oven, and it has employees like servers and chefs. When a customer enters and places an order, the components of the shop spring into action—the server takes the order, the chef makes the pizza, and so on. The following example (the file pizzashop.py) simulates all the objects and relationships in this scenario:

from employees import PizzaRobot, Server

class Customer:
    def __init__(self, name):
        self.name = name
    def order(self, server):
        print(self.name, "orders from", server)
    def pay(self, server):
        print(self.name, "pays for item to", server)

class Oven:
    def bake(self):
        print("oven bakes")

class PizzaShop:
    def __init__(self):
        self.server = Server('Pat')         # Embed other objects
        self.chef   = PizzaRobot('Bob')     # A robot named bob
        self.oven   = Oven()

    def order(self, name):
        customer = Customer(name)           # Activate other objects
        customer.order(self.server)         # Customer orders from server
        self.chef.work()
        self.oven.bake()
        customer.pay(self.server)

if __name__ == "__main__":
    scene = PizzaShop()                     # Make the composite
    scene.order('Homer')                    # Simulate Homer's order
    print('...')
    scene.order('Shaggy')                   # Simulate Shaggy's order

The PizzaShop class is a container and controller; its constructor makes and embeds instances of the employee classes we wrote in the last section, as well as an Oven class defined here. When this module’s self-test code calls the PizzaShop order method, the embedded objects are asked to carry out their actions in turn. Notice that we make a new Customer object for each order, and we pass on the embedded Server object to Customer methods; customers come and go, but the server is part of the pizza shop composite. Also notice that employees are still involved in an inheritance relationship; composition and inheritance are complementary tools.

When we run this module, our pizza shop handles two orders—one from Homer, and then one from Shaggy:

C:pythonexamples> python pizzashop.py
Homer orders from <Employee: name=Pat, salary=40000>
Bob makes pizza
oven bakes
Homer pays for item to <Employee: name=Pat, salary=40000>
...
Shaggy orders from <Employee: name=Pat, salary=40000>
Bob makes pizza
oven bakes
Shaggy pays for item to <Employee: name=Pat, salary=40000>

Again, this is mostly just a toy simulation, but the objects and interactions are representative of composites at work. As a rule of thumb, classes can represent just about any objects and relationships you can express in a sentence; just replace nouns with classes, and verbs with methods, and you’ll have a first cut at a design.

Stream Processors Revisited

For a more realistic composition example, recall the generic data stream processor function we partially coded in the introduction to OOP in Chapter 25:

def processor(reader, converter, writer):
    while 1:
        data = reader.read()
        if not data: break
        data = converter(data)
        writer.write(data)

Rather than using a simple function here, we might code this as a class that uses composition to do its work to provide more structure and support inheritance. The following file, streams.py, demonstrates one way to code the class:

class Processor:
    def __init__(self, reader, writer):
        self.reader = reader
        self.writer = writer
    def process(self):
        while 1:
            data = self.reader.readline()
            if not data: break
            data = self.converter(data)
            self.writer.write(data)
    def converter(self, data):
        assert False, 'converter must be defined'       # Or raise exception

This class defines a converter method that it expects subclasses to fill in; it’s an example of the abstract superclass model we outlined in Chapter 28 (more on assert in Part VII). Coded this way, reader and writer objects are embedded within the class instance (composition), and we supply the conversion logic in a subclass rather than passing in a converter function (inheritance). The file converters.py shows how:

from streams import Processor

class Uppercase(Processor):
    def converter(self, data):
        return data.upper()

if __name__ == '__main__':
    import sys
    obj = Uppercase(open('spam.txt'), sys.stdout)
    obj.process()

Here, the Uppercase class inherits the stream-processing loop logic (and anything else that may be coded in its superclasses). It needs to define only what is unique about it—the data conversion logic. When this file is run, it makes and runs an instance that reads from the file spam.txt and writes the uppercase equivalent of that file to the stdout stream:

C:lp4e> type spam.txt
spam
Spam
SPAM!

C:lp4e> python converters.py
SPAM
SPAM
SPAM!

To process different sorts of streams, pass in different sorts of objects to the class construction call. Here, we use an output file instead of a stream:

C:lp4e> python
>>> import converters
>>> prog = converters.Uppercase(open('spam.txt'), open('spamup.txt', 'w'))
>>> prog.process()

C:lp4e> type spamup.txt
SPAM
SPAM
SPAM!

But, as suggested earlier, we could also pass in arbitrary objects wrapped up in classes that define the required input and output method interfaces. Here’s a simple example that passes in a writer class that wraps up the text inside HTML tags:

C:lp4e> python
>>> from converters import Uppercase
>>>
>>> class HTMLize:
...      def write(self, line):
...         print('<PRE>%s</PRE>' % line.rstrip())
...
>>> Uppercase(open('spam.txt'), HTMLize()).process()
<PRE>SPAM</PRE>
<PRE>SPAM</PRE>
<PRE>SPAM!</PRE>

If you trace through this example’s control flow, you’ll see that we get both uppercase conversion (by inheritance) and HTML formatting (by composition), even though the core processing logic in the original Processor superclass knows nothing about either step. The processing code only cares that writers have a write method and that a method named convert is defined; it doesn’t care what those methods do when they are called. Such polymorphism and encapsulation of logic is behind much of the power of classes.

As is, the Processor superclass only provides a file-scanning loop. In more realistic work, we might extend it to support additional programming tools for its subclasses, and, in the process, turn it into a full-blown framework. Coding such a tool once in a superclass enables you to reuse it in all of your programs. Even in this simple example, because so much is packaged and inherited with classes, all we had to code was the HTML formatting step; the rest was free.

For another example of composition at work, see exercise 9 at the end of Chapter 31 and its solution in Appendix B; it’s similar to the pizza shop example. We’ve focused on inheritance in this book because that is the main tool that the Python language itself provides for OOP. But, in practice, composition is used as much as inheritance as a way to structure classes, especially in larger systems. As we’ve seen, inheritance and composition are often complementary (and sometimes alternative) techniques. Because composition is a design issue outside the scope of the Python language and this book, though, I’ll defer to other resources for more on this topic.

OOP and Delegation: “Wrapper” Objects

Beside inheritance and composition, object-oriented programmers often also talk about something called delegation, which usually implies controller objects that embed other objects to which they pass off operation requests. The controllers can take care of administrative activities, such as keeping track of accesses and so on. In Python, delegation is often implemented with the __getattr__ method hook; because it intercepts accesses to nonexistent attributes, a wrapper class (sometimes called a proxy class) can use __getattr__ to route arbitrary accesses to a wrapped object. The wrapper class retains the interface of the wrapped object and may add additional operations of its own.

Consider the file trace.py, for instance:

class wrapper:
    def __init__(self, object):
        self.wrapped = object                    # Save object
    def __getattr__(self, attrname):
        print('Trace:', attrname)                # Trace fetch
        return getattr(self.wrapped, attrname)   # Delegate fetch

Recall from Chapter 29 that __getattr__ gets the attribute name as a string. This code makes use of the getattr built-in function to fetch an attribute from the wrapped object by name string—getattr(X,N) is like X.N, except that N is an expression that evaluates to a string at runtime, not a variable. In fact, getattr(X,N) is similar to X.__dict__[N], but the former also performs an inheritance search, like X.N, while the latter does not (see Namespace Dictionaries for more on the __dict__ attribute).

You can use the approach of this module’s wrapper class to manage access to any object with attributes—lists, dictionaries, and even classes and instances. Here, the wrapper class simply prints a trace message on each attribute access and delegates the attribute request to the embedded wrapped object:

>>> from trace import wrapper
>>> x = wrapper([1,2,3])                         # Wrap a list
>>> x.append(4)                                  # Delegate to list method
Trace: append
>>> x.wrapped                                    # Print my member
[1, 2, 3, 4]

>>> x = wrapper({"a": 1, "b": 2})                # Wrap a dictionary
>>> list(x.keys())                               # Delegate to dictionary method
Trace: keys
['a', 'b']

The net effect is to augment the entire interface of the wrapped object, with additional code in the wrapper class. We can use this to log our method calls, route method calls to extra or custom logic, and so on.

We’ll revive the notions of wrapped objects and delegated operations as one way to extend built-in types in Chapter 31. If you are interested in the delegation design pattern, also watch for the discussions in Chapters 31 and 38 of function decorators, a strongly related concept designed to augment a specific function or method call rather than the entire interface of an object, and class decorators, which serve as a way to automatically add such delegation-based wrappers to all instances of a class.

Note

Version skew note: In Python 2.6, operator overloading methods run by built-in operations are routed through generic attribute interception methods like __getattr__. Printing a wrapped object directly, for example, calls this method for __repr__ or __str__, which then passes the call on to the wrapped object. In Python 3.0, this no longer happens: printing does not trigger __getattr__, and a default display is used instead. In 3.0, new-style classes look up operator overloading methods in classes and skip the normal instance lookup entirely. We’ll return to this issue in Chapter 37, in the context of managed attributes; for now, keep in mind that you may need to redefine operator overloading methods in wrapper classes (either by hand, by tools, or by superclasses) if you want them to be intercepted in 3.0.

Pseudoprivate Class Attributes

Besides larger structuring goals, class designs often must address name usage too. In Part V, we learned that every name assigned at the top level of a module file is exported. By default, the same holds for classes—data hiding is a convention, and clients may fetch or change any class or instance attribute they like. In fact, attributes are all “public” and “virtual,” in C++ terms; they’re all accessible everywhere and are looked up dynamically at runtime.[69]

That said, Python today does support the notion of name “mangling” (i.e., expansion) to localize some names in classes. Mangled names are sometimes misleadingly called “private attributes,” but really this is just a way to localize a name to the class that created it—name mangling does not prevent access by code outside the class. This feature is mostly intended to avoid namespace collisions in instances, not to restrict access to names in general; mangled names are therefore better called “pseudoprivate” than “private.”

Pseudoprivate names are an advanced and entirely optional feature, and you probably won’t find them very useful until you start writing general tools or larger class hierarchies for use in multiprogrammer projects. In fact, they are not always used even when they probably should be—more commonly, Python programmers code internal names with a single underscore (e.g., _X), which is just an informal convention to let you know that a name shouldn’t be changed (it means nothing to Python itself).

Because you may see this feature in other people’s code, though, you need to be somewhat aware of it, even if you don’t use it yourself.

Name Mangling Overview

Here’s how name mangling works: names inside a class statement that start with two underscores but don’t end with two underscores are automatically expanded to include the name of the enclosing class. For instance, a name like __X within a class named Spam is changed to _Spam__X automatically: the original name is prefixed with a single underscore and the enclosing class’s name. Because the modified name contains the name of the enclosing class, it’s somewhat unique; it won’t clash with similar names created by other classes in a hierarchy.

Name mangling happens only in class statements, and only for names that begin with two leading underscores. However, it happens for every name preceded with double underscores—both class attributes (like method names) and instance attribute names assigned to self attributes. For example, in a class named Spam, a method named __meth is mangled to _Spam__meth, and an instance attribute reference self.__X is transformed to self._Spam__X. Because more than one class may add attributes to an instance, this mangling helps avoid clashes—but we need to move on to an example to see how.

Why Use Pseudoprivate Attributes?

One of the main problems that the pseudoprivate attribute feature is meant to alleviate has to do with the way instance attributes are stored. In Python, all instance attributes wind up in the single instance object at the bottom of the class tree. This is different from the C++ model, where each class gets its own space for data members it defines.

Within a class method in Python, whenever a method assigns to a self attribute (e.g., self.attr = value), it changes or creates an attribute in the instance (inheritance searches happen only on reference, not on assignment). Because this is true even if multiple classes in a hierarchy assign to the same attribute, collisions are possible.

For example, suppose that when a programmer codes a class, she assumes that she owns the attribute name X in the instance. In this class’s methods, the name is set, and later fetched:

class C1:
    def meth1(self): self.X = 88         # I assume X is mine
    def meth2(self): print(self.X)

Suppose further that another programmer, working in isolation, makes the same assumption in a class that he codes:

class C2:
    def metha(self): self.X = 99         # Me too
    def methb(self): print(self.X)

Both of these classes work by themselves. The problem arises if the two classes are ever mixed together in the same class tree:

class C3(C1, C2): ...
I = C3()                                 # Only 1 X in I!

Now, the value that each class gets back when it says self.X will depend on which class assigned it last. Because all assignments to self.X refer to the same single instance, there is only one X attribute—I.X—no matter how many classes use that attribute name.

To guarantee that an attribute belongs to the class that uses it, prefix the name with double underscores everywhere it is used in the class, as in this file, private.py:

class C1:
    def meth1(self): self.__X = 88       # Now X is mine
    def meth2(self): print(self.__X)     # Becomes _C1__X in I
class C2:
    def metha(self): self.__X = 99       # Me too
    def methb(self): print(self.__X)     # Becomes _C2__X in I

class C3(C1, C2): pass
I = C3()                                 # Two X names in I

I.meth1(); I.metha()
print(I.__dict__)
I.meth2(); I.methb()

When thus prefixed, the X attributes will be expanded to include the names of their classes before being added to the instance. If you run a dir call on I or inspect its namespace dictionary after the attributes have been assigned, you’ll see the expanded names, _C1__X and _C2__X, but not X. Because the expansion makes the names unique within the instance, the class coders can safely assume that they truly own any names that they prefix with two underscores:

% python private.py
{'_C2__X': 99, '_C1__X': 88}
88
99

This trick can avoid potential name collisions in the instance, but note that it does not amount to true privacy. If you know the name of the enclosing class, you can still access either of these attributes anywhere you have a reference to the instance by using the fully expanded name (e.g., I._C1__X = 77). On the other hand, this feature makes it less likely that you will accidentally step on a class’s names.

Pseudoprivate attributes are also useful in larger frameworks or tools, both to avoid introducing new method names that might accidentally hide definitions elsewhere in the class tree and to reduce the chance of internal methods being replaced by names defined lower in the tree. If a method is intended for use only within a class that may be mixed into other classes, the double underscore prefix ensures that the method won’t interfere with other names in the tree, especially in multiple-inheritance scenarios:

class Super:
    def method(self): ...                  # A real application method

class Tool:
    def __method(self): ...                # Becomes _Tool__method
    def other(self): self.__method()       # Use my internal method

class Sub1(Tool, Super): ...
    def actions(self): self.method()       # Runs Super.method as expected

class Sub2(Tool):
    def __init__(self): self.method = 99   # Doesn't break Tool.__method

We met multiple inheritance briefly in Chapter 25 and will explore it in more detail later in this chapter. Recall that superclasses are searched according to their left-to-right order in class header lines. Here, this means Sub1 prefers Tool attributes to those in Super. Although in this example we could force Python to pick the application class’s methods first by switching the order of the superclasses listed in the Sub1 class header, pseudoprivate attributes resolve the issue altogether. Pseudoprivate names also prevent subclasses from accidentally redefining the internal method’s names, as in Sub2.

Again, I should note that this feature tends to be of use primarily for larger, multiprogrammer projects, and then only for selected names. Don’t be tempted to clutter your code unnecessarily; only use this feature for names that truly need to be controlled by a single class. For simpler programs, it’s probably overkill.

For more examples that make use of the __X naming feature, see the lister.py mix-in classes introduced later in this chapter, in the section on multiple inheritance, as well as the discussion of Private class decorators in Chapter 38. If you care about privacy in general, you might want to review the emulation of private instance attributes sketched in the section Attribute Reference: __getattr__ and __setattr__ in Chapter 29, and watch for the Private class decorator in Chapter 38 that we will base upon this special method. Although it’s possible to emulate true access controls in Python classes, this is rarely done in practice, even for large systems.

Methods Are Objects: Bound or Unbound

Methods in general, and bound methods in particular, simplify the implementation of many design goals in Python. We met bound methods briefly while studying __call__ in Chapter 29. The full story, which we’ll flesh out here, turns out to be more general and flexible than you might expect.

In Chapter 19, we learned how functions can be processed as normal objects. Methods are a kind of object too, and can be used generically in much the same way as other objects—they can be assigned, passed to functions, stored in data structures, and so on. Because class methods can be accessed from an instance or a class, though, they actually come in two flavors in Python:

Unbound class method objects: no self

Accessing a function attribute of a class by qualifying the class returns an unbound method object. To call the method, you must provide an instance object explicitly as the first argument. In Python 3.0, an unbound method is the same as a simple function and can be called though the class’s name; in 2.6 it’s a distinct type and cannot be called without providing an instance.

Bound instance method objects: self + function pairs

Accessing a function attribute of a class by qualifying an instance returns a bound method object. Python automatically packages the instance with the function in the bound method object, so you don’t need to pass an instance to call the method.

Both kinds of methods are full-fledged objects; they can be transferred around a program at will, just like strings and numbers. Both also require an instance in their first argument when run (i.e., a value for self). This is why we had to pass in an instance explicitly when calling superclass methods from subclass methods in the previous chapter; technically, such calls produce unbound method objects.

When calling a bound method object, Python provides an instance for you automatically—the instance used to create the bound method object. This means that bound method objects are usually interchangeable with simple function objects, and makes them especially useful for interfaces originally written for functions (see the sidebar Why You Will Care: Bound Methods and Callbacks for a realistic example).

To illustrate, suppose we define the following class:

class Spam:
    def doit(self, message):
        print(message)

Now, in normal operation, we make an instance and call its method in a single step to print the passed-in argument:

object1 = Spam()
object1.doit('hello world')

Really, though, a bound method object is generated along the way, just before the method call’s parentheses. In fact, we can fetch a bound method without actually calling it. An object.name qualification is an object expression. In the following, it returns a bound method object that packages the instance (object1) with the method function (Spam.doit). We can assign this bound method pair to another name and then call it as though it were a simple function:

object1 = Spam()
x = object1.doit        # Bound method object: instance+function
x('hello world')        # Same effect as object1.doit('...')

On the other hand, if we qualify the class to get to doit, we get back an unbound method object, which is simply a reference to the function object. To call this type of method, we must pass in an instance as the leftmost argument:

object1 = Spam()
t = Spam.doit           # Unbound method object (a function in 3.0: see ahead)
t(object1, 'howdy')     # Pass in instance (if the method expects one in 3.0)

By extension, the same rules apply within a class’s method if we reference self attributes that refer to functions in the class. A self.method expression is a bound method object because self is an instance object:

class Eggs:
    def m1(self, n):
        print(n)
    def m2(self):
        x = self.m1     # Another bound method object
        x(42)           # Looks like a simple function

Eggs().m2()             # Prints 42

Most of the time, you call methods immediately after fetching them with attribute qualification, so you don’t always notice the method objects generated along the way. But if you start writing code that calls objects generically, you need to be careful to treat unbound methods specially—they normally require an explicit instance object to be passed in.[70]

Unbound Methods are Functions in 3.0

In Python 3.0, the language has dropped the notion of unbound methods. What we describe as an unbound method here is treated as a simple function in 3.0. For most purposes, this makes no difference to your code; either way, an instance will be passed to a method’s first argument when it’s called through an instance.

Programs that do explicit type testing might be impacted, though—if you print the type of an instance-less class method, it displays “unbound method” in 2.6, and “function” in 3.0.

Moreover, in 3.0 it is OK to call a method without an instance, as long as the method does not expect one and you call it only through the class and never through an instance. That is, Python 3.0 will pass along an instance to methods only for through-instance calls. When calling through a class, you must pass an instance manually only if the method expects one:

C:misc> c:python30python
>>> class Selfless:
...     def __init__(self, data):
...         self.data = data
...     def selfless(arg1, arg2):               # A simple function in 3.0
...         return arg1 + arg2
...     def normal(self, arg1, arg2):           # Instance expected when called
...         return self.data + arg1 + arg2
...
>>> X = Selfless(2)
>>> X.normal(3, 4)                  # Instance passed to self automatically
9
>>> Selfless.normal(X, 3, 4)        # self expected by method: pass manually
9
>>> Selfless.selfless(3, 4)         # No instance: works in 3.0, fails in 2.6!
7

The last test in this fails in 2.6, because unbound methods require an instance to be passed by default; it works in 3.0 because such methods are treated as simple functions not requiring an instance. Although this removes some potential error trapping in 3.0 (what if a programmer accidentally forgets to pass an instance?), it allows class methods to be used as simple functions as long as they are not passed and do not expect a “self” instance argument.

The following two calls still fail in both 3.0 and 2.6, though—the first (calling through an instance) automatically passes an instance to a method that does not expect one, while the second (calling through a class) does not pass an instance to a method that does expect one:

>>> X.selfless(3, 4)
TypeError: selfless() takes exactly 2 positional arguments (3 given)

>>> Selfless.normal(3, 4)
TypeError: normal() takes exactly 3 positional arguments (2 given)

Because of this change, the staticmethod decorator described in the next chapter is not needed in 3.0 for methods without a self argument that are called only through the class name, and never through an instance—such methods are run as simple functions, without receiving an instance argument. In 2.6, such calls are errors unless an instance is passed manually (more on static methods in the next chapter).

It’s important to be aware of the differences in behavior in 3.0, but bound methods are generally more important from a practical perspective anyway. Because they pair together the instance and function in a single object, they can be treated as callables generically. The next section demonstrates what this means in code.

Note

For a more visual illustration of unbound method treatment in Python 3.0 and 2.6, see also the lister.py example in the multiple inheritance section later in this chapter. Its classes print the value of methods fetched from both instances and classes, in both versions of Python.

Bound Methods and Other Callable Objects

As mentioned earlier, bound methods can be processed as generic objects, just like simple functions—they can be passed around a program arbitrarily. Moreover, because bound methods combine both a function and an instance in a single package, they can be treated like any other callable object and require no special syntax when invoked. The following, for example, stores four bound method objects in a list and calls them later with normal call expressions:

>>> class Number:
...     def __init__(self, base):
...         self.base = base
...     def double(self):
...         return self.base * 2
...     def triple(self):
...         return self.base * 3
...
>>> x = Number(2)                                       # Class instance objects
>>> y = Number(3)                                       # State + methods
>>> z = Number(4)
>>> x.double()                                          # Normal immediate calls
4

>>> acts = [x.double, y.double, y.triple, z.double]     # List of bound methods
>>> for act in acts:                                    # Calls are deferred
...     print(act())                                    # Call as though functions
...
4
6
9
8

Like simple functions, bound method objects have introspection information of their own, including attributes that give access to the instance object and method function they pair. Calling the bound method simply dispatches the pair:

>>> bound = x.double
>>> bound.__self__, bound.__func__
(<__main__.Number object at 0x0278F610>, <function double at 0x027A4ED0>)
>>> bound.__self__.base
2
>>> bound()                             # Calls bound.__func__(bound.__self__, ...)
4

In fact, bound methods are just one of a handful of callable object types in Python. As the following demonstrates, simple functions coded with a def or lambda, instances that inherit a __call__, and bound instance methods can all be treated and called the same way:

>>> def square(arg):
...     return arg ** 2                          # Simple functions (def or lambda)
...
>>> class Sum:
...     def __init__(self, val):                 # Callable instances
...         self.val = val
...     def __call__(self, arg):
...         return self.val + arg
...
>>> class Product:
...     def __init__(self, val):                 # Bound methods
...         self.val = val
...     def method(self, arg):
...         return self.val * arg
...
>>> sobject = Sum(2)
>>> pobject = Product(3)
>>> actions = [square, sobject, pobject.method]  # Function, instance, method

>>> for act in actions:                          # All 3 called same way
...     print(act(5))                            # Call any 1-arg callable
...
25
7
15
>>> actions[-1](5)                               # Index, comprehensions, maps
15
>>> [act(5) for act in actions]
[25, 7, 15]
>>> list(map(lambda act: act(5), actions))
[25, 7, 15]

Technically speaking, classes belong in the callable objects category too, but we normally call them to generate instances rather than to do actual work, as shown here:

>>> class Negate:
...     def __init__(self, val):                 # Classes are callables too
...         self.val = -val                      # But called for object, not work
...     def __repr__(self):                      # Instance print format
...         return str(self.val)
...
>>> actions = [square, sobject, pobject.method, Negate]     # Call a class too
>>> for act in actions:
...     print(act(5))
...
25
7
15
-5
>>> [act(5) for act in actions]                     # Runs __repr__ not __str__!
[25, 7, 15, −5]

>>> table = {act(5): act for act in actions}        # 3.0 dict comprehension
>>> for (key, value) in table.items():
...     print('{0:2} => {1}'.format(key, value))    # 2.6/3.0 str.format
...
-5 => <class '__main__.Negate'>
25 => <function square at 0x025D4978>
15 => <bound method Product.method of <__main__.Product object at 0x025D0F90>>
 7 => <__main__.Sum object at 0x025D0F70>

As you can see, bound methods, and Python’s callable objects model in general, are some of the many ways that Python’s design makes for an incredibly flexible language.

You should now understand the method object model. For other examples of bound methods at work, see the upcoming sidebar as well as the prior chapter’s discussion of callback handlers in the section on the method __call__.

Multiple Inheritance: “Mix-in” Classes

Many class-based designs call for combining disparate sets of methods. In a class statement, more than one superclass can be listed in parentheses in the header line. When you do this, you use something called multiple inheritance—the class and its instances inherit names from all the listed superclasses.

When searching for an attribute, Python’s inheritance search traverses all superclasses in the class header from left to right until a match is found. Technically, because any of the superclasses may have superclasses of its own, this search can be a bit more complex for larger class tress:

  • In classic classes (the default until Python 3.0), the attribute search proceeds depth-first all the way to the top of the inheritance tree, and then from left to right.

  • In new-style classes (and all classes in 3.0), the attribute search proceeds across by tree levels, in a more breadth-first fashion (see the new-style class discussion in the next chapter).

In either model, though, when a class has multiple superclasses, they are searched from left to right according to the order listed in the class statement header lines.

In general, multiple inheritance is good for modeling objects that belong to more than one set. For instance, a person may be an engineer, a writer, a musician, and so on, and inherit properties from all such sets. With multiple inheritance, objects obtain the union of the behavior in all their superclasses.

Perhaps the most common way multiple inheritance is used is to “mix in” general-purpose methods from superclasses. Such superclasses are usually called mix-in classes—they provide methods you add to application classes by inheritance. In a sense, mix-in classes are similar to modules: they provide packages of methods for use in their client subclasses. Unlike simple functions in modules, though, methods in mix-ins also have access to the self instance, for using state information and other methods. The next section demonstrates one common use case for such tools.

Coding Mix-in Display Classes

As we’ve seen, Python’s default way to print a class instance object isn’t incredibly useful:

>>> class Spam:
...     def __init__(self):                   # No __repr__ or __str__
...         self.data1 = "food"
...
>>> X = Spam()
>>> print(X)                                  # Default: class, address
<__main__.Spam object at 0x00864818>          # Displays "instance" in Python 2.6

As you saw in Chapter 29 when studying operator overloading, you can provide a __str__ or __repr__ method to implement a custom string representation of your own. But, rather than coding one of these in each and every class you wish to print, why not code it once in a general-purpose tool class and inherit it in all your classes?

That’s what mix-ins are for. Defining a display method in a mix-in superclass once enables us to reuse it anywhere we want to see a custom display format. We’ve already seen tools that do related work:

  • Chapter 27’s AttrDisplay class formatted instance attributes in a generic __str__ method, but it did not climb class trees and was used in single-inheritance mode only.

  • Chapter 28’s classtree.py module defined functions for climbing and sketching class trees, but it did not display object attributes along the way and was not architected as an inheritable class.

Here, we’re going to revisit these examples’ techniques and expand upon them to code a set of three mix-in classes that serve as generic display tools for listing instance attributes, inherited attributes, and attributes on all objects in a class tree. We’ll also use our tools in multiple-inheritance mode and deploy coding techniques that make classes better suited to use as generic tools.

Listing instance attributes with __dict__

Let’s get started with the simple case—listing attributes attached to an instance. The following class, coded in the file lister.py, defines a mix-in called ListInstance that overloads the __str__ method for all classes that include it in their header lines. Because this is coded as a class, ListInstance is a generic tool whose formatting logic can be used for instances of any subclass:

# File lister.py

class ListInstance:
    """
    Mix-in class that provides a formatted print() or str() of
    instances via inheritance of __str__, coded here; displays
    instance attrs only; self is the instance of lowest class;
    uses __X names to avoid clashing with client's attrs
    """
    def __str__(self):
        return '<Instance of %s, address %s:
%s>' % (
                           self.__class__.__name__,         # My class's name
                           id(self),                        # My address
                           self.__attrnames())              # name=value list
    def __attrnames(self):
        result = ''
        for attr in sorted(self.__dict__):                  # Instance attr dict
            result += '	name %s=%s
' % (attr, self.__dict__ [attr])
        return result

ListInstance uses some previously explored tricks to extract the instance’s class name and attributes:

  • Each instance has a built-in __class__ attribute that references the class from which it was created, and each class has a __name__ attribute that references the name in the header, so the expression self.__class__.__name__ fetches the name of an instance’s class.

  • This class does most of its work by simply scanning the instance’s attribute dictionary (remember, it’s exported in __dict__) to build up a string showing the names and values of all instance attributes. The dictionary’s keys are sorted to finesse any ordering differences across Python releases.

In these respects, ListInstance is similar to Chapter 27’s attribute display; in fact, it’s largely just a variation on a theme. Our class here uses two additional techniques, though:

  • It displays the instance’s memory address by calling the id built-function, which returns any object’s address (by definition, a unique object identifier, which will be useful in later mutations of this code).

  • It uses the pseudoprivate naming pattern for its worker method: __attrnames. As we learned earlier in his chapter, Python automatically localizes any such name to its enclosing class by expanding the attribute name to include the class name (in this case, it becomes _ListInstance__attrnames). This holds true for both class attributes (like methods) and instance attributes attached to self. This behavior is useful in a general tool like this, as it ensures that its names don’t clash with any names used in its client subclasses.

Because ListInstance defines a __str__ operator overloading method, instances derived from this class display their attributes automatically when printed, giving a bit more information than a simple address. Here is the class in action, in single-inheritance mode (this code works the same in both Python 3.0 and 2.6):

>>> from lister import ListInstance
>>> class Spam(ListInstance):                    # Inherit a __str__ method
...     def __init__(self):
...         self.data1 = 'food'
...
>>> x = Spam()
>>> print(x)                                     # print() and str() run __str__
<Instance of Spam, address 40240880:
        name data1=food
>

You can also fetch the listing output as a string without printing it with str, and interactive echoes still use the default format:

>>> str(x)
'<Instance of Spam, address 40240880:
	name data1=food
>'
>>> x                                            # The __repr__ still is a default
<__main__.Spam object at 0x026606F0>

The ListInstance class is useful for any classes you write—even classes that already have one or more superclasses. This is where multiple inheritance comes in handy: by adding ListInstance to the list of superclasses in a class header (i.e., mixing it in), you get its __str__ “for free” while still inheriting from the existing superclass(es). The file testmixin.py demonstrates:

# File testmixin.py

from lister import *                  # Get lister tool classes

class Super:
    def __init__(self):               # Superclass __init__
        self.data1 = 'spam'           # Create instance attrs
    def ham(self):
        pass

class Sub(Super, ListInstance):       # Mix in ham and a __str__
    def __init__(self):               # listers have access to self
        Super.__init__(self)
        self.data2 = 'eggs'           # More instance attrs
        self.data3 = 42
    def spam(self):                   # Define another method here
        pass

if __name__ == '__main__':
    X = Sub()
    print(X)                          # Run mixed-in __str__

Here, Sub inherits names from both Super and ListInstance; it’s a composite of its own names and names in both its superclasses. When you make a Sub instance and print it, you automatically get the custom representation mixed in from ListInstance (in this case, this script’s output is the same under both Python 3.0 and 2.6, except for object addresses):

C:misc> C:python30python testmixin.py
<Instance of Sub, address 40962576:
        name data1=spam
        name data2=eggs
        name data3=42
>

ListInstance works in any class it’s mixed into because self refers to an instance of the subclass that pulls this class in, whatever that may be. In a sense, mix-in classes are the class equivalent of modules—packages of methods useful in a variety of clients. For example, here is Lister working again in single-inheritance mode on a different class’s instances, with import and attributes set outside the class:

>>> import lister
>>> class C(lister.ListInstance): pass
...
>>> x = C()
>>> x.a = 1; x.b = 2; x.c = 3
>>> print(x)
<Instance of C, address 40961776:
        name a=1
        name b=2
        name c=3
>

Besides the utility they provide, mix-ins optimize code maintenance, like all classes do. For example, if you later decide to extend ListInstance’s __str__ to also print all the class attributes that an instance inherits, you’re safe; because it’s an inherited method, changing __str__ automatically updates the display of each subclass that imports the class and mixes it in. Since it’s now officially “later,” let’s move on to the next section to see what such an extension might look like.

Listing inherited attributes with dir

As it is, our Lister mix-in displays instance attributes only (i.e., names attached to the instance object itself). It’s trivial to extend the class to display all the attributes accessible from an instance, though—both its own and those it inherits from its classes. The trick is to use the dir built-in function instead of scanning the instance’s __dict__ dictionary; the latter holds instance attributes only, but the former also collects all inherited attributes in Python 2.2 and later.

The following mutation codes this scheme; I’ve renamed it to facilitate simple testing, but if this were to replace the original version, all existing clients would pick up the new display automatically:

# File lister.py, continued

class ListInherited:
    """
    Use dir() to collect both instance attrs and names
    inherited from its classes; Python 3.0 shows more
    names than 2.6 because of the implied object superclass
    in the new-style class model; getattr() fetches inherited
    names not in self.__dict__; use __str__, not __repr__,
    or else this loops when printing bound methods!
    """
    def __str__(self):
        return '<Instance of %s, address %s:
%s>' % (
                           self.__class__.__name__,         # My class's name
                           id(self),                        # My address
                           self.__attrnames())              # name=value list
    def __attrnames(self):
        result = ''
        for attr in dir(self):                              # Instance dir()
            if attr[:2] == '__' and attr[-2:] == '__':      # Skip internals
                result += '	name %s=<>
' % attr
            else:
                result += '	name %s=%s
' % (attr, getattr(self, attr))
        return result

Notice that this code skips __X__ names’ values; most of these are internal names that we don’t generally care about in a generic listing like this. This version also must use the getattr built-in function to fetch attributes by name string instead of using instance attribute dictionary indexing—getattr employs the inheritance search protocol, and some of the names we’re listing here are not stored on the instance itself.

To test the new version, change the testmixin.py file to use this new class instead:

class Sub(Super, ListInherited):                            # Mix in a __str__

This file’s output varies per release. In Python 2.6, we get the following; notice the name mangling at work in the lister’s method name (I shortened its full value display to fit on this page):

C:misc> c:python26python testmixin.py
<Instance of Sub, address 40073136:
        name _ListInherited__attrnames=<bound method Sub.__attrnames of <...more...>>
        name __doc__=<>
        name __init__=<>
        name __module__=<>
        name __str__=<>
        name data1=spam
        name data2=eggs
        name data3=42
        name ham=<bound method Sub.ham of <__main__.Sub instance at 0x026377B0>>
        name spam=<bound method Sub.spam of <__main__.Sub instance at 0x026377B0>>
>

In Python 3.0, more attributes are displayed because all classes are “new-style” and inherit names from the implied object superclass (more on this in Chapter 31). Because so many names are inherited from the default superclass, I’ve omitted many here; run this on your own for the full listing:

C:misc> c:python30python testmixin.py
<Instance of Sub, address 40831792:
        name _ListInherited__attrnames=<bound method Sub.__attrnames of <...more...>>
        name __class__=<>
        name __delattr__=<>
        name __dict__=<>
        name __doc__=<>
        name __eq__=<>
        ...more names omitted...
        name __repr__=<>
        name __setattr__=<>
        name __sizeof__=<>
        name __str__=<>
        name __subclasshook__=<>
        name __weakref__=<>
        name data1=spam
        name data2=eggs
        name data3=42
        name ham=<bound method Sub.ham of <__main__.Sub object at 0x026F0B30>>
        name spam=<bound method Sub.spam of <__main__.Sub object at 0x026F0B30>>
>

One caution here—now that we’re displaying inherited methods too, we have to use __str__ instead of __repr__ to overload printing. With __repr__, this code will loop—displaying the value of a method triggers the __repr__ of the method’s class, in order to display the class. That is, if the lister’s __repr__ tries to display a method, displaying the method’s class will trigger the lister’s __repr__ again. Subtle, but true! Change __str__ to __repr__ here to see this for yourself. If you must use __repr__ in such a context, you can avoid the loops by using isinstance to compare the type of attribute values against types.MethodType in the standard library, to know which items to skip.

Listing attributes per object in class trees

Let’s code one last extension. As it is, our lister doesn’t tell us which class an inherited name comes from. As we saw in the classtree.py example near the end of Chapter 28, though, it’s straightforward to climb class inheritance trees in code. The following mix-in class makes use of this same technique to display attributes grouped by the classes they live in—it sketches the full class tree, displaying attributes attached to each object along the way. It does so by traversing the inheritance tree from an instance’s __class__ to its class, and then from the class’s __bases__ to all superclasses recursively, scanning object __dicts__s along the way:

# File lister.py, continued

class ListTree:
    """
    Mix-in that returns an __str__ trace of the entire class
    tree and all its objects' attrs at and above self;
    run by print(), str() returns constructed string;
    uses __X attr names to avoid impacting clients;
    uses generator expr to recurse to superclasses;
    uses str.format() to make substitutions clearer
    """
    def __str__(self):
        self.__visited = {}
        return '<Instance of {0}, address {1}:
{2}{3}>'.format(
                           self.__class__.__name__,
                           id(self),
                           self.__attrnames(self, 0),
                           self.__listclass(self.__class__, 4))

    def __listclass(self, aClass, indent):
        dots = '.' * indent
        if aClass in self.__visited:
            return '
{0}<Class {1}:, address {2}: (see above)>
'.format(
                           dots,
                           aClass.__name__,
                           id(aClass))
        else:
            self.__visited[aClass] = True
            genabove = (self.__listclass(c, indent+4) for c in aClass.__bases__)
            return '
{0}<Class {1}, address {2}:
{3}{4}{5}>
'.format(
                           dots,
                           aClass.__name__,
                           id(aClass),
                           self.__attrnames(aClass, indent),
                           ''.join(genabove),
                           dots)

    def __attrnames(self, obj, indent):
        spaces = ' ' * (indent + 4)
        result = ''
        for attr in sorted(obj.__dict__):
            if attr.startswith('__') and attr.endswith('__'):
                result += spaces + '{0}=<>
'.format(attr)
            else:
                result += spaces + '{0}={1}
'.format(attr, getattr(obj, attr))
        return result

Note the use of a generator expression to direct the recursive calls for superclasses; it’s activated by the nested string join method. Also see how this version uses the Python 3.0 and 2.6 string format method instead of % formatting expressions, to make substitutions clearer; when many substitutions are applied like this, explicit argument numbers may make the code easier to decipher. In short, in this version we exchange the first of the following lines for the second:

        return '<Instance of %s, address %s:
%s%s>' % (...)          # Expression
        return '<Instance of {0}, address {1}:
{2}{3}>'.format(...)  # Method

Now, change testmixin.py to inherit from this new class again to test:

class Sub(Super, ListTree):         # Mix in a __str__

The file’s tree-sketcher output in Python 2.6 is then as follows:

C:misc> c:python26python testmixin.py
<Instance of Sub, address 40728496:
    _ListTree__visited={}
    data1=spam
    data2=eggs
    data3=42

....<Class Sub, address 40701168:
        __doc__=<>
        __init__=<>
        __module__=<>
        spam=<unbound method Sub.spam>

........<Class Super, address 40701120:
            __doc__=<>
            __init__=<>
            __module__=<>
            ham=<unbound method Super.ham>
........>

........<Class ListTree, address 40700688:
            _ListTree__attrnames=<unbound method ListTree.__attrnames>
            _ListTree__listclass=<unbound method ListTree.__listclass>
            __doc__=<>
            __module__=<>
            __str__=<>
........>
....>
>

Notice in this output how methods are unbound now under 2.6, because we fetch them from classes directly, instead of from instances. Also observe how the lister’s __visited table has its name mangled in the instance’s attribute dictionary; unless we’re very unlucky, this won’t clash with other data there.

Under Python 3.0, we get extra attributes and superclasses again. Notice that unbound methods are simple functions in 3.0, as described in an earlier note in this chapter (and that again, I’ve deleted most built-in attributes in object to save space here; run this on your own for the complete listing):

C:misc> c:python30python testmixin.py
<Instance of Sub, address 40635216:
    _ListTree__visited={}
    data1=spam
    data2=eggs
    data3=42

....<Class Sub, address 40914752:
        __doc__=<>
        __init__=<>
        __module__=<>
        spam=<function spam at 0x026D53D8>

........<Class Super, address 40829952:
            __dict__=<>
            __doc__=<>
            __init__=<>
            __module__=<>
            __weakref__=<>
            ham=<function ham at 0x026D5228>

............<Class object, address 505114624:
                __class__=<>
                __delattr__=<>
                __doc__=<>
                __eq__=<>
                ...more omitted...
                __repr__=<>
                __setattr__=<>
                __sizeof__=<>
                __str__=<>
                __subclasshook__=<>
............>
........>

........<Class ListTree, address 40829496:
            _ListTree__attrnames=<function __attrnames at 0x026D5660>
            _ListTree__listclass=<function __listclass at 0x026D56A8>
            __dict__=<>
            __doc__=<>
            __module__=<>
            __str__=<>
            __weakref__=<>

............<Class object:, address 505114624: (see above)>
........>
....>
>

This version avoids listing the same class object twice by keeping a table of classes visited so far (this is why an object’s id is included—to serve as a key for a previously displayed item). Like the transitive module reloader of Chapter 24, a dictionary works to avoid repeats and cycles here because class objects may be dictionary keys; a set would provide similar functionality.

This version also takes care to avoid large internal objects by skipping __X__ names again. If you comment out the test for these names, their values will display normally. Here’s an excerpt from the output in 2.6 with this temporary change made (it’s much larger in its entirety, and it gets even worse in 3.0, which is why these names are probably better skipped!):

C:misc> c:python26python testmixin.py
...more omitted...

........<Class ListTree, address 40700688:
            _ListTree__attrnames=<unbound method ListTree.__attrnames>
            _ListTree__listclass=<unbound method ListTree.__listclass>
            __doc__=
    Mix-in that returns the __str__ trace of the entire class
    tree and all its objects' attrs at and above self;
    run by print, str returns constructed string;
    uses __X attr names to avoid impacting clients;
    uses generator expr to recurse to superclasses;
    uses str.format() to make substitutions clearer

            __module__=lister
            __str__=<unbound method ListTree.__str__>
........>

For more fun, try mixing this class into something more substantial, like the Button class of Python’s tkinter GUI toolkit module. In general, you’ll want to name ListTree first (leftmost) in a class header, so its __str__ is picked up; Button has one, too, and the leftmost superclass is searched first in multiple inheritance. The output of the following is fairly massive (18K characters), so run this code on your own to see the full listing (and if you’re using Python 2.6, recall that you should use Tkinter for the module name instead of tkinter):

>>> from lister import ListTree
>>> from tkinter import Button                  # Both classes have a __str__
>>> class MyButton(ListTree, Button): pass      # ListTree first: use its __str__
...
>>> B = MyButton(text='spam')
>>> open('savetree.txt', 'w').write(str(B))     # Save to a file for later viewing
18247
>>> print(B)                                    # Print the display here
<Instance of MyButton, address 44355632:
    _ListTree__visited={}
    _name=44355632
    _tclCommands=[]
    ...much more omitted...
>

Of course, there’s much more we could do here (sketching the tree in a GUI might be a natural next step), but we’ll leave further work as a suggested exercise. We’ll also extend this code in the exercises at the end of this part of the book, to list superclass names in parentheses at the start of instance and class displays.

The main point here is that OOP is all about code reuse, and mix-in classes are a powerful example. Like almost everything else in programming, multiple inheritance can be a useful device when applied well. In practice, though, it is an advanced feature and can become complicated if used carelessly or excessively. We’ll revisit this topic as a gotcha at the end of the next chapter. In that chapter, we’ll also meet the new-style class model, which modifies the search order for one special multiple inheritance case. For more ideas, see also Python manuals for the class.mro() new-style class object method, which returns a list giving the class tree search order used by inheritance; this could be used by a class lister to show attribute sources.

Note

Supporting slots: Because they scan instance dictionaries, the ListInstance and ListTree classes presented here don’t directly support attributes stored in slots—a newer and relatively rarely used option we’ll meet in the next chapter, where instance attributes are declared in a __slots__ class attribute. For example, if in textmixin.py we assign __slots__=['data1'] in Super and __slots__=['data3'] in Sub, only the data2 attribute is displayed in the instance by these two lister classes; ListTree also displays data1 and data3, but as attributes of the Super and Sub class objects and with a special format for their values (technically, they are class-level descriptors).

To better support slot attributes in these classes, change the __dict__ scanning loops to also iterate through __slots__ lists using code the next chapter will present, and use the getattr built-in function to fetch values instead of __dict__ indexing (ListTree already does). Since instances inherit only the lowest class’s __slots__, you may also need to come up with a policy when __slots__ lists appear in multiple superclasses (ListTree already displays them as class attributes). ListInherited is immune to all this, because dir results combine both __dict__ names and all classes’ __slots__ names.

Alternatively, as a policy we could simply let our code handle slot-based attributes as it currently does, rather than complicating it for a rare, advanced feature. Slots and normal instance attributes are different kinds of names. We’ll investigate slots further in the next chapter; I omitted addressing them in these examples to avoid a forward dependency (not counting this note, of course!)—not exactly a valid design goal, but reasonable for a book.

Classes Are Objects: Generic Object Factories

Sometimes, class-based designs require objects to be created in response to conditions that can’t be predicted when a program is written. The factory design pattern allows such a deferred approach. Due in large part to Python’s flexibility, factories can take multiple forms, some of which don’t seem special at all.

Because classes are objects, it’s easy to pass them around a program, store them in data structures, and so on. You can also pass classes to functions that generate arbitrary kinds of objects; such functions are sometimes called factories in OOP design circles. Factories are a major undertaking in a strongly typed language such as C++ but are almost trivial to implement in Python. The call syntax we met in Chapter 18 can call any class with any number of constructor arguments in one step to generate any sort of instance:[71]

def factory(aClass, *args):           # Varargs tuple
    return aClass(*args)              # Call aClass (or apply in 2.6 only)

class Spam:
    def doit(self, message):
        print(message)

class Person:
    def __init__(self, name, job):
        self.name = name
        self.job  = job

object1 = factory(Spam)                      # Make a Spam object
object2 = factory(Person, "Guido", "guru")   # Make a Person object

In this code, we define an object generator function called factory. It expects to be passed a class object (any class will do) along with one or more arguments for the class’s constructor. The function uses special “varargs” call syntax to call the function and return an instance.

The rest of the example simply defines two classes and generates instances of both by passing them to the factory function. And that’s the only factory function you’ll ever need to write in Python; it works for any class and any constructor arguments.

One possible improvement worth noting is that to support keyword arguments in constructor calls, the factory can collect them with a **args argument and pass them along in the class call, too:

def factory(aClass, *args, **kwargs):        # +kwargs dict
    return aClass(*args, **kwargs)           # Call aClass

By now, you should know that everything is an “object” in Python, including things like classes, which are just compiler input in languages like C++. However, as mentioned at the start of this part of the book, only objects derived from classes are OOP objects in Python.

Why Factories?

So what good is the factory function (besides providing an excuse to illustrate class objects in this book)? Unfortunately, it’s difficult to show applications of this design pattern without listing much more code than we have space for here. In general, though, such a factory might allow code to be insulated from the details of dynamically configured object construction.

For instance, recall the processor example presented in the abstract in Chapter 25, and then again as a composition example in this chapter. It accepts reader and writer objects for processing arbitrary data streams. The original version of this example manually passed in instances of specialized classes like FileWriter and SocketReader to customize the data streams being processed; later, we passed in hardcoded file, stream, and formatter objects. In a more dynamic scenario, external devices such as configuration files or GUIs might be used to configure the streams.

In such a dynamic world, we might not be able to hardcode the creation of stream interface objects in our scripts, but might instead create them at runtime according to the contents of a configuration file.

For example, the file might simply give the string name of a stream class to be imported from a module, plus an optional constructor call argument. Factory-style functions or code might come in handy here because they would allow us to fetch and pass in classes that are not hardcoded in our program ahead of time. Indeed, those classes might not even have existed at all when we wrote our code:

classname = ...parse from config file...
classarg  = ...parse from config file...

import streamtypes                           # Customizable code
aclass = getattr(streamtypes, classname)     # Fetch from module
reader = factory(aclass, classarg)           # Or aclass(classarg)
processor(reader, ...)

Here, the getattr built-in is again used to fetch a module attribute given a string name (it’s like saying obj.attr, but attr is a string). Because this code snippet assumes a single constructor argument, it doesn’t strictly need factory or apply—we could make an instance with just aclass(classarg). They may prove more useful in the presence of unknown argument lists, however, and the general factory coding pattern can improve the code’s flexibility.

In this chapter, we’ve seen inheritance, composition, delegation, multiple inheritance, bound methods, and factories—all common patterns used to combine classes in Python programs. We’ve really only scratched the surface here in the design patterns domain, though. Elsewhere in this book you’ll find coverage of other design-related topics, such as:

For more details on design patterns, though, we’ll delegate to other resources on OOP at large. Although patterns are important in OOP work, and are often more natural in Python than other languages, they are not specific to Python itself.

Chapter Summary

In this chapter, we sampled common ways to use and combine classes to optimize their reusability and factoring benefits—what are usually considered design issues that are often independent of any particular programming language (though Python can make them easier to implement). We studied delegation (wrapping objects in proxy classes), composition (controlling embedded objects), and inheritance (acquiring behavior from other classes), as well as some more esoteric concepts such as pseudoprivate attributes, multiple inheritance, bound methods, and factories.

The next chapter ends our look at classes and OOP by surveying more advanced class-related topics; some of its material may be of more interest to tool writers than application programmers, but it still merits a review by most people who will do OOP in Python. First, though, another quick chapter quiz.

Test Your Knowledge: Quiz

  1. What is multiple inheritance?

  2. What is delegation?

  3. What is composition?

  4. What are bound methods?

  5. What are pseudoprivate attributes used for?

Test Your Knowledge: Answers

  1. Multiple inheritance occurs when a class inherits from more than one superclass; it’s useful for mixing together multiple packages of class-based code. The left-to-right order in class statement headers determines the order of attribute searches.

  2. Delegation involves wrapping an object in a proxy class, which adds extra behavior and passes other operations to the wrapped object. The proxy retains the interface of the wrapped object.

  3. Composition is a technique whereby a controller class embeds and directs a number of objects, and provides an interface all its own; it’s a way to build up larger structures with classes.

  4. Bound methods combine an instance and a method function; you can call them without passing in an instance object explicitly because the original instance is still available.

  5. Pseudoprivate attributes (whose names begin with two leading underscores: __X) are used to localize names to the enclosing class. This includes both class attributes like methods defined inside the class, and self instance attributes assigned inside the class. Such names are expanded to include the class name, which makes them unique.



[69] This tends to scare people with a C++ background unnecessarily. In Python, it’s even possible to change or completely delete a class method at runtime. On the other hand, almost nobody ever does this in practical programs. As a scripting language, Python is more about enabling than restricting. Also, recall from our discussion of operator overloading in Chapter 29 that __getattr__ and __setattr__ can be used to emulate privacy, but are generally not used for this purpose in practice. More on this when we code a more realistic privacy decorator Chapter 38.

[70] See the discussion of static and class methods in Chapter 31 for an optional exception to this rule. Like bound methods, static methods can masquerade as basic functions because they do not expect instances when called. Python supports three kinds of class methods—instance, static, and class—and 3.0 allows simple functions in classes, too.

[71] Actually, this syntax can invoke any callable object, including functions, classes, and methods. Hence, the factory function here can also run any callable object, not just a class (despite the argument name). Also, as we learned in Chapter 18, Python 2.6 has an alternative to aClass(*args): the apply(aClass, args) built-in call, which has been removed in Python 3.0 because of its redundancy and limitations.

..................Content has been hidden....................

You can't read the all page of ebook, please click here login for view all page.
Reset