So far in this part of the book, we’ve concentrated on using Python’s OOP tool, the class. But OOP is also about design issues—i.e., how to use classes to model useful objects. This chapter will touch on a few core OOP ideas and present some additional examples that are more realistic than those shown so far.
Along the way, we’ll code some common OOP design patterns in Python, such as inheritance, composition, delegation, and factories. We’ll also investigate some design-focused class concepts, such as pseudoprivate attributes, multiple inheritance, and bound methods. Many of the design terms mentioned here require more explanation than I can provide in this book; if this material sparks your curiosity, I suggest exploring a text on OOP design or design patterns as a next step.
Let’s begin with a review—Python’s implementation of OOP can be summarized by three ideas:
Inheritance is based on attribute lookup in Python (in
X.name
expressions).
In X.method
, the
meaning of method
depends on
the type (class) of X
.
Methods and operators implement behavior; data hiding is a convention by default.
By now, you should have a good feel for what inheritance is all about in Python. We’ve also talked about Python’s polymorphism a few times already; it flows from Python’s lack of type declarations. Because attributes are always resolved at runtime, objects that implement the same interfaces are interchangeable; clients don’t need to know what sorts of objects are implementing the methods they call.
Encapsulation means packaging in Python—that is, hiding implementation details behind an object’s interface. It does not mean enforced privacy, though that can be implemented with code, as we’ll see in Chapter 38. Encapsulation allows the implementation of an object’s interface to be changed without impacting the users of that object.
Some OOP languages also define polymorphism to mean overloading functions based on the type signatures of their arguments. But because there are no type declarations in Python, this concept doesn’t really apply; polymorphism in Python is based on object interfaces, not types.
You can try to overload methods by their argument lists, like this:
class C: def meth(self, x): ... def meth(self, x, y, z): ...
This code will run, but because the def
simply assigns an object to a name in
the class’s scope, the last definition of the method function is the
only one that will be retained (it’s just as if you say X = 1
and then X
= 2
; X
will be 2
).
Type-based selections can always be coded using the type-testing ideas we met in Chapters 4 and 9, or the argument list tools introduced in Chapter 18:
class C: def meth(self, *args): if len(args) == 1: ... elif type(arg[0]) == int: ...
You normally shouldn’t do this, though—as described in Chapter 16, you should write your code to expect an object interface, not a specific data type. That way, it will be useful for a broader category of types and applications, both now and in the future:
class C:
def meth(self, x):
x.operation() # Assume x does the right thing
It’s also generally considered better to use distinct method names for distinct operations, rather than relying on call signatures (no matter what language you code in).
Although Python’s object model is straightforward, much of the art in OOP is in the way we combine classes to achieve a program’s goals. The next section begins a tour of some of the ways larger programs use classes to their advantage.
We’ve explored the mechanics of inheritance in depth already, but I’d like to show you an example of how it can be used to model real-world relationships. From a programmer’s point of view, inheritance is kicked off by attribute qualifications, which trigger searches for names in instances, their classes, and then any superclasses. From a designer’s point of view, inheritance is a way to specify set membership: a class defines a set of properties that may be inherited and customized by more specific sets (i.e., subclasses).
To illustrate, let’s put that pizza-making robot we talked about at the start of this part of the book to work. Suppose we’ve decided to explore alternative career paths and open a pizza restaurant. One of the first things we’ll need to do is hire employees to serve customers, prepare the food, and so on. Being engineers at heart, we’ve decided to build a robot to make the pizzas; but being politically and cybernetically correct, we’ve also decided to make our robot a full-fledged employee with a salary.
Our pizza shop team can be defined by the four classes in the
example file, employees.py. The most general class,
Employee
, provides common behavior
such as bumping up salaries (giveRaise
) and printing (__repr__
). There are two kinds of employees,
and so two subclasses of Employee
:
Chef
and Server
. Both override the inherited work
method to print more specific messages.
Finally, our pizza robot is modeled by an even more specific class:
PizzaRobot
is a kind of Chef
, which is a kind of Employee
. In OOP terms, we call these
relationships “is-a” links: a robot is a chef, which is a(n) employee.
Here’s the employees.py
file:
class Employee: def __init__(self, name, salary=0): self.name = name self.salary = salary def giveRaise(self, percent): self.salary = self.salary + (self.salary * percent) def work(self): print(self.name, "does stuff") def __repr__(self): return "<Employee: name=%s, salary=%s>" % (self.name, self.salary) class Chef(Employee): def __init__(self, name): Employee.__init__(self, name, 50000) def work(self): print(self.name, "makes food") class Server(Employee): def __init__(self, name): Employee.__init__(self, name, 40000) def work(self): print(self.name, "interfaces with customer") class PizzaRobot(Chef): def __init__(self, name): Chef.__init__(self, name) def work(self): print(self.name, "makes pizza") if __name__ == "__main__": bob = PizzaRobot('bob') # Make a robot named bob print(bob) # Run inherited __repr__ bob.work() # Run type-specific action bob.giveRaise(0.20) # Give bob a 20% raise print(bob); print() for klass in Employee, Chef, Server, PizzaRobot: obj = klass(klass.__name__) obj.work()
When we run the self-test code included in this module, we
create a pizza-making robot named bob
, which inherits names from three
classes: PizzaRobot
, Chef
, and Employee
. For instance, printing bob
runs the Employee.__repr__
method, and giving
bob
a raise invokes Employee.giveRaise
because that’s where the
inheritance search finds that method:
C:pythonexamples> python employees.py
<Employee: name=bob, salary=50000>
bob makes pizza
<Employee: name=bob, salary=60000.0>
Employee does stuff
Chef makes food
Server interfaces with customer
PizzaRobot makes pizza
In a class hierarchy like this, you can usually make instances
of any of the classes, not just the ones at the bottom. For instance,
the for
loop in this module’s
self-test code creates instances of all four classes; each responds
differently when asked to work because the work
method is different in each. Really,
these classes just simulate real-world objects; work
prints a message for the time being,
but it could be expanded to do real work later.
The notion of composition was introduced in Chapter 25. From a programmer’s point of view, composition involves embedding other objects in a container object, and activating them to implement container methods. To a designer, composition is another way to represent relationships in a problem domain. But, rather than set membership, composition has to do with components—parts of a whole.
Composition also reflects the relationships between parts, called a “has-a” relationships. Some OOP design texts refer to composition as aggregation (or distinguish between the two terms by using aggregation to describe a weaker dependency between container and contained); in this text, a “composition” simply refers to a collection of embedded objects. The composite class generally provides an interface all its own and implements it by directing the embedded objects.
Now that we’ve implemented our employees, let’s put them in the pizza shop and let them get busy. Our pizza shop is a composite object: it has an oven, and it has employees like servers and chefs. When a customer enters and places an order, the components of the shop spring into action—the server takes the order, the chef makes the pizza, and so on. The following example (the file pizzashop.py) simulates all the objects and relationships in this scenario:
from employees import PizzaRobot, Server class Customer: def __init__(self, name): self.name = name def order(self, server): print(self.name, "orders from", server) def pay(self, server): print(self.name, "pays for item to", server) class Oven: def bake(self): print("oven bakes") class PizzaShop: def __init__(self): self.server = Server('Pat') # Embed other objects self.chef = PizzaRobot('Bob') # A robot named bob self.oven = Oven() def order(self, name): customer = Customer(name) # Activate other objects customer.order(self.server) # Customer orders from server self.chef.work() self.oven.bake() customer.pay(self.server) if __name__ == "__main__": scene = PizzaShop() # Make the composite scene.order('Homer') # Simulate Homer's order print('...') scene.order('Shaggy') # Simulate Shaggy's order
The PizzaShop
class is a
container and controller; its constructor makes and embeds instances
of the employee classes we wrote in the last section, as well as an
Oven
class defined here. When this
module’s self-test code calls the PizzaShop
order
method, the embedded objects are asked to carry out
their actions in turn. Notice that we make a new Customer
object for each order, and we pass
on the embedded Server
object to
Customer
methods; customers come
and go, but the server is part of the pizza shop composite. Also
notice that employees are still involved in an inheritance
relationship; composition and inheritance are complementary
tools.
When we run this module, our pizza shop handles two orders—one from Homer, and then one from Shaggy:
C:pythonexamples> python pizzashop.py
Homer orders from <Employee: name=Pat, salary=40000>
Bob makes pizza
oven bakes
Homer pays for item to <Employee: name=Pat, salary=40000>
...
Shaggy orders from <Employee: name=Pat, salary=40000>
Bob makes pizza
oven bakes
Shaggy pays for item to <Employee: name=Pat, salary=40000>
Again, this is mostly just a toy simulation, but the objects and interactions are representative of composites at work. As a rule of thumb, classes can represent just about any objects and relationships you can express in a sentence; just replace nouns with classes, and verbs with methods, and you’ll have a first cut at a design.
For a more realistic composition example, recall the generic data stream processor function we partially coded in the introduction to OOP in Chapter 25:
def processor(reader, converter, writer): while 1: data = reader.read() if not data: break data = converter(data) writer.write(data)
Rather than using a simple function here, we might code this as a class that uses composition to do its work to provide more structure and support inheritance. The following file, streams.py, demonstrates one way to code the class:
class Processor:
def __init__(self, reader, writer):
self.reader = reader
self.writer = writer
def process(self):
while 1:
data = self.reader.readline()
if not data: break
data = self.converter(data)
self.writer.write(data)
def converter(self, data):
assert False, 'converter must be defined' # Or raise exception
This class defines a converter
method that it expects
subclasses to fill in; it’s an example of the abstract superclass model we outlined in
Chapter 28 (more on assert
in Part VII). Coded this way, reader
and writer
objects are embedded within the
class instance (composition), and we supply the
conversion logic in a subclass rather than passing in a converter
function (inheritance). The file converters.py shows how:
from streams import Processor class Uppercase(Processor): def converter(self, data): return data.upper() if __name__ == '__main__': import sys obj = Uppercase(open('spam.txt'), sys.stdout) obj.process()
Here, the Uppercase
class
inherits the stream-processing loop logic (and anything else that
may be coded in its superclasses). It needs to define only what is
unique about it—the data
conversion logic. When this file is run, it makes and runs an
instance that reads from the file spam.txt and writes the uppercase
equivalent of that file to the stdout
stream:
C:lp4e>type spam.txt
spam Spam SPAM! C:lp4e>python converters.py
SPAM SPAM SPAM!
To process different sorts of streams, pass in different sorts of objects to the class construction call. Here, we use an output file instead of a stream:
C:lp4e>python
>>>import converters
>>>prog = converters.Uppercase(open('spam.txt'), open('spamup.txt', 'w'))
>>>prog.process()
C:lp4e>type spamup.txt
SPAM SPAM SPAM!
But, as suggested earlier, we could also pass in arbitrary objects wrapped up in classes that define the required input and output method interfaces. Here’s a simple example that passes in a writer class that wraps up the text inside HTML tags:
C:lp4e>python
>>>from converters import Uppercase
>>> >>>class HTMLize:
...def write(self, line):
...print('<PRE>%s</PRE>' % line.rstrip())
... >>>Uppercase(open('spam.txt'), HTMLize()).process()
<PRE>SPAM</PRE> <PRE>SPAM</PRE> <PRE>SPAM!</PRE>
If you trace through this example’s control flow, you’ll see
that we get both uppercase conversion (by inheritance) and HTML
formatting (by composition), even though the core processing logic
in the original Processor
superclass knows nothing about either step. The processing code only
cares that writers have a write
method and that a method named convert
is defined; it doesn’t care what
those methods do when they are called. Such polymorphism and
encapsulation of logic is behind much of the power of
classes.
As is, the Processor
superclass only provides a file-scanning loop. In more realistic
work, we might extend it to support additional programming tools for
its subclasses, and, in the process, turn it into a full-blown
framework. Coding such a tool once in a superclass enables you to
reuse it in all of your programs. Even in this simple example,
because so much is packaged and inherited with classes, all we had
to code was the HTML formatting step; the rest was free.
For another example of composition at work, see exercise 9 at the end of Chapter 31 and its solution in Appendix B; it’s similar to the pizza shop example. We’ve focused on inheritance in this book because that is the main tool that the Python language itself provides for OOP. But, in practice, composition is used as much as inheritance as a way to structure classes, especially in larger systems. As we’ve seen, inheritance and composition are often complementary (and sometimes alternative) techniques. Because composition is a design issue outside the scope of the Python language and this book, though, I’ll defer to other resources for more on this topic.
Beside inheritance and composition, object-oriented programmers
often also talk about something called delegation, which usually implies controller
objects that embed other objects to which they pass off operation
requests. The controllers can take care of administrative activities,
such as keeping track of accesses and so on. In Python, delegation is
often implemented with the __getattr__
method
hook; because it intercepts accesses to nonexistent attributes, a
wrapper class (sometimes called a
proxy class) can use __getattr__
to route arbitrary accesses to a
wrapped object. The wrapper class retains the interface of the wrapped
object and may add additional operations of its own.
Consider the file trace.py, for instance:
class wrapper: def __init__(self, object): self.wrapped = object # Save object def __getattr__(self, attrname): print('Trace:', attrname) # Trace fetch return getattr(self.wrapped, attrname) # Delegate fetch
Recall from Chapter 29 that
__getattr__
gets the attribute name
as a string. This code makes use of the getattr
built-in function to fetch an
attribute from the wrapped object by name string—getattr(X,N)
is like X.N
, except that N
is an expression that evaluates to a
string at runtime, not a variable. In fact, getattr(X,N)
is similar to X.__dict__[N]
, but the former also performs
an inheritance search, like X.N
,
while the latter does not (see Namespace Dictionaries for more on the __dict__
attribute).
You can use the approach of this module’s wrapper class to
manage access to any object with attributes—lists, dictionaries, and
even classes and instances. Here, the wrapper
class simply prints a trace message
on each attribute access and delegates the attribute request to the
embedded wrapped
object:
>>>from trace import wrapper
>>>x = wrapper([1,2,3])
# Wrap a list >>>x.append(4)
# Delegate to list method Trace: append >>>x.wrapped
# Print my member [1, 2, 3, 4] >>>x = wrapper({"a": 1, "b": 2})
# Wrap a dictionary >>>list(x.keys())
# Delegate to dictionary method Trace: keys ['a', 'b']
The net effect is to augment the entire interface of the
wrapped
object, with additional
code in the wrapper
class. We can
use this to log our method calls, route method calls to extra or
custom logic, and so on.
We’ll revive the notions of wrapped objects and delegated operations as one way to extend built-in types in Chapter 31. If you are interested in the delegation design pattern, also watch for the discussions in Chapters 31 and 38 of function decorators, a strongly related concept designed to augment a specific function or method call rather than the entire interface of an object, and class decorators, which serve as a way to automatically add such delegation-based wrappers to all instances of a class.
Version skew note: In Python 2.6, operator overloading methods run by built-in operations are routed through
generic attribute interception methods like __getattr__
. Printing a wrapped object
directly, for example, calls this method for __repr__
or __str__
, which then passes the call on to
the wrapped object. In Python 3.0, this no longer happens: printing
does not trigger __getattr__
, and
a default display is used instead. In 3.0, new-style classes look up
operator overloading methods in classes and skip the normal instance
lookup entirely. We’ll return to this issue in Chapter 37, in the context of managed
attributes; for now, keep in mind that you may need to redefine
operator overloading methods in wrapper classes (either by hand, by
tools, or by superclasses) if you want them to be intercepted in
3.0.
Besides larger structuring goals, class designs often must address name usage too. In Part V, we learned that every name assigned at the top level of a module file is exported. By default, the same holds for classes—data hiding is a convention, and clients may fetch or change any class or instance attribute they like. In fact, attributes are all “public” and “virtual,” in C++ terms; they’re all accessible everywhere and are looked up dynamically at runtime.[69]
That said, Python today does support the notion of name “mangling” (i.e., expansion) to localize some names in classes. Mangled names are sometimes misleadingly called “private attributes,” but really this is just a way to localize a name to the class that created it—name mangling does not prevent access by code outside the class. This feature is mostly intended to avoid namespace collisions in instances, not to restrict access to names in general; mangled names are therefore better called “pseudoprivate” than “private.”
Pseudoprivate names are an advanced and entirely
optional feature, and you probably won’t find them very useful until
you start writing general tools or larger class hierarchies for use in
multiprogrammer projects. In fact, they are not always used even when
they probably should be—more commonly, Python programmers code
internal names with a single underscore (e.g., _X
), which is just an informal convention to
let you know that a name shouldn’t be changed (it means nothing to
Python itself).
Because you may see this feature in other people’s code, though, you need to be somewhat aware of it, even if you don’t use it yourself.
Here’s how name mangling works: names inside a class
statement that start with two
underscores but don’t end with two underscores are automatically
expanded to include the name of the enclosing class. For instance, a
name like __X
within a class
named Spam
is changed to _Spam__X
automatically: the original name
is prefixed with a single underscore and the enclosing class’s name.
Because the modified name contains the name of the enclosing class,
it’s somewhat unique; it won’t clash with similar names created by
other classes in a hierarchy.
Name mangling happens only in class
statements, and only for names that
begin with two leading underscores. However, it happens for
every name preceded with double
underscores—both class attributes (like method names) and instance
attribute names assigned to self
attributes. For example, in a class named Spam
, a method named __meth
is mangled to _Spam__meth
, and an instance attribute
reference self.__X
is transformed
to self._Spam__X
. Because more
than one class may add attributes to an instance, this mangling
helps avoid clashes—but we need to move on to an example to see
how.
One of the main problems that the pseudoprivate attribute feature is meant to alleviate has to do with the way instance attributes are stored. In Python, all instance attributes wind up in the single instance object at the bottom of the class tree. This is different from the C++ model, where each class gets its own space for data members it defines.
Within a class method in Python, whenever a method assigns to
a self
attribute (e.g., self.
attr
=
value
), it changes or creates an
attribute in the instance (inheritance searches happen only on
reference, not on assignment). Because this is true even if multiple
classes in a hierarchy assign to the same attribute, collisions are
possible.
For example, suppose that when a programmer codes a class, she
assumes that she owns the attribute name X
in the instance. In this class’s
methods, the name is set, and later fetched:
class C1:
def meth1(self): self.X = 88 # I assume X is mine
def meth2(self): print(self.X)
Suppose further that another programmer, working in isolation, makes the same assumption in a class that he codes:
class C2:
def metha(self): self.X = 99 # Me too
def methb(self): print(self.X)
Both of these classes work by themselves. The problem arises if the two classes are ever mixed together in the same class tree:
class C3(C1, C2): ...
I = C3() # Only 1 X in I!
Now, the value that each class gets back when it says self.X
will depend on which class assigned
it last. Because all assignments to self.X
refer to the same single instance,
there is only one X
attribute—I.X
—no matter how many
classes use that attribute name.
To guarantee that an attribute belongs to the class that uses it, prefix the name with double underscores everywhere it is used in the class, as in this file, private.py:
class C1: def meth1(self): self.__X = 88 # Now X is mine def meth2(self): print(self.__X) # Becomes _C1__X in I class C2: def metha(self): self.__X = 99 # Me too def methb(self): print(self.__X) # Becomes _C2__X in I class C3(C1, C2): pass I = C3() # Two X names in I I.meth1(); I.metha() print(I.__dict__) I.meth2(); I.methb()
When thus prefixed, the X
attributes will be expanded to include the names of their classes
before being added to the instance. If you run a dir
call on I
or inspect its namespace dictionary
after the attributes have been assigned, you’ll see the expanded
names, _C1__X
and _C2__X
, but not X
. Because the expansion makes the names
unique within the instance, the class coders can safely assume that
they truly own any names that they prefix with two
underscores:
% python private.py
{'_C2__X': 99, '_C1__X': 88}
88
99
This trick can avoid potential name collisions in the
instance, but note that it does not amount to true privacy. If you
know the name of the enclosing class, you can still access either of
these attributes anywhere you have a reference to the instance by
using the fully expanded name (e.g., I._C1__X = 77
). On the other hand, this
feature makes it less likely that you will
accidentally step on a class’s names.
Pseudoprivate attributes are also useful in larger frameworks or tools, both to avoid introducing new method names that might accidentally hide definitions elsewhere in the class tree and to reduce the chance of internal methods being replaced by names defined lower in the tree. If a method is intended for use only within a class that may be mixed into other classes, the double underscore prefix ensures that the method won’t interfere with other names in the tree, especially in multiple-inheritance scenarios:
class Super: def method(self): ... # A real application method class Tool: def __method(self): ... # Becomes _Tool__method def other(self): self.__method() # Use my internal method class Sub1(Tool, Super): ... def actions(self): self.method() # Runs Super.method as expected class Sub2(Tool): def __init__(self): self.method = 99 # Doesn't break Tool.__method
We met multiple inheritance briefly in Chapter 25 and will explore it in more
detail later in this chapter. Recall that superclasses are searched
according to their left-to-right order in class
header lines. Here, this means
Sub1
prefers Tool
attributes to those in Super
. Although in this example we could
force Python to pick the application class’s methods first by
switching the order of the superclasses listed in the Sub1
class header, pseudoprivate
attributes resolve the issue altogether. Pseudoprivate names also
prevent subclasses from accidentally redefining the internal
method’s names, as in Sub2
.
Again, I should note that this feature tends to be of use primarily for larger, multiprogrammer projects, and then only for selected names. Don’t be tempted to clutter your code unnecessarily; only use this feature for names that truly need to be controlled by a single class. For simpler programs, it’s probably overkill.
For more examples that make use of the __X
naming feature, see the lister.py mix-in classes introduced later in
this chapter, in the section on multiple inheritance, as well as the discussion of Private
class decorators in Chapter 38. If you care
about privacy in general, you might want to review the
emulation of private instance
attributes sketched in the section Attribute Reference: __getattr__ and __setattr__ in Chapter 29, and watch for the Private
class decorator in Chapter 38 that we will base upon this special method.
Although it’s possible to emulate true access controls in Python
classes, this is rarely done in practice, even for large
systems.
Methods in general, and bound methods in particular, simplify the implementation of many design goals in Python. We met
bound methods briefly while studying __call__
in Chapter 29. The full story, which we’ll flesh
out here, turns out to be more general and flexible than you might
expect.
In Chapter 19, we learned how functions can be processed as normal objects. Methods are a kind of object too, and can be used generically in much the same way as other objects—they can be assigned, passed to functions, stored in data structures, and so on. Because class methods can be accessed from an instance or a class, though, they actually come in two flavors in Python:
self
Accessing a function attribute of a class by qualifying the class returns an unbound method object. To call the method, you must provide an instance object explicitly as the first argument. In Python 3.0, an unbound method is the same as a simple function and can be called though the class’s name; in 2.6 it’s a distinct type and cannot be called without providing an instance.
self
+ function pairsAccessing a function attribute of a class by qualifying an instance returns a bound method object. Python automatically packages the instance with the function in the bound method object, so you don’t need to pass an instance to call the method.
Both kinds of methods are full-fledged objects; they can be
transferred around a program at will, just like strings and numbers.
Both also require an instance in their first argument when run (i.e.,
a value for self
). This is why we
had to pass in an instance explicitly when calling superclass methods
from subclass methods in the previous chapter; technically, such calls
produce unbound method objects.
When calling a bound method object, Python provides an instance for you automatically—the instance used to create the bound method object. This means that bound method objects are usually interchangeable with simple function objects, and makes them especially useful for interfaces originally written for functions (see the sidebar Why You Will Care: Bound Methods and Callbacks for a realistic example).
To illustrate, suppose we define the following class:
class Spam: def doit(self, message): print(message)
Now, in normal operation, we make an instance and call its method in a single step to print the passed-in argument:
object1 = Spam() object1.doit('hello world')
Really, though, a bound method object is
generated along the way, just before the method call’s parentheses. In
fact, we can fetch a bound method without actually calling it. An
object
.
name
qualification is an object expression. In the following, it returns a
bound method object that packages the instance (object1
) with the method function (Spam.doit
). We can assign this bound method
pair to another name and then call it as though it were a simple
function:
object1 = Spam() x = object1.doit # Bound method object: instance+function x('hello world') # Same effect as object1.doit('...')
On the other hand, if we qualify the
class to get to doit
, we get back
an unbound method object, which is simply a
reference to the function object. To call this type of method, we must
pass in an instance as the leftmost argument:
object1 = Spam() t = Spam.doit # Unbound method object (a function in 3.0: see ahead) t(object1, 'howdy') # Pass in instance (if the method expects one in 3.0)
By extension, the same rules apply within a class’s method if we
reference self
attributes that
refer to functions in the class. A self.
method
expression is a bound method object because self
is an instance object:
class Eggs: def m1(self, n): print(n) def m2(self): x = self.m1 # Another bound method object x(42) # Looks like a simple function Eggs().m2() # Prints 42
Most of the time, you call methods immediately after fetching them with attribute qualification, so you don’t always notice the method objects generated along the way. But if you start writing code that calls objects generically, you need to be careful to treat unbound methods specially—they normally require an explicit instance object to be passed in.[70]
In Python 3.0, the language has dropped the notion of unbound methods. What we describe as an unbound method here is treated as a simple function in 3.0. For most purposes, this makes no difference to your code; either way, an instance will be passed to a method’s first argument when it’s called through an instance.
Programs that do explicit type testing might be impacted, though—if you print the type of an instance-less class method, it displays “unbound method” in 2.6, and “function” in 3.0.
Moreover, in 3.0 it is OK to call a method without an instance, as long as the method does not expect one and you call it only through the class and never through an instance. That is, Python 3.0 will pass along an instance to methods only for through-instance calls. When calling through a class, you must pass an instance manually only if the method expects one:
C:misc>c:python30python
>>>class Selfless:
...def __init__(self, data):
...self.data = data
...def selfless(arg1, arg2):
# A simple function in 3.0 ...return arg1 + arg2
...def normal(self, arg1, arg2):
# Instance expected when called ...return self.data + arg1 + arg2
... >>>X = Selfless(2)
>>>X.normal(3, 4)
# Instance passed to self automatically 9 >>>Selfless.normal(X, 3, 4)
# self expected by method: pass manually 9 >>>Selfless.selfless(3, 4)
# No instance: works in 3.0, fails in 2.6! 7
The last test in this fails in 2.6, because unbound methods require an instance to be passed by default; it works in 3.0 because such methods are treated as simple functions not requiring an instance. Although this removes some potential error trapping in 3.0 (what if a programmer accidentally forgets to pass an instance?), it allows class methods to be used as simple functions as long as they are not passed and do not expect a “self” instance argument.
The following two calls still fail in both 3.0 and 2.6, though—the first (calling through an instance) automatically passes an instance to a method that does not expect one, while the second (calling through a class) does not pass an instance to a method that does expect one:
>>>X.selfless(3, 4)
TypeError: selfless() takes exactly 2 positional arguments (3 given) >>>Selfless.normal(3, 4)
TypeError: normal() takes exactly 3 positional arguments (2 given)
Because of this change, the staticmethod
decorator described in the next chapter is not needed in 3.0 for
methods without a self
argument
that are called only through the class name, and never through an
instance—such methods are run as simple functions, without receiving
an instance argument. In 2.6, such calls are errors unless an
instance is passed manually (more on static methods in the next
chapter).
It’s important to be aware of the differences in behavior in 3.0, but bound methods are generally more important from a practical perspective anyway. Because they pair together the instance and function in a single object, they can be treated as callables generically. The next section demonstrates what this means in code.
For a more visual illustration of unbound method treatment in Python 3.0 and 2.6, see also the lister.py example in the multiple inheritance section later in this chapter. Its classes print the value of methods fetched from both instances and classes, in both versions of Python.
As mentioned earlier, bound methods can be processed as generic objects, just like simple functions—they can be passed around a program arbitrarily. Moreover, because bound methods combine both a function and an instance in a single package, they can be treated like any other callable object and require no special syntax when invoked. The following, for example, stores four bound method objects in a list and calls them later with normal call expressions:
>>>class Number:
...def __init__(self, base):
...self.base = base
...def double(self):
...return self.base * 2
...def triple(self):
...return self.base * 3
... >>>x = Number(2)
# Class instance objects >>>y = Number(3)
# State + methods >>>z = Number(4)
>>>x.double()
# Normal immediate calls 4 >>>acts = [x.double, y.double, y.triple, z.double]
# List of bound methods >>>for act in acts:
# Calls are deferred ...print(act())
# Call as though functions ... 4 6 9 8
Like simple functions, bound method objects have introspection information of their own, including attributes that give access to the instance object and method function they pair. Calling the bound method simply dispatches the pair:
>>>bound = x.double
>>>bound.__self__, bound.__func__
(<__main__.Number object at 0x0278F610>, <function double at 0x027A4ED0>) >>>bound.__self__.base
2 >>>bound()
# Calls bound.__func__(bound.__self__, ...) 4
In fact, bound methods are just one of a handful of callable
object types in Python. As the following demonstrates, simple
functions coded with a def
or
lambda
, instances that inherit a
__call__
, and bound instance
methods can all be treated and called the same way:
>>>def square(arg):
...return arg ** 2
# Simple functions (def or lambda) ... >>>class Sum:
...def __init__(self, val):
# Callable instances ...self.val = val
...def __call__(self, arg):
...return self.val + arg
... >>>class Product:
...def __init__(self, val):
# Bound methods ...self.val = val
...def method(self, arg):
...return self.val * arg
... >>>sobject = Sum(2)
>>>pobject = Product(3)
>>>actions = [square, sobject, pobject.method]
# Function, instance, method >>>for act in actions:
# All 3 called same way ...print(act(5))
# Call any 1-arg callable ... 25 7 15 >>>actions[-1](5)
# Index, comprehensions, maps 15 >>>[act(5) for act in actions]
[25, 7, 15] >>>list(map(lambda act: act(5), actions))
[25, 7, 15]
Technically speaking, classes belong in the callable objects category too, but we normally call them to generate instances rather than to do actual work, as shown here:
>>>class Negate:
...def __init__(self, val):
# Classes are callables too ...self.val = -val
# But called for object, not work ...def __repr__(self):
# Instance print format ...return str(self.val)
... >>>actions = [square, sobject, pobject.method, Negate]
# Call a class too >>>for act in actions:
...print(act(5))
... 25 7 15 -5 >>>[act(5) for act in actions]
# Runs __repr__ not __str__! [25, 7, 15, −5] >>>table = {act(5): act for act in actions}
# 3.0 dict comprehension >>>for (key, value) in table.items():
...print('{0:2} => {1}'.format(key, value))
# 2.6/3.0 str.format ... -5 => <class '__main__.Negate'> 25 => <function square at 0x025D4978> 15 => <bound method Product.method of <__main__.Product object at 0x025D0F90>> 7 => <__main__.Sum object at 0x025D0F70>
As you can see, bound methods, and Python’s callable objects model in general, are some of the many ways that Python’s design makes for an incredibly flexible language.
You should now understand the method object model. For other
examples of bound methods at work, see the upcoming sidebar as well as the prior chapter’s
discussion of callback handlers in the section on the method
__call__
.
Many class-based designs call for combining disparate sets of
methods. In a class
statement, more
than one superclass can be listed in parentheses in the header line.
When you do this, you use something called multiple inheritance—the class and its
instances inherit names from all the listed
superclasses.
When searching for an attribute, Python’s inheritance search traverses all superclasses in the class header from left to right until a match is found. Technically, because any of the superclasses may have superclasses of its own, this search can be a bit more complex for larger class tress:
In classic classes (the default until Python 3.0), the attribute search proceeds depth-first all the way to the top of the inheritance tree, and then from left to right.
In new-style classes (and all classes in 3.0), the attribute search proceeds across by tree levels, in a more breadth-first fashion (see the new-style class discussion in the next chapter).
In either model, though, when a class has multiple superclasses,
they are searched from left to right according to the order listed in
the class
statement header
lines.
In general, multiple inheritance is good for modeling objects that belong to more than one set. For instance, a person may be an engineer, a writer, a musician, and so on, and inherit properties from all such sets. With multiple inheritance, objects obtain the union of the behavior in all their superclasses.
Perhaps the most common way multiple inheritance is used is to
“mix in” general-purpose methods from superclasses. Such superclasses
are usually called mix-in classes—they
provide methods you add to application classes by inheritance. In a
sense, mix-in classes are similar to modules: they provide packages of
methods for use in their client subclasses. Unlike simple functions in
modules, though, methods in mix-ins also have access to the self
instance, for using state information
and other methods. The next section demonstrates one common use case
for such tools.
As we’ve seen, Python’s default way to print a class instance object isn’t incredibly useful:
>>>class Spam:
...def __init__(self):
# No __repr__ or __str__ ...self.data1 = "food"
... >>>X = Spam()
>>>print(X)
# Default: class, address <__main__.Spam object at 0x00864818> # Displays "instance" in Python 2.6
As you saw in Chapter 29 when
studying operator overloading, you can provide a __str__
or __repr__
method to implement a custom
string representation of your own. But, rather than coding one of
these in each and every class you wish to print, why not code it
once in a general-purpose tool class and inherit it in all your
classes?
That’s what mix-ins are for. Defining a display method in a mix-in superclass once enables us to reuse it anywhere we want to see a custom display format. We’ve already seen tools that do related work:
Chapter 27’s AttrDisplay
class formatted instance
attributes in a generic __str__
method, but it did not climb
class trees and was used in single-inheritance mode only.
Chapter 28’s classtree.py module defined functions for climbing and sketching class trees, but it did not display object attributes along the way and was not architected as an inheritable class.
Here, we’re going to revisit these examples’ techniques and expand upon them to code a set of three mix-in classes that serve as generic display tools for listing instance attributes, inherited attributes, and attributes on all objects in a class tree. We’ll also use our tools in multiple-inheritance mode and deploy coding techniques that make classes better suited to use as generic tools.
Let’s get started with the simple case—listing attributes
attached to an instance. The following class, coded in the file
lister.py,
defines a mix-in called ListInstance
that overloads the __str__
method for all classes that
include it in their header lines. Because this is coded as a
class, ListInstance
is a
generic tool whose formatting logic can be used for instances of
any subclass:
# File lister.py class ListInstance: """ Mix-in class that provides a formatted print() or str() of instances via inheritance of __str__, coded here; displays instance attrs only; self is the instance of lowest class; uses __X names to avoid clashing with client's attrs """ def __str__(self): return '<Instance of %s, address %s: %s>' % ( self.__class__.__name__, # My class's name id(self), # My address self.__attrnames()) # name=value list def __attrnames(self): result = '' for attr in sorted(self.__dict__): # Instance attr dict result += ' name %s=%s ' % (attr, self.__dict__ [attr]) return result
ListInstance
uses some
previously explored tricks to extract the instance’s class name
and attributes:
Each instance has a built-in __class__
attribute that references
the class from which it was created, and each class has a
__name__
attribute that
references the name in the header, so the expression self.__class__.__name__
fetches the
name of an instance’s class.
This class does most of its work by simply scanning the
instance’s attribute dictionary (remember, it’s exported in
__dict__
) to build up a
string showing the names and values of all instance
attributes. The dictionary’s keys are sorted to finesse any
ordering differences across Python releases.
In these respects, ListInstance
is similar to Chapter 27’s attribute display; in
fact, it’s largely just a variation on a theme. Our class here
uses two additional techniques, though:
It displays the instance’s memory address by calling the
id
built-function, which
returns any object’s address (by definition, a unique object
identifier, which will be useful in later mutations of this
code).
It uses the pseudoprivate naming
pattern for its worker method: __attrnames
. As we learned earlier
in his chapter, Python automatically localizes any such name
to its enclosing class by expanding the attribute name to
include the class name (in this case, it becomes _ListInstance__attrnames
). This
holds true for both class attributes (like methods) and
instance attributes attached to self
. This behavior is useful in a
general tool like this, as it ensures that its names don’t
clash with any names used in its client subclasses.
Because ListInstance
defines a __str__
operator
overloading method, instances derived from this class display
their attributes automatically when printed, giving a bit more
information than a simple address. Here is the class in action, in
single-inheritance mode (this code works the same in both Python
3.0 and 2.6):
>>>from lister import ListInstance
>>>class Spam(ListInstance):
# Inherit a __str__ method ...def __init__(self):
...self.data1 = 'food'
... >>>x = Spam()
>>>print(x)
# print() and str() run __str__ <Instance of Spam, address 40240880: name data1=food >
You can also fetch the listing output as a string without
printing it with str
, and
interactive echoes still use the default format:
>>>str(x)
'<Instance of Spam, address 40240880: name data1=food >' >>>x
# The __repr__ still is a default <__main__.Spam object at 0x026606F0>
The ListInstance
class is
useful for any classes you write—even classes that already have
one or more superclasses. This is where multiple
inheritance comes in handy: by adding ListInstance
to the list of superclasses
in a class header (i.e., mixing it in), you get its __str__
“for free” while still
inheriting from the existing superclass(es). The file testmixin.py demonstrates:
# File testmixin.py from lister import * # Get lister tool classes class Super: def __init__(self): # Superclass __init__ self.data1 = 'spam' # Create instance attrs def ham(self): pass class Sub(Super, ListInstance): # Mix in ham and a __str__ def __init__(self): # listers have access to self Super.__init__(self) self.data2 = 'eggs' # More instance attrs self.data3 = 42 def spam(self): # Define another method here pass if __name__ == '__main__': X = Sub() print(X) # Run mixed-in __str__
Here, Sub
inherits names
from both Super
and ListInstance
; it’s a composite of its
own names and names in both its superclasses. When you make a
Sub
instance and print it, you
automatically get the custom representation mixed in from ListInstance
(in this case, this
script’s output is the same under both Python 3.0 and 2.6, except
for object addresses):
C:misc> C:python30python testmixin.py
<Instance of Sub, address 40962576:
name data1=spam
name data2=eggs
name data3=42
>
ListInstance
works in any
class it’s mixed into because self
refers to an instance of the
subclass that pulls this class in, whatever that may be. In a
sense, mix-in classes are the class equivalent of modules—packages
of methods useful in a variety of clients. For example, here is
Lister
working again in
single-inheritance mode on a different class’s instances, with
import
and attributes set
outside the class:
>>>import lister
>>>class C(lister.ListInstance): pass
... >>>x = C()
>>>x.a = 1; x.b = 2; x.c = 3
>>>print(x)
<Instance of C, address 40961776: name a=1 name b=2 name c=3 >
Besides the utility they provide, mix-ins optimize code
maintenance, like all classes do. For example, if you later decide
to extend ListInstance
’s
__str__
to also print all the
class attributes that an instance inherits, you’re safe; because
it’s an inherited method, changing __str__
automatically updates the
display of each subclass that imports the class and mixes it in.
Since it’s now officially “later,” let’s move on to the next
section to see what such an extension might look like.
As it is, our Lister
mix-in
displays instance attributes only (i.e., names attached to the
instance object itself). It’s trivial to extend the class to
display all the attributes accessible from an instance,
though—both its own and those it inherits from its classes. The
trick is to use the dir
built-in function instead of scanning
the instance’s __dict__
dictionary; the latter holds instance attributes only, but the
former also collects all inherited attributes in Python 2.2 and
later.
The following mutation codes this scheme; I’ve renamed it to facilitate simple testing, but if this were to replace the original version, all existing clients would pick up the new display automatically:
# File lister.py, continued class ListInherited: """ Use dir() to collect both instance attrs and names inherited from its classes; Python 3.0 shows more names than 2.6 because of the implied object superclass in the new-style class model; getattr() fetches inherited names not in self.__dict__; use __str__, not __repr__, or else this loops when printing bound methods! """ def __str__(self): return '<Instance of %s, address %s: %s>' % ( self.__class__.__name__, # My class's name id(self), # My address self.__attrnames()) # name=value list def __attrnames(self): result = '' for attr in dir(self): # Instance dir() if attr[:2] == '__' and attr[-2:] == '__': # Skip internals result += ' name %s=<> ' % attr else: result += ' name %s=%s ' % (attr, getattr(self, attr)) return result
Notice that this code skips __
X
__
names’ values; most of these are
internal names that we don’t generally care about in a generic
listing like this. This version also must use the getattr
built-in function to fetch
attributes by name string instead of using instance attribute
dictionary indexing—getattr
employs the inheritance search protocol, and some of the names
we’re listing here are not stored on the instance itself.
To test the new version, change the testmixin.py file to use this new class instead:
class Sub(Super, ListInherited): # Mix in a __str__
This file’s output varies per release. In Python 2.6, we get the following; notice the name mangling at work in the lister’s method name (I shortened its full value display to fit on this page):
C:misc>c:python26python testmixin.py
<Instance of Sub, address 40073136: name _ListInherited__attrnames=<bound method Sub.__attrnames of <...more
...>> name __doc__=<> name __init__=<> name __module__=<> name __str__=<> name data1=spam name data2=eggs name data3=42 name ham=<bound method Sub.ham of <__main__.Sub instance at 0x026377B0>> name spam=<bound method Sub.spam of <__main__.Sub instance at 0x026377B0>> >
In Python 3.0, more attributes are displayed because all
classes are “new-style” and inherit names from the implied
object
superclass (more on this
in Chapter 31). Because so many
names are inherited from the default superclass, I’ve omitted many
here; run this on your own for the full listing:
C:misc>c:python30python testmixin.py
<Instance of Sub, address 40831792: name _ListInherited__attrnames=<bound method Sub.__attrnames of <...more
...>> name __class__=<> name __delattr__=<> name __dict__=<> name __doc__=<> name __eq__=<>...more names omitted...
name __repr__=<> name __setattr__=<> name __sizeof__=<> name __str__=<> name __subclasshook__=<> name __weakref__=<> name data1=spam name data2=eggs name data3=42 name ham=<bound method Sub.ham of <__main__.Sub object at 0x026F0B30>> name spam=<bound method Sub.spam of <__main__.Sub object at 0x026F0B30>> >
One caution here—now that we’re displaying inherited methods
too, we have to use __str__
instead of __repr__
to overload
printing. With __repr__
, this
code will loop—displaying the value of a
method triggers the __repr__
of
the method’s class, in order to display the class. That is, if the
lister’s __repr__
tries to
display a method, displaying the method’s class will trigger the
lister’s __repr__
again.
Subtle, but true! Change __str__
to __repr__
here to see this for yourself.
If you must use __repr__
in
such a context, you can avoid the loops by using isinstance
to compare the type of
attribute values against types.MethodType
in the standard
library, to know which items to skip.
Let’s code one last extension. As it is, our lister
doesn’t tell us which class an inherited name comes from. As we
saw in the classtree.py
example near the end of Chapter 28,
though, it’s straightforward to climb class inheritance trees in
code. The following mix-in class makes use of this same technique
to display attributes grouped by the classes they live in—it
sketches the full class tree, displaying attributes attached to
each object along the way. It does so by traversing the
inheritance tree from an instance’s __class__
to its class, and then from
the class’s __bases__
to all
superclasses recursively, scanning object __dicts__
s along the way:
# File lister.py, continued
class ListTree:
"""
Mix-in that returns an __str__ trace of the entire class
tree and all its objects' attrs at and above self;
run by print(), str() returns constructed string;
uses __X attr names to avoid impacting clients;
uses generator expr to recurse to superclasses;
uses str.format() to make substitutions clearer
"""
def __str__(self):
self.__visited = {}
return '<Instance of {0}, address {1}:
{2}{3}>'.format(
self.__class__.__name__,
id(self),
self.__attrnames(self, 0),
self.__listclass(self.__class__, 4))
def __listclass(self, aClass, indent):
dots = '.' * indent
if aClass in self.__visited:
return '
{0}<Class {1}:, address {2}: (see above)>
'.format(
dots,
aClass.__name__,
id(aClass))
else:
self.__visited[aClass] = True
genabove = (self.__listclass(c, indent+4) for c in aClass.__bases__)
return '
{0}<Class {1}, address {2}:
{3}{4}{5}>
'.format(
dots,
aClass.__name__,
id(aClass),
self.__attrnames(aClass, indent),
''.join(genabove),
dots)
def __attrnames(self, obj, indent):
spaces = ' ' * (indent + 4)
result = ''
for attr in sorted(obj.__dict__):
if attr.startswith('__') and attr.endswith('__'):
result += spaces + '{0}=<>
'.format(attr)
else:
result += spaces + '{0}={1}
'.format(attr, getattr(obj, attr))
return result
Note the use of a generator expression to direct the
recursive calls for superclasses; it’s activated by the nested
string join
method. Also see
how this version uses the Python 3.0 and 2.6 string format
method instead of %
formatting expressions, to make
substitutions clearer; when many substitutions are applied like
this, explicit argument numbers may make the code easier to
decipher. In short, in this version we exchange the first of the
following lines for the second:
return '<Instance of %s, address %s: %s%s>' % (...) # Expression return '<Instance of {0}, address {1}: {2}{3}>'.format(...) # Method
Now, change testmixin.py to inherit from this new class again to test:
class Sub(Super, ListTree): # Mix in a __str__
The file’s tree-sketcher output in Python 2.6 is then as follows:
C:misc> c:python26python testmixin.py
<Instance of Sub, address 40728496:
_ListTree__visited={}
data1=spam
data2=eggs
data3=42
....<Class Sub, address 40701168:
__doc__=<>
__init__=<>
__module__=<>
spam=<unbound method Sub.spam>
........<Class Super, address 40701120:
__doc__=<>
__init__=<>
__module__=<>
ham=<unbound method Super.ham>
........>
........<Class ListTree, address 40700688:
_ListTree__attrnames=<unbound method ListTree.__attrnames>
_ListTree__listclass=<unbound method ListTree.__listclass>
__doc__=<>
__module__=<>
__str__=<>
........>
....>
>
Notice in this output how methods are
unbound now under 2.6, because we fetch them
from classes directly, instead of from instances. Also observe how
the lister’s __visited
table has its name
mangled in the instance’s attribute dictionary; unless we’re very
unlucky, this won’t clash with other data there.
Under Python 3.0, we get extra attributes and superclasses
again. Notice that unbound methods are simple
functions in 3.0, as described in an earlier
note in this chapter (and that again, I’ve deleted most built-in
attributes in object
to save
space here; run this on your own for the complete listing):
C:misc>c:python30python testmixin.py
<Instance of Sub, address 40635216: _ListTree__visited={} data1=spam data2=eggs data3=42 ....<Class Sub, address 40914752: __doc__=<> __init__=<> __module__=<> spam=<function spam at 0x026D53D8> ........<Class Super, address 40829952: __dict__=<> __doc__=<> __init__=<> __module__=<> __weakref__=<> ham=<function ham at 0x026D5228> ............<Class object, address 505114624: __class__=<> __delattr__=<> __doc__=<> __eq__=<>...more omitted...
__repr__=<> __setattr__=<> __sizeof__=<> __str__=<> __subclasshook__=<> ............> ........> ........<Class ListTree, address 40829496: _ListTree__attrnames=<function __attrnames at 0x026D5660> _ListTree__listclass=<function __listclass at 0x026D56A8> __dict__=<> __doc__=<> __module__=<> __str__=<> __weakref__=<> ............<Class object:, address 505114624: (see above)> ........> ....> >
This version avoids listing the same class object twice by
keeping a table of classes visited so far
(this is why an object’s id
is
included—to serve as a key for a previously displayed item). Like
the transitive module reloader of Chapter 24, a dictionary works to avoid
repeats and cycles here because class objects may be dictionary
keys; a set would provide similar functionality.
This version also takes care to avoid large internal objects
by skipping __
X
__
names again. If you comment out the
test for these names, their values will display normally. Here’s
an excerpt from the output in 2.6 with this temporary change made
(it’s much larger in its entirety, and it gets even worse in 3.0,
which is why these names are probably better skipped!):
C:misc>c:python26python testmixin.py
...more omitted...
........<Class ListTree, address 40700688: _ListTree__attrnames=<unbound method ListTree.__attrnames> _ListTree__listclass=<unbound method ListTree.__listclass> __doc__= Mix-in that returns the __str__ trace of the entire class tree and all its objects' attrs at and above self; run by print, str returns constructed string; uses __X attr names to avoid impacting clients; uses generator expr to recurse to superclasses; uses str.format() to make substitutions clearer __module__=lister __str__=<unbound method ListTree.__str__> ........>
For more fun, try mixing this class into something more
substantial, like the Button
class of Python’s tkinter
GUI
toolkit module. In general, you’ll want to name ListTree
first (leftmost) in a class
header, so its __str__
is picked up; Button
has one, too, and the leftmost
superclass is searched first in multiple inheritance. The output
of the following is fairly
massive (18K characters), so run this code on your own to see the
full listing (and if you’re using Python 2.6, recall that you
should use Tkinter
for the module name instead of tkinter
):
>>>from lister import ListTree
>>>from tkinter import Button
# Both classes have a __str__ >>>class MyButton(ListTree, Button): pass
# ListTree first: use its __str__ ... >>>B = MyButton(text='spam')
>>>open('savetree.txt', 'w').write(str(B))
# Save to a file for later viewing 18247 >>>print(B)
# Print the display here <Instance of MyButton, address 44355632: _ListTree__visited={} _name=44355632 _tclCommands=[]...much more omitted...
>
Of course, there’s much more we could do here (sketching the tree in a GUI might be a natural next step), but we’ll leave further work as a suggested exercise. We’ll also extend this code in the exercises at the end of this part of the book, to list superclass names in parentheses at the start of instance and class displays.
The main point here is that OOP is all about code reuse, and
mix-in classes are a powerful example. Like almost everything else
in programming, multiple inheritance can be a useful device when
applied well. In practice, though, it is an advanced feature and
can become complicated if used carelessly or excessively. We’ll
revisit this topic as a gotcha at the end of the next chapter. In
that chapter, we’ll also meet the new-style class model, which
modifies the search order for one special multiple inheritance
case. For more ideas, see also Python manuals for the class.mro()
new-style class object
method, which returns a list giving the class tree search order
used by inheritance; this could be used by a class lister to show
attribute sources.
Supporting slots: Because they scan instance dictionaries, the
ListInstance
and ListTree
classes presented here don’t
directly support attributes stored in
slots—a newer and relatively rarely used
option we’ll meet in the next chapter, where instance attributes
are declared in a __slots__
class attribute. For example, if in textmixin.py we assign __slots__=['data1']
in Super
and __slots__=['data3']
in Sub
, only the data2
attribute is displayed in the
instance by these two lister classes; ListTree
also displays data1
and data3
, but as attributes of the
Super
and Sub
class objects and with a special
format for their values (technically, they are class-level
descriptors).
To better support slot attributes in these classes, change
the __dict__
scanning loops
to also iterate through __slots__
lists using code the next
chapter will present, and use the getattr
built-in function to fetch
values instead of __dict__
indexing (ListTree
already
does). Since instances inherit only the lowest class’s __slots__
, you may also need to come
up with a policy when __slots__
lists appear in multiple
superclasses (ListTree
already displays
them as class attributes). ListInherited
is immune to all this,
because dir
results combine
both __dict__
names and all
classes’ __slots__
names.
Alternatively, as a policy we could simply let our code handle slot-based attributes as it currently does, rather than complicating it for a rare, advanced feature. Slots and normal instance attributes are different kinds of names. We’ll investigate slots further in the next chapter; I omitted addressing them in these examples to avoid a forward dependency (not counting this note, of course!)—not exactly a valid design goal, but reasonable for a book.
Sometimes, class-based designs require objects to be created in response to conditions that can’t be predicted when a program is written. The factory design pattern allows such a deferred approach. Due in large part to Python’s flexibility, factories can take multiple forms, some of which don’t seem special at all.
Because classes are objects, it’s easy to pass them around a program, store them in data structures, and so on. You can also pass classes to functions that generate arbitrary kinds of objects; such functions are sometimes called factories in OOP design circles. Factories are a major undertaking in a strongly typed language such as C++ but are almost trivial to implement in Python. The call syntax we met in Chapter 18 can call any class with any number of constructor arguments in one step to generate any sort of instance:[71]
def factory(aClass, *args): # Varargs tuple return aClass(*args) # Call aClass (or apply in 2.6 only) class Spam: def doit(self, message): print(message) class Person: def __init__(self, name, job): self.name = name self.job = job object1 = factory(Spam) # Make a Spam object object2 = factory(Person, "Guido", "guru") # Make a Person object
In this code, we define an object generator function called
factory
. It expects to be passed a
class object (any class will do) along with one or more arguments for
the class’s constructor. The function uses special “varargs” call
syntax to call the function and return an instance.
The rest of the example simply defines two classes and generates
instances of both by passing them to the factory
function. And that’s the only
factory function you’ll ever need to write in Python; it works for any
class and any constructor arguments.
One possible improvement worth noting is that to support keyword
arguments in constructor calls, the factory can collect them with a
**args
argument and pass them along
in the class call, too:
def factory(aClass, *args, **kwargs): # +kwargs dict return aClass(*args, **kwargs) # Call aClass
By now, you should know that everything is an “object” in Python, including things like classes, which are just compiler input in languages like C++. However, as mentioned at the start of this part of the book, only objects derived from classes are OOP objects in Python.
So what good is the factory
function (besides providing an excuse to illustrate class objects in
this book)? Unfortunately, it’s difficult to show applications of
this design pattern without listing much more code than we have
space for here. In general, though, such a factory might allow code
to be insulated from the details of dynamically configured object
construction.
For instance, recall the processor
example presented in the
abstract in Chapter 25, and then
again as a composition example in this chapter. It accepts reader
and writer objects for processing arbitrary data streams. The
original version of this example manually passed in instances of
specialized classes like FileWriter
and SocketReader
to customize the data streams
being processed; later, we passed in hardcoded file, stream, and
formatter objects. In a more
dynamic scenario, external devices such as configuration files or
GUIs might be used to configure the streams.
In such a dynamic world, we might not be able to hardcode the creation of stream interface objects in our scripts, but might instead create them at runtime according to the contents of a configuration file.
For example, the file might simply give the string name of a stream class to be imported from a module, plus an optional constructor call argument. Factory-style functions or code might come in handy here because they would allow us to fetch and pass in classes that are not hardcoded in our program ahead of time. Indeed, those classes might not even have existed at all when we wrote our code:
classname =...parse from config file...
classarg =...parse from config file...
import streamtypes # Customizable code aclass = getattr(streamtypes, classname) # Fetch from module reader = factory(aclass, classarg) # Or aclass(classarg) processor(reader, ...)
Here, the getattr
built-in
is again used to fetch a module attribute given a string name (it’s
like saying obj
.
attr
, but
attr
is a string). Because this code
snippet assumes a single constructor argument, it doesn’t strictly
need factory
or apply
—we could make an instance with just
aclass(classarg)
. They may prove
more useful in the presence of unknown argument lists, however, and
the general factory coding pattern can improve the code’s flexibility.
In this chapter, we’ve seen inheritance, composition, delegation, multiple inheritance, bound methods, and factories—all common patterns used to combine classes in Python programs. We’ve really only scratched the surface here in the design patterns domain, though. Elsewhere in this book you’ll find coverage of other design-related topics, such as:
Abstract superclasses (Chapter 28)
Type subclasses (Chapter 31)
Static and class methods (Chapter 31)
Managed attributes (Chapter 37)
For more details on design patterns, though, we’ll delegate to other resources on OOP at large. Although patterns are important in OOP work, and are often more natural in Python than other languages, they are not specific to Python itself.
In this chapter, we sampled common ways to use and combine classes to optimize their reusability and factoring benefits—what are usually considered design issues that are often independent of any particular programming language (though Python can make them easier to implement). We studied delegation (wrapping objects in proxy classes), composition (controlling embedded objects), and inheritance (acquiring behavior from other classes), as well as some more esoteric concepts such as pseudoprivate attributes, multiple inheritance, bound methods, and factories.
The next chapter ends our look at classes and OOP by surveying more advanced class-related topics; some of its material may be of more interest to tool writers than application programmers, but it still merits a review by most people who will do OOP in Python. First, though, another quick chapter quiz.
What is multiple inheritance?
What is delegation?
What is composition?
What are bound methods?
What are pseudoprivate attributes used for?
Multiple inheritance occurs when a class inherits from more
than one superclass; it’s useful for mixing together multiple
packages of class-based code. The left-to-right order in class
statement headers determines the
order of attribute searches.
Delegation involves wrapping an object in a proxy class, which adds extra behavior and passes other operations to the wrapped object. The proxy retains the interface of the wrapped object.
Composition is a technique whereby a controller class embeds and directs a number of objects, and provides an interface all its own; it’s a way to build up larger structures with classes.
Bound methods combine an instance and a method function; you can call them without passing in an instance object explicitly because the original instance is still available.
Pseudoprivate attributes (whose names begin with two leading
underscores: __
X
) are used
to localize names to the enclosing class. This includes both class
attributes like methods defined inside the class, and self
instance attributes assigned inside
the class. Such names are expanded to include the class name,
which makes them unique.
[69] This tends to scare people with a C++ background
unnecessarily. In Python, it’s even possible to change or
completely delete a class method at runtime. On the other hand,
almost nobody ever does this in practical programs. As a scripting
language, Python is more about enabling than restricting. Also,
recall from our discussion of operator overloading in Chapter 29 that __getattr__
and __setattr__
can be used to emulate
privacy, but are generally not used for this purpose in practice.
More on this when we code a more realistic privacy decorator Chapter 38.
[70] See the discussion of static and class methods in Chapter 31 for an optional exception to this rule. Like bound methods, static methods can masquerade as basic functions because they do not expect instances when called. Python supports three kinds of class methods—instance, static, and class—and 3.0 allows simple functions in classes, too.
[71] Actually, this syntax can invoke any callable object,
including functions, classes, and methods. Hence, the factory
function here can also run any
callable object, not just a class (despite the argument name).
Also, as we learned in Chapter 18, Python 2.6
has an alternative to aClass(*args)
: the apply(aClass, args)
built-in call, which
has been removed in Python 3.0 because of its redundancy and
limitations.