One way we can improve our use of classes is to add functions to them. These class functions are known by their more common name, methods. In Python, methods are defined as part of the class definition, but can be invoked only on an instance. In other words, the path one must take to finally be able to call a method goes like this: (1) define the class (and the methods), (2) create an instance, and finally, (3) invoke the method on that instance. Here is an example class with a method:

You will notice the self argument, which must be present in all method declarations. That argument, representing the instance object, is passed to the method implicitly by the interpreter when you invoke a method on an instance, so you, yourself, do not have to worry about passing anything in (specifically self, which is automatically passed in for you).

For example, if you have a method that takes two arguments, all of your calls should only pass in the second argument. Python passes in self for you as the first. If you make a mistake, do not worry about it. When an error occurs, Python will tell you that you have passed in the wrong number of arguments. You may make this mistake only once anyway... you’ll certainly remember each time after that!

The requirement of the instance (self) in each method’s signature will be something new to those of you coming from C++ or Java, so be aware of that. It is all part of Python’s philosophy of being explicitly clear. In those other languages, self is called “this.” You can find out more about self in the Core Note in Section 13.7 on page 541. Requiring the instance only applies to regular methods and not static or class methods, although the latter requires the class rather than the instance. You can find out more about static and class methods in Section 13.8 on page 542.

Now we will instantiate the class and invoke the method once we have an instance:

We conclude this introductory section by giving you a slightly more complex example of what you can do with classes (and instances) and also introducing you to the special method __init__() as well as subclassing and inheritance.

For those of you who are already familiar with object-oriented programming, __init__() is similar to the class constructor. If you are new to the world of OOP, a constructor is simply a special method that is typically called to create a new object. In Python, __init__() is not really the constructor. You do not call “new” to create a new object. (Python does not even have a keyword called “new” anyway.) Instead, Python creates the instance for you and then calls __init__() during instantiation to define additional behavior that should occur when a class is instantiated, i.e., setting up initial values or running some preliminary diagnostic code—basically performing any special tasks or setup after the instance is created but before the new instance is returned from the instantiation call.

(We will add print statements to our methods to better illustrate when certain methods are called. It is generally not typical to have input or output statements in functions unless output is a predetermined characteristic of the body of code.)

In the definition for the AddrBookEntry class, we define two methods: __init__() and updatePhone(). __init__() is called when instantiation occurs, that is, when AddrBookEntry() is invoked. You can think of instantiation as an implicit call to __init__() because the arguments given to AddrBookEntry() are exactly the same as those that are received by __init__() (except for self, which is passed automatically for you).

Recall that the self (instance object) argument is passed in automatically by the interpreter when the method is invoked on an instance, so in our __init__() above, the only required arguments are nm and ph, representing the name and telephone number, respectively. __init__() sets these two instance attributes on instantiation so that they are available to the programmer by the time the instance is returned from the instantiation call.

As you may have surmised, the purpose of the updatePhone() method is to replace an address book entry’s telephone number attribute.

These are our instantiation calls, which, in turn, invoke __init__(). Recall that an instance object is passed in automatically as self. So, in your head, you can replace self in methods with the name of the instance. In the first case, when object john is instantiated, it is john.name that is set, as you can confirm below.

Also, without the presence of default arguments, both parameters to __init__() are required as part of the instantiation.

Once our instance was created, we can confirm that our instance attributes were indeed set by __init__() during instantiation. “Dumping” the instance within the interpreter tells us what kind of object it is. (We will discover later how we can customize our class so that rather than seeing the default <...> Python object string, a more desired output can be customized.)

The updatePhone() method requires one argument (in addition to self): the new phone number. We check our instance attribute right after the call to updatePhone(), making sure that it did what was advertised.

Subclassing with inheritance is a way to create and customize a new class type with all the features of an existing class but without modifying the original class definition. The new subclass can be customized with special functionality unique only to that new class type. Aside from its relationship to its parent or base class, a subclass has all the same features as any regular class and is instantiated in the same way as all other classes. Note below that a parent class is part of the subclass declaration:

We will now create our first subclass, EmplAddrBookEntry. In Python, when classes are derived, subclasses inherit the base class attributes, so in our case, we will not only define the methods __init__() and updateEmail(), but EmplAddrBookEntry will also inherit the updatePhone() method from AddrBookEntry.

Each subclass must define its own constructor if desired, otherwise the base class constructor will be called. However, if a subclass overrides a base class constructor, the base class constructor will not be called automatically—such a request must be made explicitly as we have above. For our subclass, we make an initial call to the base class constructor before performing any “local” tasks, hence the call to AddrBookEntry.__init__() to set the name and phone number. Our subclass sets two additional instance attributes, the employee ID and e-mail address, which are set by the remaining lines of our constructor.

Note how we have to explicitly pass the self instance object to the base class constructor because we are not invoking that method on an instance. We are invoking that method on an instance of a subclass. Because we are not invoking it via an instance, this unbound method call requires us to pass an acceptable instance (self) to the method.

We close this section with examples of how to create an instance of the subclass, accessing its attributes and invoking its methods, including those inherited from the parent class.

Core Style: Naming classes, attributes, and methods

Class names traditionally begin with a capital letter. This is the standard convention that will help you identify classes, especially during instantiation (which would look like a function call otherwise). In particular, data attributes should sound like data value names, and method names should indicate action toward a specific object or value. Another way to phrase this is: Use nouns for data value names and predicates (verbs plus direct objects) for methods. The data items are the objects acted upon, and the methods should indicate what action the programmer wants to perform on the object.

In the classes we defined above, we attempted to follow this guideline, with data values such as “name,” “phone,” and “email,” and actions such as “updatePhone” and “updateEmail.” This is known as “mixedCase” or “camelCase.” The Python Style Guide favors using underscores over camelCase, i.e,. “update_phone,” “update_email.” Classes should also be well named; some of those good names include “AddrBookEntry,” “RepairShop,” etc.

We hope that you now have some understanding of how object-oriented programming is accomplished using Python. The remaining sections of this chapter will take you deeper into all the facets of object-oriented programming and Python classes and instances.

Object-oriented design does not specifically require an object-oriented programming language. Indeed, OOD can be performed in purely structural languages such as C, but this requires more effort on the part of the programmer who must build data types with object qualities and characteristics. Naturally, OOP is simplified when a language has built-in OO properties that enable smoother and more rapid development of OO programs.

Conversely, an object-oriented language may not necessarily force one to write OO programs. C++ can be used simply as a “better C.” Java, on the other hand, requires everything to be a class, and further dictates only one class definition per source file. In Python, however, neither classes nor OOP are required for everyday programming. Even though it is a language that is object-oriented by design and that has constructs to support OOP, Python does not restrict or require you to write OO code for your application. Rather, OOP is a powerful tool that is at your disposal when you are ready to evolve, learn, transition, or otherwise move toward OOD. The creator of Python often refers to this phenomenon as being able to “see the forest through the trees.”

One of the most important reasons to consider working in OOD is that it provides a direct approach to modeling and solving real-world problems and situations. For example, let us attempt to model an automobile mechanic shop where you would take your car in for repair. There are two general entities we would have to create: humans who interact with and in such a “system,” and a physical location for the activities that define a mechanic shop. Since there are more and different types of the former, we will describe them first, then conclude with the latter.

A class called Person would be created to represent all humans involved in such an activity. Instances of Person would include the Customer, the Mechanic, and perhaps the Cashier. Each of these instances would have similar as well as unique behaviors. For example, all would have the talk() method as a means of vocal communication as well as a drive_car() method. Only the Mechanic would have the repair_car() method and only the Cashier would have a ring_sale() method. The Mechanic will have a repair_certification attribute while all Person would have a drivers_license attribute.

Finally, all of these instances would be participants in one overseeing class, called the RepairShop, which would have operating_hours, a data attribute that accesses time functionality to determine when Customers can bring in their vehicles and when Employees such as Mechanics and Cashiers show up for work. The RepairShop might also have a AutoBay class that would have instances such as SmogZone, TireBrakeZone, and perhaps one called GeneralRepair.

The point of our fictitious RepairShop is to show one example of how classes and instances plus their behaviors can be used to model a true-to-life scenario. You can probably also imagine classes such as an Airport, a Restaurant, a ChipFabPlant, a Hospital, or even a MailOrderMusic business, all complete with their own participants and functionality.

For those of you who are already familiar with all the lingo associated with OOP, here is how Python stacks up:

Python classes are created using the class keyword. In the simple form of class declarations, the name of the class immediately follows the keyword:

As outlined briefly earlier in this chapter, bases is the set of one or more parent classes from which to derive; and class_suite consists of all the component statements, defining class members, data attributes, and functions. Classes are generally defined at the top-level of a module so that instances of a class can be created anywhere in a piece of source code where the class is defined.

As with Python functions, there is no distinction between declaring and defining classes because they occur simultaneously, i.e., the definition (the class suite) immediately follows the declaration (header line with the class keyword) and the always recommended, but optional, documentation string. Likewise, all methods must also be defined at this time. If you are familiar with the OOP terms, Python does not support pure virtual functions (à la C++) or abstract methods (as in Java), which coerce the programmer to define a method in a subclass. As a proxy, you can simply raise the NotImplementedError exception in the base class method to get the same effect.

Data attributes are simply variables of the class we are defining. They can be used like any other variable in that they are set when the class is created and can be updated either by methods within the class or elsewhere in the main part of the program.

Such attributes are better known to OO programmers as static members, class variables, or static data. They represent data that is tied to the class object they belong to and are independent of any class instances. If you are a Java or C++ programmer, this type of data is the same as placing the static keyword in front of a variable declaration.

Static members are generally used only to track values associated with classes. In most circumstances, you would be using instance attributes rather than class attributes. We will compare the differences between class and instance attributes when we formally introduce instances.

Here is an example of using a class data attribute (foo):

Note that nowhere in the code above do you see any references to class instances.

A method, such as the myNoActionMethod method of the MyClass class in the example below, is simply a function defined as part of a class definition (thus making methods class attributes). This means that myNoActionMethod applies only to objects (instances) of MyClass type. Note how myNoActionMethod is tied to its instance because invocation requires both names in the dotted attribute notation:

Any call to myNoActionMethod by itself as a function fails:

A NameError exception is raised because there is no such function in the global namespace. The point is to show you that myNoActionMethod is a method, meaning that it belongs to the class and is not a name in the global namespace. If myNoActionMethod was defined as a function at the top-level, then our call would have succeeded.

We show you below that even calling the method with the class object fails.

This TypeError exception may seem perplexing at first because you know that the method is an attribute of the class and so are wondering why there is a failure. We will explain this next.

There are two ways to determine what attributes a class has. The simplest way is to use the dir() built-in function. An alternative is to access the class dictionary attribute __dict__, one of a number of special attributes that is common to all classes. Let us take a look at an example:

Using the class defined above, let us use dir() and the special class attribute __dict__ to see this class’s attributes:

There are a few more attributes added for new-style classes as well as a more robust dir() function. Just for comparison, here is what you would see for classic classes:

As you can tell, dir() returns a list of (just the) names of an object’s attributes while __dict__ is a dictionary whose attribute names are the keys and whose values are the data values of the corresponding attribute objects.

The output also reveals two familiar attributes of our class MyClass, showMyVersion and myVersion, as well as a couple of new ones. These attributes, __doc__ and __module__, are special class attributes that all classes have (in addition to __dict__). The vars() built-in function returns the contents of a class’s __dict__ attribute when passed the class object as its argument.

For any class C, Table 13.1 represents a list of all the special attributes of C:

In addition to the __dict__ attribute of the class MyClass we just defined above, we have the following:

__name__ is the string name for a given class. This may come in handy in cases where a string is desired rather than a class object. Even some built-in types have this attribute, and we will use one of them to showcase the usefulness of the __name__ string.

The type object is an example of one built-in type that has a __name__ attribute. Recall that type() returns a type object when invoked. There may be cases where we just want the string indicating the type rather than an object. We can use the __name__ attribute of the type object to obtain the string name. Here is an example:

__doc__ is the documentation string for the class, similar to the documentation string for functions and modules, and must be the first unassigned string succeeding the header line. The documentation string is not inherited by derived classes, an indication that they must contain their own documentation strings.

__bases__ deals with inheritance, which we will cover later in this chapter; it contains a tuple that consists of a class’s parent classes.

The aforementioned __dict__ attribute consists of a dictionary containing the data attributes of a class. When accessing a class attribute, this dictionary is searched for the attribute in question. If it is not found in __dict__, the hunt continues in the dictionary of base classes, in “depth-first search” order. The set of base classes is searched in sequential order, left-to-right in the same order as they are defined as parent classes in a class declaration. Modification of a class attribute affects only the current class’s dictionary; no base class __dict__ attributes are ever modified.

Python supports class inheritance across modules. To better clarify a class’s description, the __module__ was introduced in version 1.5 so that a class name is fully qualified with its module. We present the following example:

The fully qualified name of class C is “__main__.C”, i.e., source_ module.class_name. If class C was located in an imported module, such as mymod, we would see the following:

In previous versions of Python without the special attribute __module__, it was much more difficult to ascertain the location of a class simply because classes did not use their fully qualified names.

Finally, because of the unification of types and classes, when you access the __class__ attribute of any class, you will find that it is indeed an instance of a type object. In other words, a class is a type now! Because classic classes do not share in this equality (a classic class is a class object, and a type is a type object), this attribute is undefined for those objects.

Many other OO languages provide a new keyword with which to create an instance of a class. Python’s approach is much simpler. Once a class has been defined, creating an instance is no more difficult than calling a function—literally. Instantiation is realized with use of the function operator, as in the following example:

As you can see, creating instance mc of class MyClass consists of “calling” the class: MyClass(). The returned object is an instance of the class you called. When you “call” a class using the functional notation, the interpreter instantiates the object, and calls the closest thing Python has to a constructor (if you have written one [see the next section]) to perform any final customization such as setting instance attributes, and finally returns the instance to you.

Core Note: Classes and instances before and after

Classes and types were unified in 2.2, making Python behave more like other object-oriented languages. Instances of any class or type are objects of those types. For example, if you ask Python to tell you, it will say that an instance mc of the MyClass class is an instance of the MyClass class. Redundant yes, but the interpreter will not lie. Likewise, it will tell you that 0 is an instance of the integer type:

But if you look carefully and compare MyClass with int, you will find that both are indeed types:

In contrast for those of you using classic classes and Python versions earlier than 2.2, classes are class objects and instances are instance objects. There is no further relationship between the two object types other than an instance’s __class__ attribute refers to the class from which it was instantiated. Redefining MyClass as a classic class and running the same calls in Python 2.1 (note that int() has not been turned into a factory function yet... it was still only a regular built-in function):

To avoid any confusion, just keep in mind that when you define a class, you are not creating a new type, just a new class object; and for 2.2 and after, when you define a (new-style) class you are creating a new type.

When the class is invoked, the first step in the instantiation process is to create the instance object. Once the object is available, Python checks if an __init__() method has been implemented. By default, no special actions are enacted on the instance without the definition of (or the overriding) of the special method __init__(). Any special action desired requires the programmer to implement __init__(), overriding its default behavior. If __init__() has not been implemented, the object is then returned and the instantiation process is complete.

However, if __init__() has been implemented, then that special method is invoked and the instance object passed in as the first argument (self), just like a standard method call. Any arguments passed to the class invocation call are passed on to __init__(). You can practically envision the call to create the instance as a call to the constructor.

In summary, (a) you do not call new to create an instance, and you do not define a constructor: Python creates the object for you; and (b) __init__(), is simply the first method that is called after the interpreter creates an instance for you in case you want to prep the object a little bit more before putting it to use.

__init__() is one of many special methods that can be defined for classes. Some of these special methods are predefined with inaction as their default behavior, such as __init__(), and must be overridden for customization while others should be implemented on an as-needed basis. We will cover many more of these special methods throughout this chapter. You will find use of __init__() everywhere, so we will not present an example here.

The __new__() special method bears a much closer resemblance to a real constructor than __init__(). With the unification of types and classes in 2.2, Python users now have the ability to subclass built-in types, and so there needed to be a way to instantiate immutable objects, e.g., subclassing strings, numbers, etc.

In such cases, the interpreter calls __new__(), a static method, with the class and passing in the arguments made in the class instantiation call. It is the responsibility of __new__() to call a superclass __new__() to create the object (delegating upward).

The reason why we say that __new__() is more like a constructor than __init__() is that it has to return a valid instance so that the interpreter can then call __init__() with that instance as self. Calling a superclass __new__() to create the object is just like using a new keyword to create an object in other languages.

__new__() and __init__() are both passed the (same) arguments as in the class creation call. For an example of using __new__(), see Section 13.11.3.

Likewise, there is an equivalent destructor special method called __del__(). However, due to the way Python manages garbage collection of objects (by reference counting), this function is not executed until all references to an instance object have been removed. Destructors in Python are methods that provide special processing before instances are deallocated and are not commonly implemented since instances are seldom deallocated explicitly. If you do override __del__(), be sure to call any parent class __del__() first so those pieces can be adequately deallocated.

Example

In the following example, we create (and override) both the __init__() and __del__() constructor and destructor functions, respectively, then instantiate the class and assign more aliases to the same object. The id() built-in function is then used to confirm that all three aliases reference the same object. The final step is to remove all the aliases by using the del statement and discovering when and how many times the destructor is called.

Notice how, in the above example, the destructor was not called until all references to the instance of class C were removed, e.g., when the reference count has decreased to zero. If for some reason your __del__() method is not being called when you are expecting it to be invoked, this means that somehow your instance object’s reference count is not zero, and there may be some other reference to it that you are not aware of that is keeping your object around.

Also note that the destructor is called exactly once, the first time the reference count goes to zero and the object deallocated. This makes sense because any object in the system is allocated and deallocated only once. Summary:

• Do not forget to call a superclass __del__() first.

• Invoking del x does not call x.__del__()—as you saw above, it just decrements the reference count of x.

• If you have a cycle or some other cause of lingering references to an instance, an object’s __del__() may never be called.

• Uncaught exceptions in __del__() are ignored (because some variables used in __del__() may have already been deleted). Try not to do anything in __del__() not related to an instance.

• Implementing __del__() is not a common occurrence—only do it if you really know what you are doing.

• If you define __del__, and instance is part of a cycle, the garbage collector will not break the cycle—you have to do it yourself by explicitly using del.

Core Note: Keeping track of instances

Python does not provide any internal mechanism to track how many instances of a class have been created or to keep tabs on what they are. You can explicitly add some code to the class definition and perhaps __init__() and __del__() if such functionality is desired. The best way is to keep track of the number of instances using a static member. It would be dangerous to keep track of instance objects by saving references to them, because you must manage these references properly or else your instances will never be deallocated (because of your extra reference to them)! An example follows:

Instance attributes can be set any time after an instance has been created, in any piece of code that has access to the instance. However, one of the key places where such attributes are set is in the constructor, __init__().

Core Note: Instance attributes

Being able to create an instance attribute “on-the-fly” is one of the great features of Python classes, initially (but gently) shocking those coming from C++ or Java in which all attributes must be explicitly defined/ declared first.

Python is not only dynamically typed but also allows for such dynamic creation of object attributes during run-time. It is a feature that once used may be difficult to live without. Of course, we should mention to the reader that one must be cautious when creating such attributes.

One pitfall is when such attributes are created in conditional clauses: if you attempt to access such an attribute later on in your code, that attribute may not exist if the flow had not entered that conditional suite. The moral of the story is that Python gives you a new feature you were not used to before, but if you use it, you need to be more careful, too.

The dir() built-in function can be used to show all instance attributes in the same manner that it can reveal class attributes:

Similar to classes, instances also have a __dict__ special attribute (also accessible by calling vars() and passing it an instance), which is a dictionary representing its attributes:

Instances have only two special attributes (see Table 13.2). For any instance I:

We will now take a look at these special instance attributes using the class C and its instance c:

As you can see, c currently has no data attributes, but we can add some and recheck the __dict__ attribute to make sure they have been added properly:

The __dict__ attribute consists of a dictionary containing the attributes of an instance. The keys are the attribute names, and the values are the attributes’ corresponding data values. You will only find instance attributes in this dictionary—no class attributes or special attributes.

Core Style: Modifying __dict__

Although the __dict__ attributes for both classes and instances are mutable, it is recommended that you not modify these dictionaries unless or until you know exactly what you are doing. Such modification contaminates your OOP and may have unexpected side effects. It is more acceptable to access and manipulate attributes using the familiar dotted-attribute notation. One of the few cases where you would modify the __dict__ attribute directly is when you are overriding the __setattr__ special method. Implementing __setattr__() is another adventure story on its own, full of traps and pitfalls such as infinite recursion and corrupted instance objects—but that is another tale for another time.

Built-in types are classes, too... do they have the same attributes as classes? (The same goes for instances.) We can use dir() on built-in types just like for any other object to get a list of their attribute names:

Now that we know what kind of attributes a complex number has, we can access the data attributes and call its methods:

Attempting to access __dict__ will fail because that attribute does not exist for built-in types:

We first described class data attributes in Section 13.4.1. As a brief reminder, class attributes are simply data values associated with a class and not any particular instances like instance attributes are. Such values are also referred to as static members because their values stay constant, even if a class is invoked due to instantiation multiple times. No matter what, static members maintain their values independent of instances unless explicitly changed. (Comparing instance attributes to class attributes is barely like that of automatic vs. static variables, but this is just a vague analogy . . . do not read too much into it, especially if you are not familiar with auto and static variables.)

Classes and instances are both namespaces. Classes are namespaces for class attributes. Instances are namespaces for instance attributes.

There are a few aspects of class attributes and instance attributes that should be brought to light. The first is that you can access a class attribute with either the class or an instance, provided that the instance does not have an attribute with the same name.

Class Attributes More Persistent

Static members, true to their name, hang around while instances (and their attributes) come and go (hence independent of instances). Also, if a new instance is created after a class attribute has been modified, the updated value will be reflected. Class attribute changes are reflected across all instances:

Core Tip: Use a class attribute to modify itself (not an instance attribute)

As we have seen above, it is perilous to try and modify a class attribute by using an instance attribute. The reason is because instances have their own set of attributes, and there is no clear way in Python to indicate that you want to modify the class attribute of the same name, e.g., there is no global keyword like there is when setting a global inside a function (instead of a local variable of the same name). Always modify a class attribute with the class name, not an instance.

Methods, whether bound or not, are made up of the same code. The only difference is whether there is an instance present so that the method can be invoked. In most cases, you the programmer will be calling a bound method. Let us say that you have a class MyClass and an instance of it called mc, and you want to call the MyClass.foo() method. Since you already have an instance, you can just call the method with mc.foo(). Recall that self is required to be declared as the first argument in every method declaration. Well, when you call a bound method, self never needs to be passed explicitly when you invoke it with an instance. That is your bonus for being “required” to declare self as the first argument. The only time when you have to pass it in is when you do not have an instance and need to call a method unbound.

Calling an unbound method happens less frequently. The main use case for calling a method belonging to a class that you do not have an instance for is the case where you are deriving a child class and override a parent method where you need to call the parent’s constructor you are overriding. Let us look at an example back in the chapter introduction:

EmplAddrBookEntry is a subclass of AddrBookEntry, and we are overriding the constructor __init__(). Rather than cutting and pasting code from the parent constructor, we want to have as much code reuse as possible. This will also prevent bugs from being propagated because any fixes made would be propagated to us here in the child. This is exactly what we want—there is no need to copy lines of code. This all hinges on somehow being able to call the parent constructor, but how?

We would not have an instance of AddrBookEntry at runtime. What do we have? Well, we will have an instance of EmplAddrBookEntry, and it is so similar to AddrBookEntry, can’t we somehow use it instead? The answer is yes!

When an EmplAddrBookEntry is instantiated and __init__() called, there is very little difference between it and an instance of AddrBookEntry, mainly because we have not yet had a chance to customize our EmplAddrBookEntry instance to really make it different from AddrBookEntry.

This is the perfect place to call an unbound method. We will call the parent class constructor from the child class constructor and explicitly pass in the self argument as required by the (parent class) constructor (since we are without a parent class instance). The first line of __init__() in the child consists of a call to __init__() of the parent. We call it via the parent class name and pass in self plus its required arguments. Once that call returns, we can perform the (instance) customization that is unique to our (child) class.

Now let us look at some examples of these types of methods using classic classes (you can also use new-style classes if you want to):

The corresponding built-in functions are converted into their respective types and are reassigned back to the same variable name. Without the function calls, both would generate errors from the Python compiler, which is expecting regular method declarations with self. We can then call these functions from either the class or an instance... it makes no difference:

Now, seeing code like foo = staticmethod(foo) can irritate some programmers. There is something unsettling about it, and many folks were upset with such a flimsy syntax, although van Rossum had pointed out that it was to be temporary until the semantics were worked out with the community. In Section 11.3.6 of Chapter 11, “Functions,” we looked at decorators, a new feature introduced in Python 2.4. They are used in places where you want to apply a function to a function object but want to rebind the new function object to the original variable. This is a perfect place to use them to partially clean up the syntax. By using decorators, we can avoid the reassignment above:

The syntax for creating a subclass looks just like that for a regular (new-style) class, a class name followed by one or more parent classes to inherit from:

If your class does not derive from any ancestor class, use object as the name of the parent class. The only example that differs is the declaration of a classic class that does not derive from ancestor classes—in this case, there are no parentheses:

We have already seen some examples of classes and subclasses so far, but here is another simple example:

In Section 13.4.4, we briefly introduced the __bases__ class attribute, which is a tuple containing the set of parent classes for any (sub)class. Note that we specifically state “parents” as opposed to all base classes (which includes all ancestor classes). Classes that are not derived will have an empty __bases__ attribute. Let us look at an example of how to make use of __bases__.

In the example above, although C is a derived class of both A (through B) and B, C’s parent is B, as indicated in its declaration, so only B will show up in C.__bases__. On the other hand, D inherits from two classes, A and B. (Multiple inheritance is covered in Section 13.11.4.)

Let us create another function in P that we will override in its child class:

Now let us create the child class C, subclassed from parent P:

Although C inherits P’s foo() method, it is overridden because C defines its own foo() method. One reason for overriding methods is because you may want special or different functionality in your subclass. Your next obvious question then must be, “Can I call a base class method that I overrode in my subclass?”

The answer is yes, but this is where you will have to invoke an unbound base class method, explicitly providing the instance of the subclass, as we do here:

Notice that we already had an instance of P called p from above, but that is nowhere to be found in this example. We do not need an instance of P to call a method of P because we have an instance of a subclass of P which we can use, c. You would not typically call the parent class method this way. Instead, you would do it in the overridden method and call the base class method explicitly:

Note how we pass in self explicitly in this (unbound) method call. A better way to make this call would be to use the super() built-in method:

super() will not only find the base class method, but pass in self for us so we do not have to as in the previous example. Now when we call the child class method, it does exactly what you think it should do:

Core Note: Overriding __init__ does not invoke base class __init__

Similar to overriding non-special methods above, when deriving a class with a constructor __init__(), if you do not override __init__(), it will be inherited and automatically invoked. But if you do override __init__() in a subclass, the base class __init__() method is not invoked automatically when the subclass is instantiated. This may be surprising to those of you who know Java.

If you want the base class __init__() invoked, you need to do that explicitly in the same manner as we just described, calling the base class (unbound) method with an instance of the subclass. Updating our class C appropriately results in the following desired execution:

In the above example, we call the base class __init__() method before the rest of the code in our own __init__() method. It is fairly common practice (if not mandatory) to initialize base classes for setup purposes, then proceed with any local setup. This rule makes sense because you want the inherited object properly initialized and “ready” by the time the code for the derived class constructor runs because it may require or set inherited attributes.

Those of you familiar with C++ would call base class constructors in a derived class constructor declaration by appending a colon to the declaration followed by calls to any base class constructors. Whether the programmer does it or not, in Java, the base class constructor always gets called (first) in derived class constructors.

Python’s use of the base class name to invoke a base class method is directly comparable to Java’s when using the keyword super, and that is why the super() built-in function was eventually added to Python, so you could “do the correct thing” functionally:

The nice thing about using super() is that you do not need to give any base class name explicitly... it does all the legwork for you! The importance of using super() is that you are not explicitly specifying the parent class. This means that if you change the class hierarchy, you only need to change one line (the class statement itself) rather than tracking through what could be a large amount of code in a class to find all mentions of what is now the old class name.

Not being able to subclass a standard data type was one of the most significant problems of classic classes. Fortunately that was remedied back in 2.2 with the unification of types and classes and the introduction of new-style classes. Below we present two examples of subclassing a Python type, one mutable and the other not.

Like C++, Python allows for subclassing from multiple base classes. This feature is commonly known as multiple inheritance. The concept is easy, but the hard work is in how to find the correct attribute when it is not defined in the current (sub)class. There are two different aspects to remember when using multiple inheritance. The first is, again, being able to find the correct attribute. Another is when you have overridden methods, all of which call parent class methods to “take care of their responsibilities” while the child class takes care of its own obligations. We will discuss both simultaneously but focus on the latter as we describe the method resolution order.

Method Resolution Order (MRO)

In Python versions before 2.2, the algorithm was simple enough: a depth-first left-to-right search to obtain the attribute to use with the derived class. Unlike other Python algorithms that override names as they are found, multiple inheritance takes the first name that is found.

Because of the entirely new structure of classes and types and the subclassing of built-in types, this algorithm was no longer feasible, so a new MRO algorithm had to be developed. The initial one debuting in 2.2 was a good attempt but had a flaw (see Core Note below). It was immediately replaced in 2.3, which is the current one that is in use today.

The exact resolution order is complex and is beyond the scope of this text, but you can read about it in the references given later on in this section. We can say that the new resolution method is more breadth-first than it is depth-first.

Core Note: uses a unique yet faulty MRO

Python 2.2 was the first release using a new-style MRO that had to replace the algorithm from classic classes due to the reasons outlined above.

For 2.2, the algorithm had the basic idea of following the hierarchy of each ancestor class and building a list of classes encountered, strategically removing duplicates. However, it was pointed out on the core Python developers mailing list that it fails to maintain monotonicity (order preservation), and had to be replaced by the new C3 algorithm that has been in place since 2.3.

Let us give you an example to see how the method resolution order differs between classic and new-style classes.

Simple Attribute Lookup Example

The simple example below will highlight the differences between the old and new styles of resolution. The script consists of a pair of parent classes, a pair of child classes, and one grandchild class.

In Figure 13-2, we see the class relationships between the parent, children, and grandchildren classes. P1 defines foo(), P2 defines foo() and bar(), and C2 defines bar(). Let us now demonstrate the behavior of both classic and new-style classes.

Figure 13-2. Relationships between parent, children, and grandchild classes as well as the methods they define.

Classic Classes

We are going to use classic classes first. Upon executing the above declarations in the interactive interpreter, we can confirm the resolution order that classic classes use, depth-first, left to right:

When calling foo(), it looks in the current class (GC) first. If it cannot be found, it goes up to its immediate parent, C1. The search fails there so it continues up the tree to its parent, P1, which is where foo() is found.

Likewise for bar(), it searches through GC, C1, and P1 before then finding it in P2. C2.bar() is never found because of the resolution order used. Now, you may be thinking, “I would prefer to call C2’s bar() because it is closer to me in the inheritance tree, thus more relevant.” In this case, you can still use it, but you have to do it in the typical unbound fashion by invoking its fully qualified name and providing a valid instance:

    >>> C2.bar(gc)
    called C2-bar()

New-Style Classes

Now uncomment the (object) next to the class declarations for P1 and P2 and reexecute. The new-style method resolution gives us something different:

Instead of following the tree up each step, it looks at the siblings first, giving it more of a breadth-first flavor. When looking for foo(), it checks GC, followed by C1 and C2, and then finds it in P1. If P1 did not have it, it would have gone to P2. The bottom line for foo() is that both classic and new-style classes would have found it in P1, but they took different paths to get there.

The result for bar() is different, though. It searches GC and C1, and finds it next in C2 and uses it there. It does not continue up to the grandparents P1 and P2. In this case, the new-style resolution fit into the scheme better if you did prefer to call the “closest” bar() from GC. And of course, if you still need to call one higher up, just do it in an unbound manner as before:

    >>> P2.bar(gc)
    called P2-bar()

New-style classes also have an __mro__ attribute that tells you what the search order is:

*MRO Problems Caused by Diamonds

The classic class method resolution never gave folks too many problems. It was simple to explain and easy to understand. Most classes were single inheritance, and multiple inheritance was usually limited to mixing two completely discrete classes together. This is where the term mix-in classes (or “mix-ins”) comes from.

Why the Classic Classes MRO Fails

The unification of types and classes in 2.2 brought about a new “problem,” and that is related to all (root) classes inheriting from object, the mother of all types. The diagram of a simple multiple inheritance hierarchy now formed a diamond. Taking some inspiration from Guido van Rossum’s essay, let us say that you have classic classes B and C, as defined below where C overrides its constructor but B does not, and D inherits from both B and C:

When we instantiate D, we get:

    >>> d = D()
    the default constructor

Figure 13.3 illustrates the class hierarchy for B, C, and D, as well as the problem introduced when we change the code to use new-style classes:

Figure 13-3. Inheritance problems are caused by the appearance of the base class required by new-style classes, forming a diamond shape in the inheritance hierarchy. An instance of D should not miss an upcall to C nor should it upcall to A twice (since both B and C derive from A). Be sure to read the “Cooperative Methods” section of Guido van Rossum’s essay for further clarification.

Not much change here other than adding (object) to both class declarations, right? That is true, but as you can see in the diagram, the hierarchy is now a diamond; the real problem is in the MRO now. If we used the classic class MRO, when instantiating D, we no longer get C.__init__()... we get object.__init__()! This is the exact reason why the MRO needed to be changed.

Although we saw that it does change the way attributes are looked up in our example above with the GC class, you do not have to worry about lots of code breaking. Classic classes will still use the old MRO while new-style classes will use its MRO. Again, if you do not need all of the features of the new-style classes, there is nothing wrong with continuing to develop using classic classes.

The issubclass() Boolean function determines if one class is a subclass or descendant of another class. It has the following syntax:

issubclass() returns True if the given subclass sub is indeed a subclass of the superclass sup (and False otherwise). This function allows for an “improper” subclass, meaning that a class is viewed as a subclass of itself, so the function returns True if sub is either the same class as sup or derived from sup. (A “proper” subclass is strictly a derived subclass of a class.)

Beginning with Python 2.3, the second argument of issubclass() can be tuple of possible parent classes for which it will return True if the first argument is a subclass of any of the candidate classes in the given tuple.

The isinstance() Boolean function is useful for determining if an object is an instance of a given class. It has the following syntax:

isinstance() returns True if obj1 is an instance of class obj2 or is an instance of a subclass of obj2 (and False otherwise), as indicated in the following examples:

Note that the second argument should be a class; otherwise, you get a TypeError. The only exception is if the second argument is a type object. This is allowed because you can also use isinstance() to check if an object obj1 is of the type obj2, i.e.,

If you are coming from Java, you may be aware of the warning against using its equivalent, instanceof(), due to performance reasons. A call to Python’s isinstance() will not have the same performance hit primarily because it only needs it to perform a quick search up the class hierarchy to determine what classes it is an instance of, and even more importantly, it is written in C!

Like issubclass(), isinstance() can also take a tuple as its second argument. This feature was added in Python 2.2. It will return True if the first argument is an instance of any of the candidate types and classes in the given tuple. Also be sure to read more about isinstance() in Section 13.16.1 on page 595.

The *attr() functions can work with all kinds of objects, not just classes and instances. However, since they are most often used with those objects, we present them here. One thing that might throw you off is that when using these functions, you pass in the object you are working on as the first argument, but the attribute name, the second argument to these functions, is the string name of the attribute. In other words, when operating with obj.attr, the function call will be like *attr(obj, 'attr'...)—this will be clear in the examples that follow.

The hasattr() function is Boolean and its only purpose is to determine whether or not an object has a particular attribute, presumably used as a check before actually trying to access that attribute. The getattr() and setattr() functions retrieve and assign values to object attributes, respectively. getattr() will raise an AttributeError exception if you attempt to read an object that does not have the requested attribute, unless a third, optional default argument is given. setattr() will either add a new attribute to the object or replace a pre-existing one. The delattr() function removes an attribute from an object.

Here are some examples using all the *attr() BIFs:

We first experienced dir() in Exercises 2-12, 2-13, and 4-7. In those exercises, we used dir() to give us information about all the attributes of a module. We now know that dir() can be applied to objects as well.

In Python 2.2, dir() received a significant upgrade. Because of these changes, the voluntarily implemented __members__ and __methods__ data attributes have been deprecated. dir() provides more details than the old one. According to the documentation, “In addition to the names of instance variables and regular methods, it also shows the methods that are normally invoked through special notations, like __iadd__ (+=), __len__ (len()), __ne__ (!=).” Here are more specifics from the Python documentation:

The super() function was added in 2.2 for new-style classes. The purpose of this function is to help the programmer chase down the appropriate superclass with which the proper method can be invoked. In simple cases, the programmer will likely just call the ancestor class method in an unbound fashion. Using super() simplifies the task of search for a suitable ancestor and passes in the instance or type object on your behalf when you call it.

In Section 13.11.4, we described the method resolution order (MRO) that is used to chase down attributes in ancestor classes. For each class defined, an attribute named __mro__ is created as a tuple that lists the classes that need to be searched, in the order they are searched. Here is its syntax:

Given type, super() “returns the superclass” of type. You may also pass in obj, which should be of the type type if you want the superclass to be bound, otherwise it will be unbound. The obj argument can also be a type, but it needs to be a subclass of type. In summary, when obj is given:

Actually, super() is a factory function that makes a super object that uses the __mro__ attribute for a given class to find the appropriate superclass. Most notably, it searches that MRO starting from the point where the current class is found. For more details, again please see Guido van Rossum’s essay on type and class unification, where he even gives a pure Python implementation of super() so you can get a better idea of how it works!

Final thought... super()’s primary use is in the lookup of a superclass attribute, e.g., super(MyClass, self).__init__(). If you are not performing such a lookup, you probably do not need to be using super().

There are various examples how to use super() scattered throughout this chapter. Also be sure to read the important notes about super() in Section 13.11.2, especially the Core Note in that section.

The vars() built-in function is similar to dir() except that any object given as the argument must have a __dict__ attribute. vars() will return a dictionary of the attributes (keys) and values of the given object based on the values in its __dict__ attribute. If the object provided does not have such an attribute, an TypeError exception is raised. If no object is provided as an argument to vars(), it will display the dictionary of attributes (keys) and the values of the local namespace, i.e., locals(). We present an example of calling vars() with a class instance:

Table 13.3 summarizes the built-in functions for classes and class instances.

Table 13.3. Built-in Functions for Classes, Instances, and Other Objects

Our first example is totally trivial. It is based to some extent on the RoundFloat class we saw earlier in the section on subclassing Python types. This example is simpler. In fact, we are not even going to subclass anything (except object of course)... we do not want to “take advantage” of all the “goodies” that come with floats. No, this time, we want to create a barebones example so that you have a better idea of how class customization works. The premise of this class is still the same as the other one: we just want a class to save a floating point number rounded to two decimal places.

This class takes a single floating point value—it asserts that the type must be a float as it is passed to the constructor—and saves it as the instance attribute value. Let us try to execute it and create an instance of this class:

As you can see, it chokes on invalid input, but provides no output if input was valid. But look what happens when we try to dump the object in the interactive interpreter. We get some information, but this is not what we were looking for. (We wanted to see the numeric value, right?) And calling print does not apparently help, either.

Unfortunately, neither print (using str()) nor the actual object’s string representation (using repr()) reveals much about our object. One good idea would be to implement either __str__() or __repr__(), or both so that we can “see” what our object looks like. In other words, when you want to display your object, you actually want to see something meaningful rather than the generic Python object string (<object object at id>). Let us add a __str__() method, overriding the default behavior:

Now we get the following:

We still have a few problems ... one is that just dumping the object in the interpreter still shows the default object notation, but that is not so bad. If we wanted to fix it, we would just override __repr__(). Since our string representation is also a Python object, we can make the output of __repr__() the same as __str__().

To accomplish this, we can just copy the code from __str__() to __repr__(). This is a simple example, so it cannot really hurt us, but as a programmer, you know that is not the best thing to do. If a bug existed in __str__(), then we will copy that bug to __repr__().

The best solution is to recall that the code represented by __str__() is an object too, and like all objects, references can be made to them, so let us just make __repr__() an alias to __str__():

In the second example with 5.5964, we see that it rounds the value correctly to 5.6, but we still wanted two decimal places to be displayed. One more tweak, and we should be done. Here is the fix:

And here is the resulting output with both str() and repr() output:

In our original RoundFloat example at the beginning of this chapter, we did not have to worry about all the fine-grained object display stuff; the reason is that __str__() and __repr__() have already been defined for us as part of the float class. All we did was inherit them. Our more “manual” version required additional work from us. Do you see how useful derivation is? You do not even need to know how far up the inheritance tree the interpreter needs to go to find a declared method that you are using without guilt. We present the full code of this class in Example 13.2.

Now let us try a slightly more complex example.

For our first realistic example, let us say we wanted to create a simple application that manipulated time as measured in hours and minutes. The class we are going to create can be used to track the time worked by an employee, the amount of time spent online by an ISP (Internet service provider) subscriber, the amount of total uptime for a database (not inclusive of downtime for backups and upgrades), the total amount of time played in a poker tournament, etc.

For our Time60 class, we will take integers as hours and minutes as input to our constructor.

Let us create another new class, NumStr, consisting of a number-string ordered pair, called n and s, respectively, using integers as our number type. Although the “proper” notation of an ordered pair is (n, s), we choose to represent our pair as [n :: s] just to be different. Regardless of the notation, these two data elements are inseparable as far as our model is concerned. We want to set up our new class, called NumStr, with the following characteristics:

You may also wrap classes, but this does not make as much sense because there is already a mechanism for taking an object and wrapping it in a manner as described above for a standard type. How would you take an existing class, mimic the behavior you desire, remove what you do not like, and perhaps tweak something to make the class perform differently from the original class? That process, as we discussed recently, is derivation.

Figure 13-4. Wrapping a Type

Let us take a look at an example. Here we present a class that wraps nearly any object, providing such basic functionality as string representations with repr() and str(). Additional customization comes in the form of the get() method, which removes the wrapping and returns the raw object. All remaining functionality is delegated to the object’s native attributes as retrieved by __getattr__() when necessary.

Here is an example of a wrapping class:

In our first example, we will use complex numbers, because of all Python’s numeric types, complex numbers are the only one with attributes: data attributes as well as its conjugate() built-in method. Remember that attributes can be both data attributes as well as functions or methods:

Once we create our wrapped object type, we obtain a string representation, silently using the call to repr() by the interactive interpreter. We then proceed to access all three complex number attributes, none of which is defined for our class. Confirm this by looking for real, imag, and conjugate in our class definition ... they are not there!

The accessing of these attributes is delegated to the object via the getattr() method. The final call to get() is not delegated because it is defined for our object—it returns the actual data object that we wrapped.

Our next example using our wrapping class uses a list. We will create the object, then perform multiple operations, delegating each time to list methods.

Notice that although we are using a class instance for our examples, they exhibit behavior extremely similar to the data types they wrap. Be aware, however, that only existing attributes are delegated in this code.

Special behaviors that are not in a type’s method list will not be accessible since they are not attributes. One example is the slicing operations of lists which are built-in to the type and not available as an attribute like the append() method, for example. Another way of putting it is that the slice operator ( [ ] ) is part of the sequence type and is not implemented through the __getitem__() special method.

The AttributeError exception results from the fact that the slice operator invokes the __getitem__() method, and __getitem__() is not defined as a class instance method nor is it a method of list objects. Recall that getattr() is called only when an exhaustive search through an instance’s or class’s dictionaries fails to find a successful match. As you can see above, the call to getattr() is the one that fails, triggering the exception.

However, we can always cheat by accessing the real object [with our get() method] and its slicing ability instead:

You probably have a good idea now why we implemented the get() method—just for cases like this where we need to obtain access to the original object. We can bypass assigning local variable (realList) by accessing the attribute of the object directly from the access call:

The get() method returns the object, which is then immediately indexed to obtain the sliced subset.

Once you become familiar with an object’s attributes, you begin to understand where certain pieces of information originate and are able to duplicate functionality with your newfound knowledge:

This concludes the sampling of our simple wrapping class. We have only just begun to touch on class customization with type emulation. You will discover that you can an infinite number of enhancements make to further increase the usefulness of your code. One such enhancement is to add timestamps to objects. In the next subsection, we will add another dimension to our wrapping class: time.

Creation time, modification time, and access time are familiar attributes of files, but nothing says that you cannot add this type of information to objects. After all, certain applications may benefit from these additional pieces of information.

If you are unfamiliar with using these three pieces of chronological data, we will attempt to clarify them. The creation time (or “ctime”) is the time of instantiation, the modification time (or “mtime”) refers to the time that the core data was updated [accomplished by calling the new set() method], and the access time (or “atime”) is the timestamp of when the data value of the object was last retrieved or an attribute was accessed.

Proceeding to updating the class we defined earlier, we create the module twrapme.py, given in Example 13.7.

After the usual setup, we create our descriptor class with a class attribute (saved) that keeps track of all attributes with descriptor access. When a descriptor is created, it registers and saves the name of the attribute (passed in from the user).

When fetching an attribute, we need to ensure that users do not use it before they have even assigned a value to it. If it passes that test, then we attempt to open the pickle file to read in the saved value. An exception is raised if somehow the file cannot be opened, either because it was erased (or never created), or if it was corrupted or somehow cannot be unserialized by the pickle module.

Saving the attribute takes several steps: open the pickle file for write (either creating it for the first time or wiping out one that was already there), serializing the object to disk, and registering the name so users can retrieve the value. An exception is thrown if the object cannot be pickled. Note that if you are using Python older than 2.5 you can never merge the try-except and try finally statements together (lines 30-38).

Finally, if the attribute is explicitly deleted, the file is removed, and the name unregistered.

Here is some sample usage of this class:

Attribute access appears normal, and the programmer cannot really tell that an object is pickled and stored to the filesystem (except in the last example where we tried to pickle a module, a no-no). We also put in a handler for cases when the pickle file gets corrupted. This is also the first descriptor where we have implemented __delete__().

One thing to keep in mind with all of our examples is that we did not use the instance obj at all. Do not confuse obj with self as the latter is the instance of the descriptor, not the instance of the original class.