Numeric Types: Integer and Float

Python supports several numeric types including the most basic forms: integer and floating point.

Python integers are unusual in that they are theoretically of infinite size. In practice, integers are limited by the size of your computer’s memory. Integers support all the usual numeric operations, such as addition, subtraction, multiplication and so on. You perform arithmetic operations using traditional infix notation. For example, to add two integers,

  >>> 5 + 4
  9

or:

  >>> result = 12 + 8
  >>> print (result)
  20

Literal integer values are, by default, expressed in decimal. You can use other bases by prefixing the number with a zero and the base’s initial. Thus, binary is represented as 0bnnn, octal as 0onnn, and hexadecimal as 0xnnn.

The type of an integer is int, and you can use it to create integers from floating-point numbers or numeric string representations such as '123', like this:

  >>> int(5.0)
  5
  >>> int('123')
  123

int can also convert from nondecimal bases (covering any base up to 36, not just the usual binary, octal, and hexadecimal) using a second, optional, parameter. To convert a hexadecimal (base 16) string representation to an integer, you can use:

  >>> intValue = int('AB34',16)
  43828

Python floating-point numbers are of type float. Like int you can use float() to convert string representations, like '12.34' to float, and you can also use it to convert an integer number to a float value. Unlike integers, float() cannot handle strings for different bases.

The float type also supports the normal arithmetic operations, as well as several rounding options. Python floats are based on the Institute of Electrical & Electronic Engineering (IEEE) standards and have the same ranges as the underlying computer architecture. They also suffer the same levels of imprecision that make comparing float values a risky option. Python provides modules for handling fixed precision decimal numbers (decimal) and rational fractions (fractions) to help alleviate this issue. Python also natively supports a complex, or imaginary, number type called complex. These are all typically used for fairly special purposes, so they are not covered here.

The Boolean Type

Python supports a Boolean type, bool, with literal values True and False. The default value of a bool is False; that is, bool() yields False.

Python also supports the concept of truth-like values for other types. For example, integers are considered False if their value is zero. Anything else is considered True. The same applies to float values where 0.0 is False and anything else is True.

You can convert Boolean values to integers using int(), in which case False is represented as 0 and True as 1.

The Boolean type has most of the Boolean algebra operations you’d expect, including and, or, and not, but—surprisingly—not xor.

In addition to the Boolean type, Python also supports bitwise Boolean operations on integers. That is to say that Python treats each corresponding pair of bits within two integers as Boolean values in their own right and applies the corresponding operation to each pair of bits. These operations include bitwise and (&), or (|), not (^) and, this time, xor (~), as well as bit shift operators for moving bit patterns left (<<) or right (>>). You look more closely at these bitwise operations later in the chapter.

The None Type

The None type represents a null object. There is only one None object in the Python environment, and all references to None use that same single instance. This means that equality tests with None are usually replaced by an identity test, like so,

  aVariable is None

rather than:

  aVariable == None

None is the default return value of a Python function. It is also often used as a place marker or flag for default parameters in functions. None is not callable and so cannot be used as a conversion function to convert other types to None. None is considered to have a Boolean value of False.

Collection Types

As already mentioned Python has several types representing different kinds of collections or sequences. These are: strings, bytes, tuples, lists, dictionaries and sets. You will see the similarities and differences in each as they are discussed in the following sections. A standard library module called collections provides several other more specialized collection types. You will see occasional references to these in the sections that follow.

Several common features apply to all collections, and rather than bore you by repeating them for each type, they are covered here.

You can get the length of any collection in Python by using the built-in len() function. It takes a collection object argument and returns the number of elements.

You can access the individual elements of a collection using indexing. You do this by providing an index number (or a valid key value for dictionaries) inside square brackets. Collection indices start at zero. You can also index backward from the end by using negative indices so that the last item in the collection will have an index of –1.

Whereas indexing is used to access just one particular element of a collection, you can use slicing to access multiple items in the collection. Slicing consists of a start index, an end index, and a step size, and the numbers are separated by colons. Slicing is not valid for dictionaries or sets. The step size argument enables you to, for example, select every other element. All values are optional, and the defaults are the start of the collection, the last item in the collection, and a step size of one. The slice returned consists of all (selected) elements from start to end-1.

Here are a few examples of slicing applied to a string, entered at the Python interactive prompt:

  >>> '0123456789'[:]
  '0123456789'
  >>> '0123456789'[3:]
  '3456789'
  >>> '0123456789'[:3]
  '012'
  >>> '0123456789'[3:7]
  '3456'
  >>> '0123456789'[3:7:2]
  '35'
  >>> '0123456789'[::3]
  '0369'

You can sort most collections by using the sorted() function. The return value is a sorted list containing the original collection elements. Optional arguments to sorted() provide flexibility in how the elements are sorted and in what order.

In general, empty collections are treated as False in Boolean expressions and True otherwise. Two functions, any() and all(), refine the concept to allow more precise tests. The any() function takes a collection as an argument and returns True if any member of the collection is true. The all() function takes a collection as an argument and returns True if—and only if—all the members are true.

  >>> 'using single quotes'
  'using single quotes'
  >>> "using double quotes"
  'using double quotes'
  >>> print('''triple single quotes spanning
  ... multiple lines ''')
  triple single quotes spanning
  multiple lines

  >>> s = b'Alphabet soup'
  >>> c = b'A'
  >>> s[0] == c
  False
  >>> s[0] == c[0]
  True

  >>> print(divmod(12,7))
  (1, 5)
  >>> q,r = divmod(12,7)
  >>> print (q)
  1
  >>> print (r)
  5

  >>> [n*n for n in range(1,11) if not n*n % 2]
  [4, 16, 36, 64, 100]

  >>> {'aKey':'avalue', 2:7, 'booleans':{False:0,True:1}}
  {'aKey': 'avalue', 2: 7, 'booleans': {False: 0, True: 1}}

  >>> D['aKey']
  'avalue'
  >>> D['booleans'][True]
  1

  myset = {1,2,3,4,5}

Structuring Your Program

Python programs do not have any required, predefined entry point (for example a main() function) and are simply expressed as source code in a text file that is read and executed in order starting at the top of the file. (Definitions, such as functions, are executed in the sense that the function is created and assigned to a name, but the internal code is not executed until the function is called.)

Python does not have any special syntax to indicate whether a source file is a program or a module and, as you will see, a given file can be used in either role. A typical executable program file consists of a set of import statements to bring in any code modules needed, some function and class definitions, and some directly executable code.

In practice, for a nontrivial program, most function and class definitions will exist in module files and be included in the imports. This leaves a short section of driver code to start the application. Often this code will be placed in a function, and the function will often be called main(), but that is purely a nod to programming convention, not a requirement of Python.

Finally this “main” function needs to be called. This is often done within a special if statement at the end of the main script. It looks like this:

  if __name__ == "__main__":
      main()

When Python detects that a program file is being executed by the interpreter rather than imported as a module, it sets the special variable __name__ (note the double underscores on either side) to "__main__". This means that any code inside this if block is executed only when the script is run as a main program and not when the file is imported by another program. If the file is only ever expected to be used as a module, the main() function may be replaced by a test() function that executes a set of unit tests. Again, the actual name used is of no significance to Python.

Using Sequences, Blocks and Comments

The most fundamental programming structure is a sequence of statements. Normally Python statements occur on a line by themselves, so a sequence is simply a series of lines.

  x = 2
  y = 7
  z = 9

In this case the statements are all assignments. Other valid statements include function calls, module imports or definitions. Definitions include functions and classes. The various control structures described in the following sections are also valid statements.

Python is a block-structured language, and blocks of code are indicated by indentation level. The amount of indentation is quite flexible; although most Python programmers stick to using three or four spaces to optimize readability, Python doesn’t care. Different Integrated Development Environments (IDEs) and text editors have their own ideas about how indentation is done. If you use multiple programming tools, you may find you get indentation errors reported because the tools have used different combinations of tabs and spaces. If possible, set up your editor to use spaces instead of tabs.

The exception to the indentation rule is comments. A Python comment starts with a # symbol and continues to the end of the line. Python accepts comments that start anywhere on the line regardless of current indentation level, but by convention, programmers tend to retain indentation level, even for comments.

Selecting an Execution Path

Python supports a limited set of selection options. The most basic structure is the if/elif/else construct. The elif and else parts are optional. It looks like this:

  if pages < 9:
      print("It's too short")
  elif pages > 99:
      print("It's too long")
  else: print("Perfect")

Notice the colon at the end of each test expression. That is Python’s indicator that a new block of code is coming up. It has no start and end block markers (such as {}); the colon is the only indication, and the block must either occur on the same line as the colon, if it’s only a single line block, or as an indented block of code. Many Python programmers prefer the indented block style even for single line blocks.

Also note that there can be an arbitrary number of elif tests, but only a single else clause—or none at all.

The other selection structure you will find in Python is the conditional expression selector. This produces one of several values depending on the given test conditions. It looks like this:

  <a value> if <an expression> else <another value>

An example might be where a screen coordinate is being incremented until a certain limit (perhaps the screen’s maximum resolution) and then reset to 0. That could be written as:

  coord = coord + increment if coord < limit else 0

This is equivalent to the more traditional structure shown here:

  if coord < limit
      coord += increment
  else:
      coord = 0

You should use caution when using conditional expressions because it is very easy to create obscure code. If in doubt you should use the expanded if/else form.

One final comment on Python’s comparison expressions is worth making here. In many programming languages, if you want to test whether a value lies between two limits, you need two separate tests, like this:

  if aValue < upperLimit and aValue > lowerLimit:
      # do something here

Python will be quite happy to process code like that, but it offers a useful shortcut in that you can combine the comparisons as shown here:

  if lowerLimit < avalue < upperLimit:
      # do something here

Iteration

Python offers several alternatives for iteration. The most fundamental and general is the while loop. It looks like this:

  while BooleanExpression:
      aBlockOfCode
  else:
      anotherBlock

Notice the colon (:) at the end of the while statement, which signifies that a block of code follows. Also notice the indentation of the block. The block will, in principle, be executed for as long as BooleanExpression remains true. However, there are two ways you can exit out of a while loop regardless of the BooleanExpression value. These are the break statement, which exits the loop immediately, and a return statement if the loop is inside a function definition. A return statement exits the function immediately and so will also exit any loop within the function.

The else clause is optional and is rarely used in practice. It is executed any time the BooleanExpression is False, including when the loop exits normally. It will not be executed if the loop is exited by a break or return statement.

One very common while loop idiom is to use True as the test condition to create an infinite loop and then have a break test within the body of the loop. Here is an example where the loop reads user commands and processes them. If the command contains the letter q, it exits.

  while True:
      command = input('Enter a command[rwq]: ')
      if 'q' in command.lower(): break
      if command.lower() == 'r':
         # process 'r'
      elif command.lower() == 'w':
         # process 'w'
      else:
         print('Invalid command, try again')

There is a companion statement to break, namely continue. Whereas break exits both the block and the loop, continue exits the block for the current loop iteration only. Control then returns to the while statement and, if appropriate, a new iteration of the block will commence.

The next significant looping construct in Python is the for loop, which looks like this:

  for item in <iterable>:
      code block
  else:
      another code block

The for loop takes each item in the iterable and executes the code block once per item. You can terminate the loop early using break or return as described for the while loop. You can terminate a single iteration of the loop using continue as described earlier.

The else block is executed when all the iterations are completed. It will not be executed if the loop exits via a break or return.

The iterable is anything that complies with Python’s iterator protocol. In practice this is usually a collection such as a list, tuple, or a function that returns a collection of values such as range(). The open() function returns a file iterator that enables you to loop over a file without first reading it into memory. It’s possible to define your own custom iterable classes, too.

One common function that is particularly used with for loops is enumerate(). This function returns tuples containing both the iterable item and a sequence number that, by default, is equivalent to a list index. This means that the for block can more easily update the iterable directly. enumerate() takes a second optional argument that specifies the sequence starting number, which you could use, for example, to indicate the line number in a file, starting with 1 rather than the 0 default.

Here is an example that illustrates some of these points printing a file with associated line numbers:

  for number, line in enumerate(open('myfile.txt')):
      print(number, '	', line)

Finally, Python has a couple of inline loop structures. You saw one of these, the list comprehension, in the discussion of lists earlier in the chapter.

A list comprehension is a specific application of a more general loop form known as a generator expression that you can use where you might otherwise have a sequence of literal values. If you recall the list comprehension example earlier in the chapter, you used it to populate a list with the even squares from 1 to 10, like this:

  >>> [n*n for n in range(1,11) if not n*n % 2]
  [4, 16, 36, 64, 100]

The part inside the square brackets is a generator expression and the general form is as follows:

  <result expression> for <loop variable> in <iterable> if <filter expression>

Comparing that with the list comprehension example you see that the result expression was n*n, the loop variable was n, and the iterable was range(1,11). The filter expression was if not n*n % 2.

You can rewrite that as a conventional for loop, like this:

  result = []
  for n in range(1,11):
      if n*n % 2:
          result.append(n)

One important point to appreciate about generator expressions is that they do not generate the whole data set at once. Rather they generate (hence the name) the data items on demand, which can lead to a significant saving in memory resources when dealing with large data sets. You find out more about this later in the chapter when you look at a special type of function called a generator function.

Handling Exceptions

There are two approaches to detecting errors. The first involves explicitly checking each action as it’s performed; the other attempts the operations and relies on the system generating an error condition, or exception, if something goes wrong. Although the first approach is appropriate in some situations, in Python, it’s far more common to use the second. Python supports this technique with the try/except/else/finally construct. In its general form, it looks like this:

  try:
      A block of application code
  except <an error type> as <anExceptionObject>:
      A block of error handling code
  else:
      Another block of application code
  finally:
      A block of clean-up code

The except, else, and finally are all optional, although at least one of except or finally must exist if a try statement is used. There can be multiple except clauses, but only one else or finally. You can omit the as… part of an except statement line if the exception details are not required.

The try block is executed and, if an error is encountered, the exception class is tested. If an except statement exists for that type of error, the corresponding block is executed. (If multiple exception blocks specify the same exception type, only the first matching clause is executed.) If no matching except statement is found, the exception is propagated upwards until the top-level interpreter is reached and Python generates its usual traceback error report. Note that an empty except statement will catch any error type; however, this is usually a bad idea because it hides the occurrence of any unexpected errors.

The else block is executed if the try block succeeds without any errors. In practice the else is rarely used. Regardless of whether an error is caught or propagated, or whether the else clause is executed, the finally clause will always be executed, thus providing an opportunity to release any computing resources in a locked state. This is true even when the try/except clause is left via a break or return statement.

You can use a single except statement to process multiple exception types. You do this by listing the exception classes in a tuple (parentheses are required). The optional exception object contains details of where the exception occurred and provides a string conversion method so that a meaningful error message may be provided by printing the object.

It is possible to raise exceptions from your own code. It is also possible to use any of the existing exception types or to define your own by subclassing from the Exception class. You can also pass arguments to exceptions you raise, and you can access these in the exception object in the except clause using the args attribute of the error object.

Here is an example of raising a standard ValueError with a custom argument and then catching that error and printing the given argument.

  >>> try:
  ... raise ValueError('wrong value')
  ... except ValueError as error:
  ... print (error.args)
  ...
  ('wrong value',)

Note that you didn’t get a full traceback, only the print output from the except block. You can also re-raise the original exception after processing by simply calling raise with no arguments.

Managing Context

Python has the concept of a runtime context. This typically includes a temporary resource of some kind that your program wants to interact with. A typical example might be an open file or a concurrent thread of execution. To handle this Python uses the keyword with and a context manager protocol. This protocol enables you to define your own context manager classes, but you will mostly use the managers provided by Python.

You use a context manager by invoking the with statement:

  with open(filename, mode) as contextName:
      process file here

The context manager ensures the file is closed after use. This is fairly typical of a context manager’s role—to ensure that valuable resources are freed after use or that proper sharing precautions are taken on first use. Context managers often remove the need for a try/finally construct. The contextlib module provides support for building your own context managers.

You have now seen the different types of data that Python can process as well as the control structures you can use to do that processing. It is now time to find out how to get data into and out of your Python programs and that is the subject of the next section.

Interacting with Users

To send data to users via stdout, you use the print() function, which you’ve seen several times already. You learn how to control the output more precisely in this section. To read data from users, you use the input() function, which prompts the user for input and then returns a string of raw characters from stdin.

The print() function is more complex than it first appears in that it has several optional parameters. At its simplest level, you simply pass a string and print() displays it on stdout followed by an end-of-line (eol) character. The next level of complexity involves passing non-string data that print() then converts to a string (using the str() type function) before displaying the result. Stepping up a gear, you can pass multiple items at once to print(), and it will convert and display them in turn separated by a space.

The previous paragraph identified three fixed elements in print()’s behavior:

• It displays output on stdout.
• It terminates with an eol character.
• It separates items with a space.

In fact, none of these are really fixed, and print() enables you to modify any or all of them using optional parameters. You can change the output by specifying a file argument; the separator is defined by the sep argument, and the terminating character is defined by the end argument. The following line prints the infamous “hello world” message, specified as two strings, to a file using a hyphen separator and the string "END" as an end marker:

  with open("tmp.txt", "w") as tmp:
      print("Hello", "World", end="END", sep="-", file=tmp)

The content of the file should be: "Hello-WorldEND".

The string format() method really comes into its own when combined with print(). This combination is capable of presenting your data neatly and clearly separated. In addition using format() can often be more efficient that trying to print a long list of objects and string fragments. There are many examples of how to use format() in the Python documentation.

You can also communicate with the user using the input() function that reads values typed by the user in response to a given on-screen prompt. It is your responsibility to convert the returned characters to whatever data type is represented and handle any errors resulting from that conversion.

Here is an example that asks the user to enter a number. If the number is too high or too low it prints a warning. (This could form the core of a guessing game if you cared to experiment with it.)

  target = 66

  while True :
      value = input("Enter an integer between 1 and 100")
      try:
      value = int(value)
      break
  except ValueError:
      print("I said enter an integer!")

  if value > target:
      print (value, "is too high")
  elif int(value) < target:
      print("too low")
  else:
      print("Pefect")

Here the user is provided with a prompt to enter an integer in the appropriate range. The value read is then converted to an integer using int(). If the conversion fails, a ValueError exception is raised, and the error message is then displayed. If the conversion succeeds, you can break out of the while loop and proceed to test it against the target, confident that you have a valid integer.

Using Text Files

Text files are the workhorses of programming when it comes to saving data, and Python supports several functions for dealing with text files.

You saw the open() function used in previous sections and it takes a filename and a mode as arguments. The mode can be any of r, w, rw, and a for read, write, read-write, and append respectively. (Some other modes are less often used. There are also a few optional parameters that control how the data is interpreted, see the documentation for details.) The r mode requires the file to exist; the w and rw modes create a new empty file (or overwrite any existing file of the same name). The a mode opens an existing file or creates a new empty file if a file of the specified filename does not already exist. The file object returned is also a context manager and can be used in a with block as you saw in the context manager section. If a with block is not used, you should explicitly close the file using the close() method when you are finished with it, thus ensuring that any data sitting in memory buffers is sent to the physical file on disk. The with construct calls close() automatically, which is one of the advantages of using the context manager approach.

Once you have an open file object, you can use read(), readlines(), or readline() as required. read()reads the entire file contents as a single string complete with embedded newline characters. readlines() reads line by line into a list, and the newline characters are retained. readline() reads the next line in the file, again retaining the newline. The file object is an iterable, so you can use it directly in a for loop without the need for any of the read methods. The recommended pattern for reading the lines in a file is therefore:

  with open(filename, mode) as fileobject:
      for line in fileobject:
          # process line

You can write to a writable file object using the write() or writelines() methods, which are the equivalents to the similarly named read methods. Note that there is no writeline() method for writing a single line.

If you are using the rw mode, you might want to move to a specific position in the file to overwrite the existing data. You can find your current position in the file using the tell() method. You can go to a specific position (possibly one you recorded with tell() earlier) using the seek() method. seek() has several modes of calculating position; the default is simply the offset from the start of the file.

You now have all of the basic skills to write working Python programs. However, to tackle larger projects, which are the focus of this book, you will want to extend Python’s capabilities. The next section starts to explore how you can do just that.

Defining and Using Functions

Several types of functions are available in Python. This section looks at the standard variety first, followed by a generator function, and concludes with the slightly enigmatic lambda function.

You define functions in Python using the def keyword. The form looks like this:

  def functionName(parameter1, param2,...):
      function block

Python functions always return a value. You can specify an explicit return value using the return keyword; otherwise, Python returns None by default. (If you find unexpected None values appearing in your output, check that the function concerned has an explicit return statement in its body.) You can give default values to parameters by following the name with an equals sign and the value. You see an example in the odds() generator function in the next section.

You can most easily understand how a function definition is created and used by trying it out.

  def odds(start=1):
      ''' return all odd numbers from start upwards'''
      if int(start) % 2 == 0: start = int(start) + 1
      while True:
          yield start
          start += 2

  for n in odds():
       if n > 7: break
       else: print(n)

  lambda <param1, param2, ...,paramN> : <expression>

  >>> straight_line = lambda m,x,c: m*x+c
  >>> straight_line(2,4,-3)
  5

Defining and Using Classes and Objects

Python supports object-oriented programming using a traditional class-based approach. Python classes support multiple inheritance and operator overloading (but not method overloading), as well as the usual mechanisms of encapsulation and message passing. Python classes do not directly implement data hiding, although some naming conventions are used to provide a thin layer of privacy for attributes and to suggest when attributes should not be used directly by clients. Class methods and data are supported as well as the concepts of properties and slots. Classes have both a constructor (__new__()) and an initializer (__init__()), as well as a destructor mechanism (__del__()), although the latter is not always guaranteed to be called. Classes act as namespaces for the methods and data defined therein.

Objects are instances of classes. Instances can have their own attributes added after instantiation, although this is not normal practice.

A class is defined using the class keyword followed by the class name and a parenthesized list of super-classes. The class definition contains a set of class data and method definitions. A method definition has as its first parameter a reference to the calling instance, traditionally called self. A simple class definition looks like this:

  class MyClass(object):
      instance_count = 0
      def __init__(self, value):
          self.__value = value
          MyClass.instance_count += 1
          print("instance No {} created".format(MyClass.instance_count))
      def aMethod(self, aValue):
          self.__value *= aValue
      def __str__(self):
          return "A MyClass instance with value: " + str(self.__value)
      def __del__(self):
          MyClass.instance_count -= 1

The class name traditionally starts with an uppercase letter. In Python 3 the super class is always object unless specifically stated otherwise, so the use of object as the super class in the preceding example is actually redundant. The instance_count data item is a class attribute because it does not appear inside any of the class’ methods. The __init__() function is the initializer (constructors are rarely used in Python unless inheriting from a built-in class). It sets the instance variable self.__value, increments the previously defined class variable instance_count, and then prints a message. The double underscores before value indicate that it is effectively private data and should not be used directly. The__init__() method is called automatically by Python immediately after the object is constructed. The instance method aMethod() modifies the instance attribute created in the __init__() method. The __str__() method is a special method used to return a formatted string so that instances passed to the print function, for example, will be printed in a meaningful way. The destructor__del__() decrements the class variable instance_count when the object is destroyed.

You can create an instance of the class like this:

  myInstance = MyClass(42)

This creates an instance in memory and then calls MyClass.__init__() with the new instance as self and 42 as value.

You can call the aMethod() method using dot notation like this:

  myInstance.aMethod(66)

This is translated to the more explicit invocation,

  MyClass.aMethod(myInstance, 66)

and results in the desired behavior whereby the value of the __value attribute is adjusted.

You can see the __str__() method in action if you print the instance, like this:

  print(myInstance)

This should print the message:

  A MyClass instance with value: 2772

You could also print the instance_count value before and after creating/destroying an instance:

  print(MyClass.instance_count)
  inst = MyClass(44)
  print(MyClass.instance_count)
  del(inst)
  print(MyClass.instance_count)

This should show the count being incremented and then later decremented again. (There may be a slight delay before the destructor is called during garbage collection, but it should only be a few moments.)

The __init__(), __del__(), and __str__() methods are not the only special methods. Several of these exist, all signified by the use of double underscores (they are sometimes called dunder methods). Operator overloading is supported via a set of these special methods including: __add__(), __sub__(), __mul__(), __div__(), and so on. Other methods provide for the implementation of Python protocols such as iteration or context management. You can override these methods in your own classes. You should never define your own dunder methods; otherwise, future enhancements to Python could break your code.

You can override methods in subclasses, and the new definitions can invoke the inherited version of the method by using the super() function, like this:

  class SubClass(Parent):
      def __init__(self, aValue):
          super().__init__(aValue)

The call to super().__init__() translates to a call to the __init__() method of Parent. Using super() avoids problems, particularly with multiple inheritance, where a class could be inherited multiple times and you usually don’t want it to be initialized more than once.

Slots are a memory-saving device, and you invoke them by using the __slots__ special attribute and providing a list of the object attribute names. Often __slots__ are a premature optimization, and you should use them only if you have a specific, known need.

Properties are another feature available for data attributes. They enable you to make an attribute read only (or even write only) by forcing access to be via a set of methods even though the usual method syntax is not used. This is best seen by an example where you create a Circle class with a radius attribute and area() method. You want the radius value to always be positive, so you don’t want clients changing it directly in case they pass a negative value. You also want area to look like a read-only data attribute even though it is implemented as a method. You achieve both objectives by making radius and area properties.

Having seen how to extend Pythons capabilities using functions and classes, the next section shows you how to enclose these extensions in modules and packages for reusability.

Using and Creating Modules

You access modules using the import keyword, which has many variations in Python. The most common forms are shown here:

  import aModule
  import aModule as anAlias
  import firstModule, secondModule, thirdModule...
  from aModule import anObject
  from aModule import anObject as anAlias
  from aModule import firstObject,secondObject, thirdObject...
  from aModule import *

The last form imports all the visible names from aModule into the current namespace. (You learn how to control visibility shortly.) This carries a significant risk of creating naming conflicts with built-in names or names you have defined, or will define, locally. It is therefore recommended that you use only the last import form for testing modules at the Python prompt. The small amount of extra typing involved in using the other forms is a small price to pay compared to the confusion that can result from a name clash. The other from... forms are much safer because you only import specified names and, if necessary, rename them with an alias. This makes clashes with other local names much less likely.

Once you import a module using any of the first three forms, you can access its contents by prefixing the required name with the module name (or alias) using dot notation. You have already seen examples of that in the previous sections; for example, sys.path is an attribute of the sys module.

Having said that modules are simply source files, in practice, you should observe some do’s and don’ts when creating modules. You should avoid top-level code that will be run when the module is imported, except, possibly, for some initialization of variables that may depend on the local environment. This means that the code you want to reuse should be packaged as a function or a class. It’s also common to provide a test function that exercises all the functions and classes in the module. Module names are also traditionally lowercase only.

You can control visibility of the objects within your module in one of two ways. The first is similar to the privacy mechanism used in classes in that you can prefix a name with an underscore. Such names will not be exported when a from x import * style statement is used. The other way to control visibility is to list only the names that you want exported in a top-level variable called __all__. This ensures that only the names you specifically want to be exported will be and is recommended over the underscore method if visibility is important to you.

You put most of this into practice in the next Try It Out, but first, you need to look at packages and how they differ from modules.

Using and Creating Packages

You discovered at the start of this section that a Python package is just a folder with a file called __init__.py. All other Python files within that folder are the modules of the package. Python considers packages as just another type of module, which means that Python packages can contain other packages within them to an arbitrary depth—provided each subpackage also has its own __init__.py file, it is a valid package.

The __init__.py file itself is not particularly special; it is just another Python file, and it will be loaded when you import the package. This means that the file can be empty, in which case importing the package simply gives access to the included modules, or it can have Python code within it like any other module. In particular it can define an __all__ list, as described earlier, to control visibility, effectively enabling a package to have private implementation files that are not exported when a client imports the package.

A common requirement when you create a package is to have a set of functions or data shared between all the included modules. You can achieve this by putting the shared code into a module file called, say, common.py at the top level of the package and having all of the other modules import common. It will be visible to them as part of the package, but if it is not explicitly included in the __all__ list, external consumers of the packages will not see it.

When it comes to using a package, you treat it much like any other module. You have all the usual styles of import available, but the naming scheme is extended by using dot notation to specify which submodules you need from the package hierarchy. When you import a submodule using the dot notation, you effectively define two new names: the package and the module. Consider, for example, this statement:

  import os.path

This code imports the path submodule of the os package. But it also makes the whole of the os module visible as well. You can proceed to access os functions without having a separate import statement for os itself. One implication of this is that Python automatically executes all of the __init__.py files that it finds in the imported hierarchy. So in the case of os.path, it executes os.__init__.py and then path.__init__.py.

On the other hand, if you use an alias after the import, like this,

  import os.path as pth

only the os.path module is exposed. If you want to use the os module functions, you will need an explicit import. Although only path is exposed, as the name pth, both os and path __init__.py files will still be run.

The Python standard library contains several packages including the os package just mentioned. Others of note include the UI frameworks tkinter and curses, the email package, and the web-focused urllib, http, and html packages. You use several of these later in the book.

You have now covered all of the theory about modules and packages. In the next section, you put this information to work by creating some modules and a package.