14.5. Executing Other (Non-Python) Programs
We can also execute non-Python programs from within Python. These include binary executables, other shell scripts, etc. All that is required is a valid execution environment, i.e., permissions for file access and execution must be granted, shell scripts must be able to access their interpreter (Perl, bash, etc.), binaries must be accessible (and be of the local machine's architecture).
Finally, the programmer must bear in mind whether our Python script is required to communicate with the other program that is to be executed. Some programs require input, others return output as well as an error code upon completion (or both). Depending on the circumstances, Python provides a variety of ways to execute non-Python programs. All of the functions discussed in this section can be found in the os module. We provide a summary for you here in Table 14.7 (where appropriate, we annotate those which are available only for certain platforms) and introduce them to you for the remainder of this section.
As we get closer to the operating system layer of software, you will notice that the consistency of executing programs, even Python scripts, across platforms starts to get a little dicey. We mentioned above that the functions
Unix only, 
Windows only)
described in this section are in the os module. Truth is, there are multiple os modules. For example, the one for Unix is the posix module. The one for Windows is nt (regardless of which version of Windows you are running; DOS users get the dos module), and the one for the Macintosh is the mac module. Do not worry, Python will load the correct module when you call “import os”. You should never need to import a specific operating system module directly.
14.5.1. os.system()
The first function on our list is system(), a rather simplistic function which takes a system command as a string name and executes it. Python execution is suspended while the command is being executed. When execution has completed, the exit status will be given as the return value from system() and Python execution resumes. This function is available for Unix and Windows only.
system() preserves the current standard files, including standard output, meaning that executing any program or command displaying output will be passed on to standard output. Be cautious here because certain applications such as common gateway interface (CGI) programs will cause web browser errors if output other than valid hypertext markup language (HTML) strings are sent back to the client via standard output. system() is generally used with commands producing no output, some of which include programs to compress or convert files, mount disks to the system, or any other command to perform a specific task that indicates success or failure via its exit status rather than communicating via input and/or output. The convention adopted is an exit status of 0 indicating success and non-zero for some sort of failure.
For the sake of providing an example, we will execute two commands which do have program output from the interactive interpreter so that you can observe how system() works.
>>> import os
>>> result = os.system('cat /etc/motd')
Have a lot of fun…
>>> result
0
>>> result = os.system('uname -a')
Linux solo 2.2.13 #1 Mon Nov 8 15:08:22 CET 1999 i586 unknown
>>> result
0
You will notice the output of both commands as well as the exit status of their execution which we saved in the result variable. Here is an example executing a DOS command:
>>> import os
>>> result = os.system('dir')
Volume in drive C has no label
Volume Serial Number is 43D1-6C8A
Directory of C:WINDOWSTEMP
. <DIR> 01-08-98 8:39a .
.. <DIR> 01-08-98 8:39a ..
0 file(s) 0 bytes
2 dir(s) 572,588,032 bytes free
>>> result
0
14.5.2. os.popen()
The popen() function is a combination of a file object and the system() function. It works in the same way as system() does, but in addition, it has the ability to establish a one-way connection to that program and then to access it like a file. If the program requires input, then you would call popen() with a mode of 'w' to “write” to that command. The data that you send to the program will then be received through its standard input. Likewise, a mode of 'r' will allow you to spawn a command, then as it writes to standard output, you can read that through your file-like handle using the familiar read*() methods of file object. And just like for files, you will be a good citizen and close() the connection when you are finished.
In one of the system() examples we used above, we called the Unix uname program to give us some information about the machine and operating system we are using. That command produced a line of output that went directly to the screen. If we wanted to read that string into a variable and perform internal manipulation or store that string to a log file, we could, using popen(). In fact, the code would look like the following:
>>> import os >>> f = os.popen('uname -a') >>> data = f.readline() >>> f.close() >>> print data, Linux solo 2.2.13 #1 Mon Nov 8 15:08:22 CET 1999 i586 unknown
As you can see, popen() returns a file-like object; also notice that readline(), as always, preserves the newline character found at the end of a line of input text.
14.5.3. os.fork(), os.exec*(), os.wait*()
Without a detailed introduction to operating systems theory, we present a “light” introduction to processes in this section. fork() takes your single executing flow of control known as a “process” and creates a “fork-in-the-road,” if you will. The interesting thing is that your system takes both forks—meaning that you will have two consecutive and parallel running programs (running the same code no less because both processes resume at the next line of code immediately succeeding the fork() call).
The original process which called fork() is called the “parent” process, and the new process created as a result of the call is known as the “child process.” When the child process returns, its return value is always zero; when the parent process returns, its return value is always the process identifier (a.k.a. process ID, or “PID” for short) of the child process (so the parent can keep tabs on all its children). The PIDs are the only way to tell them apart, too!
We mentioned that both processes will resume immediately after the call to fork(). Because the code is the same, we are looking at identical execution if no other action is taken at this time. This is usually not the intention. The main purpose for creating another process is to run another program, so we need to take divergent action as soon as parent and child return. As we stated above, the PIDs differ, so this is how we tell them apart:
The following snippet of code will look familiar to those who have experience managing processes. However, if you are new, it may be difficult to see how it works at first, but once you get it, you get it.
ret = os.fork() # spawn 2 processes, both return if ret == 0: # child returns with PID of 0 child_suite # child code else: # parent returns with child's PID parent_suite # parent code
The call to fork() is made in the first line of code. A new process called a child process is created. The original process is called the parent process. The child process has its own copy of the virtual memory address space and contains an exact replica of the parent's address space—yes, both processes are nearly identical. Recall that fork() returns twice, meaning that both the parent and the child return. You might ask, how can you tell them apart if they both return? When the parent returns, it comes back with the PID of the child process. When the child returns, it has a return value of 0. This is how we can differentiate both processes.
Using an if-else statement, we can direct code for the child to execute (i.e., the if clause) as well as the parent (the else clause). The code for the child is where we can make a call to any of the exec*() functions to run a completely different program or some function in the same program (as long as both child and process take divergent paths of execution). The general convention is to let the children do all the dirty work while the parent either waits patiently for the child to complete its task or continues execution and checks later to see if the child finished properly.
All of the exec*() functions load a file or command and execute it with an argument list (either individually given or as part of an argument list). If applicable, an environment variable dictionary can be provided for the command. These variables are generally made available to programs to provide a more accurate description of the user's current execution environment. Some of the more well-known variables include the user name, search path, current shell, terminal type, localized language, machine type, operating system name, etc.
All versions of exec*() will replace the Python interpreter running in the current (child) process with the given file as the program to execute now. Unlike system(), there is no return to Python (since Python was replaced). An exception will be raised if exec*() fails because the program cannot execute for some reason.
The following code starts up a cute little game called “xbill” in the child process while the parent continues running the Python interpreter. Because the child process never returns, we do not have to worry about any code for the child after calling exec*(). Note that the command is also a required first argument of the argument list.
ret = os.fork() if ret == 0: # child code execvp('xbill', ['xbill']) else: # parent code os.wait()
In this code, you also find a call to wait(). When children processes have completed, they need their parents to clean up after them. This task, known as “reaping a child,” can be accomplished with the wait*() functions. Immediately following a fork(), a parent can wait for the child to complete and do the clean-up then and there. A parent can also continue processing and reap the child later, also using one of the wait*() functions.
Regardless of which method a parent chooses, it must be performed. When a child has finished execution but has not been reaped yet, it enters a limbo state and becomes known as a “zombie” process. It is a good idea to minimize the number of zombie processes in your system because children in this state retain all the system resources allocated in their lifetimes, which do not get freed or released until they have been reaped by the parent.
A call to wait() suspends execution (i.e., waits) until a child process (any child process) has completed, terminating either normally or via a signal. wait() will then reap the child, releasing any resources. If the child has already completed, then wait() just performs the reaping procedure. waitpid() performs the same functionality as wait() with the additional arguments PID to specify the process identifier to a specific child process to wait for, and options, which is normally zero or a set of optional flags logically OR'd together. Refer to the Python, your operating system documentation, or any general operating system textbook such as Silbershatz and Galvin, Tanenbaum and Woodhull, or Stallings for more details.
14.5.4. os.spawn*()
The spawn*() family of functions work only in the world of Windows. Depending on the mode chosen, spawn*() functions can duplicate the functionality of fork(), exec*(), system(), wait*(), and/or a combination of those popular Unix functions. Both spawn() and spawnve() were introduced in Python 1.5.2. For more information, go to the os module documentation in the Python Library Reference manual.
14.5.5. Other Functions
Table 14.8 lists some of the functions (and their modules) which can perform some of the tasks described.
| File Object Attribute | Description (available only on Unix
or Windows
platforms) |
|---|---|
| popen2.popen2() | execute a file and open file read and write access from (stdout) and to (stdin) the newly-created running program  |
| popen2.popen3() | execute a file and open file read and write access from (stdout and stderr) and (stdin) to the newly-created running program  |
| commands.getoutput() | executes a file in a subprocess, return all output as a string  |