Besides starting processes, spawn
can be used to begin interactions with files and pipelines. In this chapter, I will go into detail on the spawn
command. I will also cover ptys—what they are, how to control them, andtheir features and pitfalls.
The spawn
command follows the “usual” rules in finding programs to invoke. Both relative and absolute filenames are acceptable. If a filename is specified with no directory at all, the value of the environment variable PATH
is treated as a list of directories and each directory is searched until the given file is found. This searching is performed by the operating system and behaves identically to the way that programs are found from shells such as the Bourne shell and the C shell.
spawn /bin/passwd ;# absolute spawn passwd ;# relative
In some cases, naming programs absolutely is a good idea. In other cases, relative names make more sense. If you do use relative names, it is a good idea to set PATH
explicitly. For example:
set env(PATH) "/bin:/usr/bin"
Setting the path avoids the possibility of picking up local versions of utilities that users might otherwise have on their paths. Users at other sites may need to change the path, but the single definition at the top of a script makes the path easy to change.
While resetting the path is easy enough, there are circumstances when it makes more sense to refer to programs absolutely. For example, your system may have several versions of a utility (e.g., BSD, SV, POSIX, GNU). Naming a utility absolutely may be safer than using the path.
Rather than embedding a name literally, use variables so that all the references in a script can be changed easily if necessary. For example, you might localize a particular telnet
as:
set telnet /usr/ucb/telnet
Later in the script, telnet
would then be started this way:
spawn $telnet
Sets of programs that live in different subdirectories under a common directory could be localized with separate variables such as in this example:
set root "/usr/kidney" set bindir "$root/bin" set testdir "$root/test/bin" set demoprog "$bindir/nephron-demo"
Scripts that use these initializations would have spawn
commands such as:
spawn $bindir/control spawn $testdir/simulate spawn $demoprog
If you have a number of scripts using these definitions, they can be stored in a common file that is sourced by the scripts at start-up.
Like Tcl’s exec
command and the C-shell, spawn
also supports the tilde notation. A tilde preceeding the first component of a filename is understood to mean the home directory corresponding to the user named by that component. For example, the following command spawns the pike
program from the files of a user named shaney
:
spawn ~shaney/bin/pike
The previous chapter demonstrated how to open files or devices and send commands to them with puts
. This technique calls upon another program to do the handling of the device. At first glance, you might consider this inefficient. Why run two programs when one will do? Consider that reusing another program allows you to isolate all the device-specific problems in one place. If the tip
program already knows about serial devices and how, for example, to choose among baud rates, why burden other programs with the same knowledge?
Ideally, if you just had one program that knew everything about, say, your serial devices, you would not need any others. Or perhaps, other programs could call upon your serial device program when they needed to access a serial device. This is analogous to the concept of device drivers. Unfortunately, real device drivers are not high-level enough to isolate out the device dependencies for our purposes.
It is all too common to have numerous programs (kermit
, procomm
, tip
, cu
, etc.) that all do the same thing. The reason for having multiple programs to do the same thing is that one has features that another does not have and vice versa. So you keep them all around. Not only is this a problem when you require both features at the same time, but it is a problem when you upgrade or modify your serial device. For example, if you change some phone numbers in your kermit
scripts, you also have to change them in any procomm
scripts. Some of these programs use a database but none use the same one, so you have to change multiple databases, too.
Expect tries to avoid this quagmire by reusing programs and their knowledge. Expect does not have to be told any device-specific information—it relies entirely upon the device-specific program. If you want to communicate with a serial device, Expect can spawn tip
(or kermit
, etc.). To communicate with sockets, Expect can spawn telnet
. And so on. This works because each of these devices is controlled by interactive programs which in turn can be controlled by Expect.
If you have a device unique to your machine with its own interactive interface, Expect can use it. By unburdening Expect from a lot of device specific information, it is simpler to use, plus you get to use the interface with which you are already familiar.
In UNIX, devices are files. Or at least, it is convenient to think that they are. Devices appear in the file system, and file-like operations can be performed on them. Expect uses these beliefs to support operations like expect
and send
.
In fact, these can be applied to files as well. In the previous chapter, I described how you can make any file look like a spawned process—for example, by spawning a process to read it with cat
. A more direct way is possible.
Tcl’s open
command is capable of opening files. The open
command returns a file identifier which can be used as an argument to gets
, puts
, etc. Using the -open
argument of spawn
, the file id can be turned into a spawn id.
set file [open "/etc/passwd" r] spawn -open $file expect . . .
This example opens the /etc/passwd
file and searches through it until the given pattern appears. After the spawn
, the file behaves exactly like a process. While there is no real process, for consistency, the file must be waited for after being closed. The value returned by wait
always indicates a normal exit and can be ignored. This allows you to write code in which it does not matter whether a real process was spawned or not.
The first two commands in the fragment above can be condensed to one:
spawn -open [open "/etc/passwd" r]
Once the file has been passed to spawn
, it should not be accessed by gets
, puts
, or any other Tcl function. Expect and Tcl do not share buffers internally, so joint access should generally be avoided. The file will be closed automatically when the spawn id is closed. If the file must be left open after the spawn id is closed, use -leaveopen
instead of -open
.
The spawn id in this example cannot be written to (with send
) because open
only opened the file for reading. To open for writing, the w
flag should be used. "w+
" allows reading and writing. Several variations on this exist. Read Tcl’s open
documentation for the complete details.
While the spawn
command can convert a Tcl file to a spawn id, it is also possible to do the opposite. On page 303, I will describe how to convert a spawn id to a Tcl file identifier.
Normally, Expect calls upon tty-aware programs such as tip
or cu
to interact with ttys. However, using the technique described in the previous section, it is possible to open ttys or other devices directly. For example, if you have a tty port named /dev/ttya
, you can open it as follows:
spawn -open [open /dev/ttya w+]
Unfortunately, that is not the whole story. Tty devices must be initialized. This is something that a program such as tip
would do for you. Without such a program, you must do it yourself. In the interest of generality, Expect performs no special handling depending upon the type of file it has been handed. That is up to you.
You can initialize a tty using stty
with the appropriate arguments. The only hard part is figuring out the arguments. There is no standard, and the parameters can vary depending on your task.
For most tasks, you want to disable special character processing. You also want to disable echo. These are accomplished as follows:
stty raw -echo < /dev/ttya
Flags such as -echo
apply to the tty input. Unlike a traditional tty to which you log in, this tty is being used to “go out”. So the tty’s input (in the sense that stty
cares) is provided by the remote side. For example, if you use the tty to connect to another serial port, then the output of that serial port is the input of this one.
If the serial port generates parity, you may need to handle that either by disabling it or telling stty
to ignore it. Another stty
command suffices:
stty istrip < /dev/ttya
The istrip
flag strips off the parity bit. If you need an eight-bit connection, you can modify the terminal modes of both the tty and the remote serial port after you are logged in.
Note that raw
is a conglomeration of other stty
modes. This includes parity on some systems, so you may have to issue the stty
commands separately as I have shown here.
The dozens of flags supported by stty
vary from system to system. If the stty
man page is unenlightening, examine your tty modes while your are using tip
(or whatever program you are trying to simulate). This will tell you what the correct configuration should be.
On some systems, bugs in the operating system prevent correct operation of "spawn -open [open . . .]
" on some special files. For example, on SunOS, Expect cannot detect an eof from a fifo. Yet another operating system bug prevents AIX 3.2 from detecting input from physical tty devices.[46]
Fortunately, OS bugs like these are infrequent but Expect does a number of unusual things and you must gird yourself to work around such bugs. In the next section, I will describe a workaround for the fifo problem. A similar workaround can be applied to the other problem.
The spawn
command does not provide any facility for redirection or pipelines because the need for it is almost non-existent. Automating interactive programs virtually always require much more sophisticated handling of input and output, such as looking at the output before deciding on the next input.
Rather than burden the spawn
command with features that are almost never used, it is easier to call upon existing programs in those rare occasions. There are a variety of programs that support redirection including Tcl’s open
command. Tcl’s open
command is also capable of building pipelines.
In the previous section, I noted that fifos cannot be handled with spawn
on some operating systems. The following command interposes a cat
process. The "|
" indicates that the following file should be started as a process rather than a file that is simply read or written.
spawn -open [open "|cat -u $fifo" r]
While seemingly redundant, the fifo eof is now handled by cat
which converts this to an eof that is detectable by Expect.[47] The same solution works with other devices that are not supported by select
or poll
.[48]
Additional processes can be strung together in the first argument to open
by separating them with "|
" symbols. Bidirectional processes and process pipelines can be generated. They require the w+
flag to indicate that they will be both read and written.
All of the files, processes, and pipelines opened by open
are opened without a terminal interface. For many programs, this is a problem. They expect to be dealing with a terminal. For example, they may want to change the terminal modes. This will fail if there is no terminal involved.
In contrast, each spawned process has a terminal interface associated with it. To programs, the interface “feels” like a real tty is behind it. The program can tell the tty to echo or not echo, for example. But the tty does not physically exist. Rather, it is simulated, and for this reason is known as a pseudoterminal or pty (pronounced “pity”) for short. With a pty, interactive programs work properly.
When spawn
is called with -open
, no pty is provided. In the rare cases that a pty is needed along with a process pipeline or redirection, /bin/sh
can be spawned before invoking whatever is needed. For example, in Chapter 7 (p. 174), I described how spawn
normally combines the standard output and standard error. In contrast, the following command separates the standard error of a process so that it is written to a file.
spawn /bin/sh -c "exec 2> error.out; exec prog"
It works as follows: /bin/sh
is spawned. The -c
flag tells /bin/sh
to execute the following argument as a command. The argument is composed of two commands that will be executed sequentially.
The first command is "exec 2> error.out
“. This command directs the shell to associate file descriptor 2 (i.e., the standard error) with the file error.out
.
The next command is "exec prog
“. This command runs prog
. Its input and output are still associated with the input and output of the shell (from /bin/sh
) which in turn are associated with the spawn id. But the standard error remains tied to the file error.out
. The exec
in front of prog
tells the shell that prog
can take over the process resources of the shell. In effect, the shell exits leaving prog
in its place.
While initially this may look rather confusing and complex, the result effectively leaves only the intended process running. The shell goes away after setting up the indirection. More complex redirection and pipelines can be constructed but they usually share the same underlying ideas.
Expect normally works with programs that read either from the standard input or /dev/tty
. Some programs do not read their input in this way. A good example is the xterm
program. xterm
is an X Window System client that provides a shell in a terminal emulator. In this section, I will describe three different ways to control xterm
.
The xterm
program reads user input from a network socket. The standard input and /dev/tty
are both ignored. Hence, spawning xterm
in the usual way is fruitless.
spawn xterm ;# WRONG
Interacting in this way—with no special knowledge of xterm
—requires a program that can drive X applications the way Expect drives character-oriented programs. Such programs exist. However, discussion of them is beyond the scope of this book.
Instead of attempting to control an xterm
, it often suffices to have an xterm
execute an Expect script. For example, suppose you want to be able to pop up a window that automatically runs the chess
script defined in Chapter 10 (p. 234). The following command would suffice:
xterm -e chess.exp
The xterm
continues to take input in the usual way. Therefore it is even possible to have scripts that accept user input. For example, the auto-ftp
script defined in Chapter 3 (p. 83) could be started up with the following command. Once running, it is controllable from the keyboard just the way an xterm
normally is.
xterm -e aftp.exp
Both of these examples give up the possibility of controlling the xterm
from another script. It is possible to do this by having xterm
run an open-ended script such as kibitz
. I will present an example of this in Chapter 16 (p. 355).
A third way to control xterm
is to spawn it so that the script replaces the process that xterm
normally spawns internally. When xterm
starts, it no longer starts a new process but talks to Expect. Expect reads what the user types and tells the xterm
what to display.
This is a little more complicated, but it allows a script the ability to start multiple xterm
s and interact with them. Rather than the xterm
driving Expect, Expect drives the xterm
.
In order to talk to an xterm
in this way, Expect must obtain a pty and pass it to the xterm
. Expect will communicate using one end of it (the master), and the xterm
will communicate using the other end (the slave).
When a process is spawned, Expect allocates the pty, creates the process, and arranges for the process to use the pty for its standard input among other things. However, as I mentioned earlier, xterm
does not read its standard input. xterm
normally spawns its own process. It is possible to start xterm
so that it does not spawn a process but instead interacts with an existing one.
To do this, xterm
requires that the pty name and file descriptor be passed to it when it is invoked. Expect normally allocates the pty as the process is created, but this is too late for xterm
. xterm
wants the pty name on the command line. The -pty
flag causes the spawn
command to generate a pty with no new process.
spawn -pty
During the spawn
command, the name of the slave end of the pty is written to the variable spawn_out(slave,name)
. This occurs whenever a process is spawned, whether or not the -pty
flag is present. (In Chapter 14 (p. 315), I will show another use for this variable.)
The pty must be initialized to raw mode and have echoing disabled.
stty raw -echo < $spawn_out(slave,name)
In X11R5 and earlier versions, the flag to run xterm
in slave mode is rather peculiar. The flag is -S
. It is followed by two characters denoting the suffix of the pty name and an integer representing the file descriptor. For example, if the slave is named /dev/ttyp0
and the file descriptor is 6, xterm
is started with the flag "-Sp06
“.
The two-character format does not support all pty names and because of this, many vendors have modified xterm
. For example, some versions of xterm
use a 0 to pad suffixes that would otherwise be one character. The following code generates the two-character suffix, padding if necessary:
regexp ".*(.)(.)" $spawn_out(slave,name) dummy c1 c2 if {[string compare $c1 "/"] == 0} { set c1 "0" }
There is no backward-compatible solution for ptys that use more than three characters for identification. However, as of X11R5, xterm
does not actually use the information. So the code above suffices (unless your vendor has made radical changes).[49]
xterm
also requires the open file descriptor corresponding to the slave. This information is written to the variable spawn_out(slave,fd)
by the spawn
command.
Now the xterm
can be started. It is not necessary to use spawn
since the pty has already been allocated. The exec
command is appropriate. An ampersand forces it to run in the background so the script can go on to do other things.
exec xterm -S$c1$c2$spawn_out(slave,fd) &
Like spawn
, the exec
command returns the process id of the xterm
.
Once the xterm
is running, Expect should close its copy of the slave file descriptor. This is done by invoking the close
command with the -slave
argument.
close -slave
When xterm
starts this way, it immediately sends back an X window id on a line by itself. Extensions such as TkSteal can use the X window id to provide reparenting, allowing an xterm
to appear in a Tk widget hierarchy. If you do not want the X window id, just discard it.
expect " " ;# match and discard X window id
At this point, the xterm
can now be controlled. The send
command will print strings on the xterm
display. The expect
command will read input from the user (including insertions made using the mouse).
For example, the following code spawns a shell and lets the user interact in the xterm
until X
is pressed. Then the user is prompted for a return, after which the xterm
is killed and the script exits. (The "interact -u
" ties the xterm
and the shell together—this will be explained further in Chapter 16 (p. 350).)
# assume xterm is initialized, spawn id is in $xterm, # and xterm pid is in $xterm_pid spawn $env(SHELL) interact -u $xterm "X" { send −i $xterm "Press return to go away: " set timeout −1 expect −i $xterm " " { send −i $xterm "Thanks! " exec kill $xterm_pid exit } }
A real example that is more sophisticated than this one will be shown in Chapter 16 (p. 361).
All of the examples so far have assumed that spawn
always succeeds. The bad news is that spawn
does not always succeed. The good news is that it only fails in peculiar environments or in peculiar situations. In this section, I will describe the meaning of “peculiar” and how to check whether spawn
succeeded or not.
The spawn
command normally returns the process id of the newly spawned process.[50] This is generally of little value since spawned processes are more easily manipulable by their spawn ids. However, it is occasionally useful to be able to kill a process using its process id rather than going through some long interaction.
set pid [spawn program] . . . # some time later exec kill $pid
Once killed, the process connection should be recycled by calling close
and wait
.
Running out of various system resources can cause spawn
to fail. For example, spawn
allocates dynamic memory as well as a logical terminal interface. Failures like this can be caught using Tcl’s catch
command:
if [catch "spawn program" reason] { send_user "failed to spawn program: $reason " exit 1 }
Even if spawn
does not return an error, that is not a guarantee that it was entirely successful. To understand why, it is necessary to explain a little of how spawn
is implemented.
The spawn
command follows the traditional UNIX paradigm for running a new program. First, Expect forks. Forking is the UNIX way of generating a new process. Initially, the new process is still running Expect code. This allows Expect to prepare the environment appropriately for the new program. The last step is for Expect (in the new process) to overlay itself with the new program. At this point, the original Expect process is still running, and the new process is running the requested program.
This last step of loading the program can fail if, for example, the program does not exist. If it does not exist, the new process must communicate this back to the Expect process. Ironically, the failure of the program to be found can be communicated but not its success. The reason is that the very act of successfully running the program removes any functionality of the earlier program (i.e., Expect). Thus, the new program has no idea how to signal success or even that it should.
Because of this, the original Expect process cannot wait around for a possible failure. The spawn
command returns immediately. If the process does fail however, the new process sends back an error report in such a way that the Expect process hears it at the next convenient moment—the first expect
command.
Here is an example of interactively running Expect and attempting to spawn a non-existent program:
%expect
expect1.1>spawn noprog
spawn noprog 18961 expect1.2>expect -re .+
noprog: No such file or directory expect1.3>puts "expect_out = <$expect_out(buffer)> "
expect_out = <noprog: No such file or directory>
The error message is returned exactly the way any other output from the spawned process is—via expect_out
. Differentiating between an error from the shell and real program output from the process may be difficult, if not impossible. The recognition problem is identical to what a real human faces when using interactively starting programs from the shell. How one differentiates between an error message and real program output is left to the user.
The format of the error message is as shown above. It is the program name, followed by a colon and space, followed by your particular system’s standard error message. Other messages are possible in other scenarios, such as if the file exists but is not executable.
Checking the return value of spawn
(as shown above with catch
) is a good idea if you want your code to be bulletproof. These kinds of errors are often due to transient conditions that may go away if the operation is retried, such as a lack of memory.
On the other hand, checking spawn
’s success via the first expect
is less valuable. For example, if a standard utility such as /bin/sh
is being spawned, there is little point in checking if it succeeded. If it did not, the computer has such severe problems that few programs will be able to continue to run.
The primary circumstance in which to check the first expect
after a spawn
is when the program is unknown at the time the script is written. For example, if a user can type in arbitrary command names dynamically, these names should be checked. Note that using "file executable
" is a reasonable test but it is not guaranteed since there is a window between the time the file can be tested and the time it is executed, during which the file can change.
In the previous example, all of the commands were entered interactively. When this is done, the return values of all commands are automatically printed by Expect. In the case of the spawn
command, the return value was the process id. In that example, the process id was 18961. The command also echoed itself as a side effect. This is not the return value. If a spawn
command appears in a script, the process id will no longer be printed to the standard output, but the command itself still echoes.
This echoing is intended as a convenience for simple scripts, much as the echoing performed by the expect
command itself is. Both of these can be disabled with the log_user
command. However, the log_user
command disables all of the spawned program’s output. To disable just the echoing produced by the spawn
command, use the -noecho
flag. This flag affects nothing else. Here is the previous example repeated using that flag.
%expect
expect1.1>spawn -noecho noprog
18961 expect1.2>expect -re .+
noprog: No such file or directory expect1.3>puts "expect_out = <$expect_out(buffer)> "
expect_out = <noprog: No such file or directory>
Here is the same example, but using "log_user 0
" instead of -noecho
. Notice that both spawn
and expect
no longer echo anything.
%expect
expect1.1>log_user 0
expect1.2>spawn noprog
18961 expect1.3>expect -re .+
expect1.4>puts "expect_out = <$expect_out(buffer)> "
expect_out = <noprog: No such file or directory>
In all cases, spawn
still produces a return value. This and all other return values disappear if run from a script.
Most non-interactive programs behave differently depending on whether their output goes to the terminal or is redirected. In particular, output to a terminal normally appears as soon as a full line is produced. In contrast, output that is redirected to a file or a process is buffered by much larger amounts in the name of efficiency. This difference in buffering is automatically chosen by the UNIX stdio system.
Unfortunately, this means that some simple UNIX commands do not work as nicely as you might expect. For example, suppose a slow source is sending output to a fifo called /tmp/fifo
and you want to read it using od
and then pipe it into a pager such as more
. The obvious shell command to do this is:
od -c /tmp/fifo | more
Alas, the stdio system compiled into od
sees that its output is a pipe so the output is automatically buffered. Even if od
receives a complete line, od
does not send anything down the pipe until the buffer has been filled.
There is no way to fix od
short of modifying and recompiling it. However, by using Expect, it is possible to make od
think that its output is destined for a terminal. Since Expect connects processes to a pty, this is sufficient to satisfy the stdio system, and it changes to line-buffered I/O.
A script to do this is simple. All it has to do is spawn the process and wait for it to finish. Here is a script which does this, called unbuffer
:
#/usr/local/bin/expect — # Name: unbuffer # Description: unbuffer stdout of a program eval spawn -noecho $argv set timeout −1 expect
The original command can now be rewritten to use unbuffer
:
unbuffer od -c /tmp/fifo | more
Most other non-interactive UNIX utilities share the problem exhibited here by od
. Dealing with the stdio system is one of the few times where it makes sense to run Expect on non-interactive processes.
Historically, the console was a dedicated terminal to which critical messages were sent concerning the status of the computer. The idea was that a person would be watching at all times and could take immediate action if necessary. With modern workstations, there is no physical console with a dedicated operator. Instead, the console is simulated with a dedicated window. For example, in the X window system, the command "xterm -C
" starts an xterm
window and tells the operating system to send all console messages to it.
Expect can do the same thing with the spawn
command. The -console
flag redirects all console messages so that they appear to be generated from a spawned process. It is sufficient to spawn
any process. Even cat
will do.
spawn -console cat
A simple use for this flag is to watch for errors from device drivers. For example, when performing backups, errors writing to the backup media may be sent to the console rather than the backup program. This is a consequence of the way certain drivers are written and is surprisingly common.
By spawning the backup program using the -console
flag, it is possible to catch problems with the backup that might not otherwise be reported. In Chapter 17 (p. 380), I will describe how to make an Expect script actively look for a skilled user to fix any problems encountered, and initiate a session for the user connected to the spawned process automatically.
The -console
feature can only be used by one program at a time. It is also a relatively recent addition to UNIX. Therefore, it is not yet supported by all systems. The -console
flag is ignored on systems that do not support the ability to redirect console output.
Pty modes can have a big effect on scripts. For example, if a script is written to look for echoing, it will misbehave if echoing is turned off. Suppose a script is driving a shell that prompts with a bare "%
“. If the script sends the command "whoami
“, the shell might return "whoami
don
%
“. In this case, the response could be matched with:
expect -re " (.*) % "
If the shell did not echo its input, the shell would return "don
%
“. But the expect
command just shown fails to match this output.
For this reason, Expect forces “sane” pty modes by default. In fact, the sane
flag is known to stty
, the program which configures ttys and ptys. The particulars of sane
differ from system to system; however, sanity typically implies characteristics as echoing, and recognition of erase and kill characters. Expect invokes stty
to set the pty, so you can be assured that Expect’s version of sanity is just what your local stty
thinks. If for some reason you believe stty
’s understanding of sane
is flawed and you are not in the position to change it (i.e., you do not have the source), you can redefine it when installing Expect on your system. This is covered in the installation procedure.
Unfortunately, one program’s sanity is another program’s gibberish. Some programs have special demands. As an example, it is possible to interact with a shell from inside of the Emacs editor (this has nothing to do with Expect so far). The shell session appears as a normal file (or “buffer” in Emacs-speak) except that when you press return, the current line is sent as a command to the shell and the results are appended to the end of the buffer. This has many benefits. For example, with an Emacs shell session, you can use Emacs commands to directly edit the input and output.
To make the Emacs shell-session work similarly to a session outside Emacs, Emacs changes the pty modes. For example, echoing is disabled so that you can edit the command line before passing it to the shell. Also, newlines produced by programs are no longer translated to carriage-return linefeed sequences. Instead, newlines remain as newlines.
Expect scripts written for the “normal” pty modes could fail if they were to only use Emacs’ idea of pty modes. To avoid this, Expect performs a three-step pty initialization which leaves the pty with a suitable mixture of Emacs and user pty characteristics.
The first step initializes the pty so that it is configured just like the user’s terminal. Next, the pty is forced into a sane state (as I described earlier). In most cases, this changes nothing; however, anything too unusual is reset. This is also important when the process is running from cron
where there is no terminal from which to copy attributes in the first place. Finally, any other pty modes are changed according to the requirements of the script.
Each of these steps is controllable. The first step, copying the user’s terminal modes, is done unless spawn
is invoked with the -nottycopy
flag. The second step, forcing the pty into a sane state, is done unless spawn
is invoked with the -nottyinit
flag. The third step is only done if the stty_init
variable is defined, in which case it is passed as arguments to the stty
program.
The order that the flags are given to spawn
is irrelevant, but they must appear before the program name. Here are several examples. In each case, prog
stands for a program to be spawned.
spawn -nottyinitprog
spawn -nottyinit -nottycopyprog
set stty_init "kill ! susp ?" spawnprog
The last example sets the kill character to “!” and the suspend character to “?”. Conceivably, this could be useful or necessary for running or testing a particular program. The spawn
command does not enforce any kind of pty initialization. It is possible to use -nottycopy
and -nottyinit
and not define stty_init
but this is not a good idea. Ptys are not otherwise initializated by most systems.
These options may seem complex, but in most cases they are not necessary. Going back to Emacs for a moment, the default behavior of spawn
allows the correct functioning of Expect scripts. Expect scripts may “look funny” inside of Emacs with respect to character echoing, but then, so do commands such as telnet
and rlogin
. If you absolutely have to have Emacs look correct, use the -nottyinit
flag. However, you must then go to extra effort to make your scripts avoid any dependencies on echoing, line termination characters, and anything else that the Emacs terminal modes affect.
Another example of how stty
can introduce unexpected results is with the line kill character. On some UNIX implementations, stty
believes the @
is the default line kill character (i.e., pressing it removes all previous characters typed on the line). The @
was a popular convention many years ago. Now, it is just archaic and ^U is much more common. On such archaic systems, sending strings such as "user@hostname
" ends up sending only "hostname
“.
Yet another problem that occasionally crops up is what to do with parity. On some UNIX implementations, stty
believes that parity should be disabled. This confuses programs that work with 8-bit character sets. If you can not fix your local stty
, work around the problem by using the -nosttyinit
flag or by setting stty_init
to -istrip
.
Historically, UNIX systems have provided a fixed number of ptys, pre-allocating filenames in the file system for each one. Most versions of UNIX no longer do this, but there are still some that do. With a static set of ptys, it is necessary to search through the list of files. Expect performs several tests on each pty before using it. These tests ensure that no other process is still using the pty.
Usually these tests are very quick, but programs that have misbehaved and are sloppy in their pty allocation and deallocation can force Expect to take up to ten seconds, waiting for a response from a pty that is still in use.[51] Normally, Expect goes on and continues trying other ptys until it finds one that can be allocated; however, such ptys can cause problems for most other programs. For example, programs that use ptys, such as xterm
and Emacs, simply give up when encountering such a pty. If you see this happening, you can try spawning a process with Expect’s diagnostic mode enabled. Expect will then report the ptys it is ignoring and you can verify that each one is in use by a functioning program. In some cases, the program may have exited but left the pty in a bizarre state. Expect’s thorough pty-initialization procedure will reset the pty so that other processes can use it.
You can take advantage of Expect’s ability to fix ptys with the following script called ptyfix
.
#!/usr/local/bin/expect-- spawn cat
Or even simpler, just put the following shell command in an alias or menu:
expect -c "spawn cat"
There is no explicit restriction on spawning multiple processes—any number of processes may be running under control of Expect. However, some old—perhaps archaic is a better word—systems do not provide a facility for listening from multiple processes simultaneously. When Expect is installed, it looks for the presence of the select
or poll
system call. Either of these usually indicates that Expect can listen to multiple processes simultaneously.
Some systems provide select
or poll
but do not allow them to be used the way Expect needs. In this case, Expect simulates this functionality using the read
system call with alarms. When using read
, Expect has one major restriction. Only one process can be listened to (with either expect
or interact
) at a time.
Fortunately, such systems are rare and growing rarer.[52] Although you cannot run Expect with all of its power on them, you can still get useful work done even by automating one application at a time.
While the spawn
command returns a process id, Expect can provide this information at any time by using the exp_pid
command. With no arguments, exp_pid
returns the process id of the currently spawned process. The process id of a particular spawn id can be returned by using a −i
flag. For example:
expect1.1> exp_pid −i $shell
20004
Do not confuse this command with pid
. The pid
command is a built-in Tcl command that returns the process id of the Expect process itself.
You cannot directly read from or write to spawned processes with puts
and gets
. In general, there is little need for it because you can emulate the behavior with suitable send
and expect
commands. Nonetheless, it may be convenient to do so at times.
Earlier, I showed how the -open
flag of the spawn
command converts a file identifier to a spawn id. The exp_open
command does the opposite. It converts a spawn id to a file identifier that may be used with gets
and puts
. The file identifier will be open for both reading and writing. If exp_open
is called with no arguments, it converts the spawn id of the currently spawned process. If called with a −i
argument, exp_open
converts the given spawn id.
By default, after calling exp_open
, the spawn id can no longer be accessed using send
and expect
. It becomes owned entirely by Tcl and should eventually be closed in the Tcl style and without doing a wait
. On some systems, processes return spurious error indications during a close
operation. Expect knows to ignore these errors; however, you may have to explicitly catch them from Tcl.
spawn /bin/csh set file [exp_open] catch {close $file}
You may have to call flush
explicitly after I/O operations because the file commands normally buffer internally. Process output that does not terminate with a newline may be impossible to read unless you disable buffering or read it explicitly with read
. In the following example, the first output from telnet
is read using read
since it does not end with a newline.
%expect
expect1.1>spawn -noecho telnet
4484 expect1.2>exp_open
file5 expect1.3>read file5 7
telnet>
The spawn id can be left open by calling exp_open
with the -leaveopen
flag. In this case, both the file and the spawn id must be closed explicitly. A wait
must be executed. As with the -leaveopen
flag in the spawn
command, alternation of Tcl and Expect commands is best avoided because Tcl and Expect do not share buffers internally.
On page 146 of Advanced UNIX Programming (Prentice Hall), Marc Rochkind describes how deadlock can occur when using pipes for bidirectional communication. Why does this not apply to Expect?
Modify the firstline
script (page 178) to make it check that the spawn
command succeeds and that the program is successfully executed. Upon failure, send any diagnostics to the standard error and return a nonzero status.
On page 292, there are two commands in the string passed to /bin/sh
. Simplify the string.
Write a script that starts two xterm
s, each of which use a separate shell (as usual). Make the script create a transcript of both xterm
s in a single file. Provide a parameter that switches from logging by line to logging by individual character.
It is possible to write a better version of ptyfix
(page 302) using the diagnostic output from Expect. Modify the script so that when Expect reports that a pty is hung, the new version finds the process that is responsible and kills it.
[46] The effect of this particular bug also shows up in the failure of expect_user to read input. A patch is available from the vendor.
[47] On some systems, the -u flag to cat has a negative impact on performance.
Chapter 23(p. 526) shows a fast and simple way to test for this.
[48] Deficient select or poll support is such a severe defect that it is usually documented in the brief cover notes that accompany the operating system. For this reason, it is not necessary to understand further details about select and poll, so I will not provide any.
[49] As this book is being written, an initial release of R6 has appeared in which the relevant flag to xterm is identical to R5. So it is likely that the description of xterm in this section will continue to be valid. I have recommended to the X Consortium that xterm be modified so that it takes complete pty names. Then, xterm would not have to make any assumptions about the structure of the names. As of this writing, the code shown here is the most portable that can be written.
[50] "spawn -open" returns a process id of 0 to indicate no process was spawned. There is no process to kill.
[51] Expect leaves a timestamp in the form of a file in /tmp recording such ptys so that later attempts do not bother waiting. The file is left even after Expect exits, allowing later Expect processes to take advantage of this information. After an hour, the next Expect deletes the file and retests the pty.
[52] Expect detects and reports at installation time if your system cannot spawn multiple processes simultaneously.