As you start building shell scripts, you’ll probably start to wonder how to run and control them on your Linux system. So far in this book, we’ve only run commands and scripts directly from the command-line interface in real-time mode. This isn’t the only way to run scripts in Linux. There are quite a few other options available for running your shell scripts on Linux systems. This chapter examines different ways you can use to get your scripts started. Also, sometimes you might run into the problem of a script that gets stuck in a loop and you need to figure out how to get it to stop without having to turn off your Linux system. This chapter also examines the different ways you can control how and when your shell script runs on your system.
There are times when running a shell script directly from the command-line interface is inconvenient. Some scripts can take a long time to process, and you may not want to tie up the command-line interface waiting. While the script is running, you can’t do anything else in your terminal session. Fortunately, there’s a simple solution to that problem. The following sections describe how to run your scripts in background mode on your Linux system.
Running a shell script in background mode is a fairly easy thing to do. To run a shell script in background mode from the command-line interface, just place an ampersand symbol after the command:
$ ./test1.sh &
[1] 19555
$ This is a test program
Loop #1
Loop #2
$ ls -l
total 8
-rwxr--r-- 1 rich rich 219 Feb 26 19:27 test1.sh
$ Loop #3
When you place the ampersand symbol after a command, it separates the command from the Bash shell and runs it as a separate background process on the system. The first thing that displays is the line
[1] 19555
The number in the square brackets is the job number the shell assigns to the background process. The shell assigns each process started a unique job number. The next number is the process ID (PID) the Linux system itself assigns to the process. So every process running in a shell has a unique job number, and every process running on the Linux system has a unique PID.
As soon as the system displays these items, a new command-line interface prompt appears. You are returned to the shell, and the command you executed runs safely in background mode.
At this point, you can enter new commands at the prompt (as shown in the example). However, while the background process is still running, it still uses your terminal monitor for output messages. You’ll notice from the example that the output from the test1.sh
script appears in the output intermixed with any other commands that are run from the shell.
When the background process finishes, it displays a message on the terminal:
[1]+ Done ./test1.sh
This shows the job number and the status of the job (Done), along with the command used to start the job.
You can start any number of background jobs at the same time from the command-line prompt:
$ ./test1.sh &
[1] 19582
$ This is the test1 program output
Test 1 Loop #1 output
$ ./test2.sh &
[2] 19597
$ This is the test2 program output
Test 2 Loop #1 output
$ ./test3.sh &
[3] 19612
$ This is the test3 program output
Test 3 Loop #1 output
Test 1 Loop #2 output
Test 2 Loop #2 output
Test 3 Loop #2 output
Each time you start a new job, the shell assigns it a new job number, and the Linux system assigns it a new PID. You can see that all of the scripts are running by using the ps
command:
$ ps au
USER PID %CPU %MEM VSZ RSS TTY STAT START TIME COMMAND
rich 19498 0.0 1.2 2688 1628 pts/0 S 11:38 0:00 -bash
rich 19582 0.0 0.9 2276 1180 pts/0 S 11:55 0:00 /bin/bash ./test3.sh
rich 9597 0.1 0.9 2276 1180 pts/0 S 11:55 0:00 /bin/bash ./test2.sh
rich 19612 0.1 0.9 2276 1180 pts/0 S 11:55 0:00 /bin/bash ./test1.sh
rich 19639 0.0 0.4 1564 552 pts/0 S 11:56 0:00 sleep 10
rich 19640 0.0 0.4 1564 552 pts/0 S 11:56 0:00 sleep 10
rich 19641 0.0 0.4 1564 552 pts/0 S 11:56 0:00 sleep 10
rich 19642 0.0 0.5 2588 744 pts/0 R 11:56 0:00 ps au
$
Each of the background processes you start appears in the ps
command output listing of running processes. If all of the processes display output in your terminal session, things can get pretty messy pretty quickly. Fortunately, there’s a simple way to solve that problem, which we’ll discuss in the next section.
You need to be careful when using background processes from a terminal session. Notice in the output from the ps
command that each of the background processes is tied to the terminal session (pts/0) terminal. If the terminal session exits, the background process also exits. Some terminal emulators warn you if you have any running background processes associated with the terminal, while others don’t. If you want your script to continue running in background mode after you’ve logged off the console, there’s something else you need to do. The next section discusses that process.
There will be times when you want to start a shell script from a terminal session and then let the script run in background mode until it finishes, even if you exit the terminal session. You can do this by using the nohup command.
The nohup
command runs another command blocking any SIGHUP signals that are sent to the process. This prevents the process from exiting when you exit your terminal session.
You can combine the nohup
command with the ampersand to run a script in the background and not allow it to be interrupted:
$ nohup ./test1.sh &
[1] 19831
$ nohup: appending output to 'nohup.out'
$
Just as with a normal background process, the shell assigns the command a job number, and the Linux system assigns a PID number. The difference is that when you use the nohup
command, the script ignores any SIGHUP signals sent by the terminal session if you close the session.
Because the nohup
command disassociates the process from the terminal, the process loses the output link to your monitor. To accommodate any output generated by the command, the nohup
command automatically redirects output messages to a file, called nohup.out
, in the current working directory.
The nohup.out
file contains all of the output that would normally be sent to the terminal monitor. After the process finishes running, you can view the nohup.out
file for the output results:
$ cat nohup.out
This is a test program
Loop #1
Loop #2
Loop #3
Loop #4
Loop #5
Loop #6
Loop #7
Loop #8
Loop #9
Loop #10
This is the end of the test program
$
The output appears in the nohup.out
file just as if the process ran on the command line!
If you run another command using nohup
, the output is appended to the existing nohup.out
file. Be careful when running multiple commands from the same directory, as all of the output will be sent to the same nohup.out
file, which can get confusing.
As discussed in Chapter 21, the Bash shell can send signals to processes running on the system. This allows you to stop or interrupt a runaway application process if necessary. While the kill
and pkill
commands discussed in Chapter 21 are good for stopping background processes, applications running in the foreground on the console are harder to control. Fortunately, there are two basic Linux signals you can generate using key combinations on the keyboard to interrupt or stop a foreground process.
The Ctrl+C key combination generates a SIGINT signal and sends it to any processes currently running in the shell. You can test this by running a command that normally takes a long time to finish and pressing the Ctrl+C key combination:
$ sleep 100
$
The Ctrl+C key combination doesn’t produce any output on the monitor; it just stops the current process running in the shell.
Instead of terminating a process, you can pause it in the middle of whatever it’s doing. Sometimes this can be a dangerous thing (for example, if a script has a file lock open on a crucial system file), but often it allows you to peek inside what a script is doing without actually terminating the process.
The Ctrl+Z key combination generates a SIGTSTP signal, stopping any processes running in the shell. Stopping a process is different than terminating the process, as stopping the process leaves the program still in memory and able to continue running from where it left off. In the section “Job Control” later, you’ll learn how to restart a process that’s been stopped.
When you use the Ctrl+Z key combination, the shell informs you that the process has been stopped:
$ sleep 100
[1]+ Stopped sleep 100
$
The number in the square brackets indicates the job number for the process in the shell. If you have a stopped job assigned to your shell session, Bash will warn you if you try to exit the shell:
$ exit
logout
There are stopped jobs.
$
You can view the stopped job by using the ps
command:
$ ps au
USER PID %CPU %MEM VSZ RSS TTY STAT START TIME COMMAND
rich 20560 0.0 1.2 2688 1624 pts/0 S 05:15 0:00 -bash
rich 20605 0.2 0.4 1564 552 pts/0 T 05:22 0:00 sleep 100
rich 20606 0.0 0.5 2584 740 pts/0 R 05:22 0:00 ps au
$
The ps
command shows the status of the stopped job as T
, which indicates the command either is being traced or is stopped. The original Bash shell is shown as S
, indicating that it’s sleeping, waiting for the script to end. The ps
command itself is shown with an R
status, indicating that it’s the currently running job.
If you really want to exit the shell with the stopped job still active, just type the exit
command again. The shell will exit, terminating the stopped job. Alternately, now that you know the PID of the stopped job, you can use the kill
command to send a SIGKILL signal to terminate it:
$ kill -9 20605
$
[1]+ Killed sleep 100
$
When you kill the job, initially you won’t get any response. However, the next time you do something that produces a shell prompt, you’ll see a message indicating that the job was killed. Each time the shell produces a prompt, it also displays the status of any jobs that have changed states in the shell.
In the previous section you saw how to use the Ctrl+Z key combination to stop a job running in the shell. After you stop a job, the Linux system lets you either kill or restart it. Restarting a stopped process requires sending it a SIGCONT signal.
The function of starting, stopping, killing, and resuming jobs is called job control. With job control, you have full control over how processes run in your shell environment.
The following sections describe the commands to use to view and control jobs running in your shell.
The key command for job control is the jobs command. The jobs
command allows you to view the current jobs being handled by the shell. Listing 26.1 uses a shell script to demonstrate viewing a stopped job.
Listing 26.1: Stopping a running job
$ cat test2.sh
#!/bin/bash
# testing job control
echo "This is a test program $$"
count=1
while [ $count -le 10 ] ; do
echo "Loop #$count"
sleep 10
count=$[ $count + 1 ]
done
echo "This is the end of the test program"
$ ./test2.sh
This is a test program 29011
Loop #1
[1]+ Stopped ./test2.sh
$ ./test2.sh > test2.sh.out &
[2] 28861
$
$ jobs
[1]+ Stopped ./test2.sh
[2]- Running ./test2/sh >test2.shout &
$
The script shown in Listing 26.1 uses the $$
variable to display the PID that the Linux system assigns to the script; then it goes into a loop, sleeping for 10 seconds at a time for each iteration. In the example, we start the first script from the command-line interface and then stop it using the Ctrl+Z key combination. Next, another job is started as a background process, using the ampersand symbol. To make life a little easier, we redirected the output of that script to a file so that it wouldn’t appear on the monitor.
After the two jobs were started, we used the jobs
command to view the jobs assigned to the shell. The jobs
command shows both the stopped and the running jobs along with their job numbers and the commands used in the jobs.
The jobs
command uses a few different command-line parameters, shown in Table 26.1.
Table 26.1 The jobs
command parameters
Parameter | Description |
-l |
Lists the PID of the process along with the job number |
-n |
Lists only jobs that have changed their status since the last notification from the shell |
-p |
Lists only the PIDs of the jobs |
-r |
Lists only the running jobs |
-s |
Lists only stopped jobs |
You probably noticed the plus and minus signs in the output in Listing 26.1. The job with the plus sign is considered the default job. It would be the job referenced by any job control commands if a job number wasn’t specified in the command line. The job with the minus sign is the job that would become the default job when the current default job finishes processing. There will only be one job with the plus sign and one job with the minus sign at any time, no matter how many jobs are running in the shell.
Listing 26.2 shows an example of how the next job in line takes over the default status when the default job is removed.
Listing 26.2: Demonstrating job control
$ ./test2.sh
This is a test program 29075
Loop #1
[1]+ Stopped ./test2.sh
$ ./test2.sh
This is a test program 29090
Loop #1
[2]+ Stopped ./test2.sh
$ ./test2.sh
This is a test program 29105
Loop #1
[3]+ Stopped ./test2.sh
$ jobs -l
[1] 29075 Stopped ./test2.sh
[2]- 29090 Stopped ./test2.sh
[3]+ 29105 Stopped ./test2.sh
$ kill -9 29105
$ jobs -l
[1]- 29075 Stopped ./test2.sh
[2]+ 29090 Stopped ./test2.sh
$
In Listing 26.2 we started, then stopped, three separate processes. The jobs
command listing shows the three processes and their status. Note by the PID numbers that the default process (the one listed with the plus sign) is 29105, the last process started.
We then used the kill
command to send a SIGHUP signal to the default process. In the next jobs listing, the job that previously had the minus sign, 29090, is now the default job.
Under Bash job control, you can restart any stopped job as either a background process or a foreground process. A foreground process takes over control of the terminal you’re working on, so be careful about using that feature.
To restart a job in background mode, use the bg
command along with the job number:
$ bg 2
[2]+ ./test2.sh &
Test 2 Loop #2 output
$ Test 2 Loop #3 output
Test 2 Loop #4 output
$ jobs
[1]+ Stopped ./test2.sh
[2]- Running ./test2.sh &
$ Test 2 Loop #6 output
Test 2 Loop #7 output
Test 2 Loop #8 output
Test 2 Loop #9 output
Test 2 Loop #10 output
This is the end of the test2 program
[2]- Done ./test2.sh
$
Since we restarted the job in background mode, the command-line interface prompt appears, allowing us to continue with other commands. The output from the jobs
command now shows that the job is indeed running (as you can tell from the output now appearing on the monitor).
To restart a job in foreground mode, use the fg
command along with the job number:
$ jobs
[1]+ Stopped ./test2.sh
$ fg 1
./test4
Loop #2
Loop #3
Since the job is running in foreground mode, we don’t get a new command-line interface prompt until the jobs finishes.
We’re sure that, as you start working with scripts, there’ll be a situation in which you’ll want to run a script at a preset time, usually at a time when you’re not there. There are two common ways of running a script at a preselected time:
at
commandcron
tableEach method uses a different technique for scheduling when and how often to run scripts. The following sections describe each of these methods.
The at command allows you to specify a time when the Linux system will run a script. It submits a job to a queue with directions on when the shell should run the job. Another command, atd
, runs in the background and checks the job queue for jobs to run. Most Linux distributions start this automatically at boot time.
The atd
command checks a special directory on the system (usually /var/spool/at
) for jobs submitted using the at
command. By default, the atd
command checks this directory every 60 seconds. When a job is present, the atd
command checks the time the job is set to be run. If the time matches the current time, the atd
command runs the job.
The following sections describe how to use the at
command to submit jobs to run and how to manage jobs.
The basic at
command format is pretty simple:
at [-f filename] time
By default, the at
command submits input from STDIN to the queue. You can specify a file name used to read commands (your script file) using the -f
parameter.
The time
parameter specifies when you want the Linux system to run the job. You can get pretty creative with how you specify the time. The at
command recognizes lots of different time formats:
If you specify a time that’s already past, the at
command runs the job at that time on the next day.
Besides specifying the time to run the job, you can also include a specific date, using a few different date formats:
When you use the at
command, the job is submitted into a job queue. The job queue holds the jobs submitted by the at
command for processing. There are 26 different job queues available for different priority levels. Job queues are referenced using lowercase letters, a
through z
.
By default all at
jobs are submitted to job queue a
, the highest-priority queue. If you want to run a job at a lower priority, you can specify the letter using the -q
parameter.
When the job runs on the Linux system, there’s no monitor associated with the job. Instead, the Linux system uses the email address of the user who submitted the job. Any output destined to STDOUT or STDERR is mailed to the user via the mail system.
Listing 26.3 shows a simple example of using the at
command to schedule a job to run.
Listing 26.3: Using the at command to start a job
$ date
Thu Feb 28 18:48:20 EST 2019
$ at -f test3.sh 18:49
job 2 at Thu Feb 28 18:49:00 2019
Heirloom Mail version 12.5 7/5/10. Type ? for help.
"/var/spool/mail/rich": 1 message 1 new
>N 1 Rich Thu Feb 28 18:49 15/568 "Output from your job "
&
Message 1:
From [email protected] Thu Feb 28 18:49:00 2019
Return-Path: <[email protected]>
X-Original-To: rich
Delivered-To: [email protected]
Subject: Output from your job 2
To: [email protected]
Date: Thu, 28 Feb 2019 18:49:00 -0500 (EST)
From: [email protected] (Rich)
Status: R
"This script ran at 18:49:00"
"This is the end of the script"
&
As shown in Listing 26.3, when we ran the at
command, it produced a warning message, indicating what shell the system uses to run the script (the default shell assigned to /bin/sh
, which for Linux is the Bash shell) along with the job number assigned to the job and the time the job is scheduled to run.
When the job completes, nothing appears on the monitor, but the system generates an email message. The email message shows the output generated by the script. If the script doesn’t produce any output, it won’t generate an email message, by default. You can change that by using the -m
option in the at
command. This generates an email message, indicating the job completed, even if the script doesn’t generate any output.
The atq
command allows you to view what jobs are pending on the system:
$ at -f test3.sh 19:15
warning: commands will be executed using /bin/sh
job 7 at 2007-11-04 10:15
$ at -f test5 4PM
warning: commands will be executed using /bin/sh
job 8 at 2007-11-03 16:00
$ at -f test5 1PM tomorrow
warning: commands will be executed using /bin/sh
job 9 at 2007-11-04 13:00
$ atq
7 2007-11-04 10:15 a
8 2007-11-03 16:00 a
9 2007-11-04 13:00 a
$
The job listing shows the job number, the date and time the system will run the job, and the job queue the job is stored in.
After you know the information about what jobs are pending in the job queues, you can use the atrm
command to remove a pending job:
$ atrm 8
$ atq
7 2007-11-04 10:15 a
9 2007-11-04 13:00 a
$
Just specify the number of the job you want to remove. You can only remove jobs that you submit for execution. You can’t remove jobs submitted by others.
Using the at
command to schedule a script to run at a preset time is great, but what if you need that script to run at the same time every day or once a week or once a month? Instead of having to continually submit at
jobs, you can use another feature of the Linux system.
The Linux system uses the cron program to allow you to schedule jobs that need to run on a regular basis. The cron
program runs in the background and checks special tables, called cron tables, for jobs that are scheduled to run.
The cron table uses a special format for allowing you to specify when a job should be run. The format for the cron table is as follows:
min hour dayofmonth month dayofweek command
The cron table allows you to specify entries as specific values, ranges of values (such as 1–5) or as a wildcard character (the asterisk). For example, if you want to run a command at 10:15 a.m. every day, you would use the cron table entry of
15 10 * * * command
Because you can’t indicate a.m. or p.m. in the cron table, you’ll need to use the 24-hour clock format for p.m. times. The wildcard character used in the dayofmonth
, month
, and dayofweek
fields indicates that cron
will execute the command every day of every month at 10:15 a.m. To specify a command to run at 4:15 p.m. every Monday, you would use
15 16 * * 1 command
You can specify the dayofweek
entry either as a three-character text value (mon, tue, wed, thu, fri, sat, sun) or as a numeric value, with 0 being Sunday and 6 being Saturday.
Here’s another example: to execute a command at 12 noon on the first day of every month, you’d use the format
00 12 1 * * command
The dayofmonth
entry specifies a date value (1–31) for the month.
When specifying the command or shell to run, you must use its full pathname. You can add any command-line parameters or redirection symbols you like, as a regular command line:
15 10 * * * /home/rich/test4.sh > test4out
The cron
program runs the script using the user account that submitted the job. Thus, you must have the proper permissions to access the command and output files specified in the command listing.
All system users can have their own cron table (including the root user) for running scheduled jobs. Linux provides the crontab
command for handling the cron table. To list an existing cron table, use the -l
parameter:
$ crontab -l
no crontab for rich
$
To add entries to your cron table, use the -e
parameter. When you do that, the crontab command automatically starts the vi
editor with the existing cron table or an empty file if it doesn’t yet exist.
This exercise walks through how to run, pause, stop, and view jobs running within the Bash shell.
jobtest.sh
by typing nano jobtest.sh
, pico jobtest.sh
, or vi jobtest.sh
.#!/bin/bash
# jobtest.sh - run the sleep command in a loop for job testing
echo "This is a test program $$"
count=1
while [ $count -le 10 ] ; do
echo "Program: $$ Loop #$count"
sleep 10
count=$[ $count + 1 ]
done
echo "This is the end of the test program"
chmod u+x jobtest.sh
../jobtest.sh
.
../jobtest.sh &.
jobs
.bg n
, where n is the job number assigned to the paused job.jobs
. Note the status of the job that was previously paused.kill -9
command along with the appropriate PID values assigned to each job.jobs
to view the currently running jobs.By default, when you run a script in a terminal session shell, the interactive shell is suspended until the script completes. You can cause a script or command to run in background mode by adding an ampersand sign (&
) after the command name. When you run a script or command in background mode, the interactive shell returns, allowing you to continue entering more commands. Any background processes run using this method are still tied to the terminal session. If you exit the terminal session, the background processes also exit.
To prevent this from happening, use the nohup
command. This command intercepts any signals intended for the command that would stop it, such as, for example, when you exit the terminal session. This allows scripts to continue running in background mode even if you exit the terminal session.
When you move a process to background mode, you can still control what happens to it. The jobs
command allows you to view processes started from the shell session. Once you know the job ID of a background process, you can use the kill
command to send Linux signals to the process or use the fg
command to bring the process back to the foreground in the shell session. You can suspend a running foreground process by using the Ctrl+Z key combination and then place it back in background mode using the bg
command.
Besides controlling processes while they’re running, you can also determine when a process starts on the system. Instead of running a script directly from the command-line interface prompt, you can schedule the process to run at an alternative time. There are several different ways to accomplish this. The at
command allows you to run a script once at a preset time. The cron
program provides an interface that can run scripts at a regularly scheduled interval.
Describe how to run a shell script in background mode from your console or terminal session. To run a shell script in background mode, include the ampersand sign (&
) after the shell script command on the command line. The shell will run the script in background mode and produce another command prompt for you to continue on within the shell.
Explain how to disconnect a shell script from the console or terminal session so it continues running if the session closes. The nohup
command disconnects the shell script from the shell session and runs it as a separate process. If the console or terminal session exits, the shell script will continue running.
Explain how to stop or pause a shell script running in the foreground on a console or terminal session. To stop a shell script running in the foreground of a console or terminal session, press the Ctrl+C key combination. To pause a running shell script, press the Ctrl+Z key combination.
Describe how to list shell scripts running in background mode within a console or terminal session. The jobs
command allows you to list the commands that are running within the console or terminal session. The output from the jobs
command displays both the job number assigned by the shell and the process ID assigned by the Linux system.
Describe how to run a shell script at a specific time. The at
command allows you to schedule a job to run at a specific time. You can specify the time using an exact value, such as 10:00 p.m., or using common date and time references, such as 10:00 a.m. tomorrow.
Explain how to run a shell script automatically at a set time every day. The cron
process runs every minute and checks for jobs that are scheduled to run. You must define the jobs to run in the cron table by using the crontab
command.
Frank wants to run his large number-crunching application in background mode on his console session. What command does he need to use to do that?
>
&
|
>>
nohup
What command do you use to disconnect a shell script from the current console so that it can continue to run after the console exits?
>
&
|
>>
nohup
When Melanie runs a shell script, she notices that it takes up all of the memory on her Linux system and she needs to stop it. How can she do that?
nohup
command.&
) command.kill
command to stop it.How can you temporarily pause a shell script from running in foreground mode in a console session?
nohup
command.&
) command.fg
command.How do you determine the default job running in a console session?
jobs
outputjobs
outputps
commandBarbara has an application running in background mode in her console session and needs to bring it to foreground mode. What command should she use to do that?
bg
fg
nohup
&
at
What command allows you to run a shell script at a specific time?
nohup
&
at
>
Nick needs to run a report at midnight every day on his Linux system. How should he do that?
at
command to schedule the job.nohup
command.&
) symbol.cron
.atq
command.When will the cron table entry 10 5 * * * myscript
run the specified shell script?
Jane needs to check on what jobs are scheduled to run automatically for her user account. What command should she use to list the cron table entries for her user account?
cron
at
crontab
jobs
nohup