Commands in Use

In this chapter, we introduce the ftp command.

ftp 192.168.0.125

ftp 192.168.0.125 50

Crawling Websites

To copy a web page from across a network, you can use the curl command. At its core, curl is simple to use, but it has many advanced options such as the ability to handle remote authentication and session cookies. It is common to use the -L option with curl, as it will then follow HTTP redirects if the page’s location has changed. By default, curl will display the raw HTML to stdout, but it can be sent to a file by using redirection or the -o option:

curl -L -o output.html https://www.oreilly.com

The curl command can also be used to gather header information from a server by using the -I option. This can be useful when trying to identify the web server version or operating system. As you can see in this example, the server is reporting that it is using Apache 2.4.7 and the Ubuntu operating system:

$ curl -LI https://www.oreilly.com

HTTP/1.1 200 OK
Server: Apache/2.4.7 (Ubuntu)
Last-Modified: Fri, 19 Oct 2018 08:30:02 GMT
Content-Type: text/html
Cache-Control: max-age=7428
Expires: Fri, 19 Oct 2018 16:16:48 GMT
Date: Fri, 19 Oct 2018 14:13:00 GMT
Connection: keep-alive

$ curl -LIs https://www.oreilly.com | grep '200 OK'

HTTP/1.1 200 OK

One significant limitation of curl is that it will retrieve only the page specified; it does not have functionality to crawl an entire website or follow links within a page.

Automated Banner Grabbing

When you connect to a server, it sometimes reveals information about the web service application or the operating system. This is called a banner. When connecting to the O’Reilly web server, you’ll see an operating system banner in the HTTP header:

HTTP/1.1 200 OK
Server: Apache/2.4.7 (Ubuntu)
Last-Modified: Fri, 19 Oct 2018 08:30:02 GMT
Content-Type: text/html
Cache-Control: max-age=7428
Expires: Fri, 19 Oct 2018 16:16:48 GMT
Date: Fri, 19 Oct 2018 14:13:00 GMT
Connection: keep-alive

Information about the operating system of a potential target is valuable. It can inform you as to what vulnerabilities might exist in the system, which can later be used during the Initial Compromise phase of the Attack Life Cycle.

Several types of systems commonly display banners including web servers, FTP servers, and Simple Mail Transfer Protocol (SMTP) servers. Table 13-1 shows the network ports normally used by these services.

Recall in Chapter 9 that we looked at how to perform a network port scan with scan.sh. That script can be extended such that each time a host is found with one of the FTP, SMTP, or HTTP ports open, the script will attempt to retrieve and save the server’s banner.

You have already seen how the curl command can be used to capture an HTTP header, which can include a banner:

curl -LI https://www.oreilly.com

To capture the banner from an FTP server, the ftp command can be used:

$ ftp -n 192.168.0.16

Connected to 192.168.0.16.
220 (vsFTPd 3.0.3)
ftp>

The -n option is used to stop the ftp command from automatically trying to log into the server. Once connected, to close the FTP connection, type quit at the ftp> terminal.

The easiest way to capture the banner from an SMTP server is to use the telnet command with network port 25:

$ telnet 192.168.0.16 25

Connected to 192.168.0.16
Escape character is '^]'.
220 localhost.localdomain ESMTP Postfix (Ubuntu)

The telnet command is available in most versions of Linux, but not Git Bash and not in many versions of Windows. In these cases, you can write a small script using the /dev/tcp bash file descriptor to accomplish the same thing.

Example 13-1 illustrates how to use the bash TCP file descriptor to connect to an SMTP server and capture a banner.

#!/bin/bash -
#
# Cybersecurity Ops with bash
# smtpconnect.sh
#
# Description:
# Connect to a SMTP server and print welcome banner
#
# Usage:
# smtpconnect.sh <host>
#   <host> SMTP server to connect to
#

exec 3<>/dev/tcp/"$1"/25
echo -e 'quit
' >&3
cat <&3

Here is the output when run:

$ ./smtpconnect.sh 192.168.0.16

220 localhost.localdomain ESMTP Postfix (Ubuntu)

Example 13-2 demonstrates how to put all of this together to automatically pull the banners from FTP, SMTP, and HTTP servers.

#!/bin/bash -
#
# Cybersecurity Ops with bash
# bannergrabber.sh
#
# Description:
# Automatically pull the banners from HTTP, SMTP,
# and FTP servers
#
# Usage: ./bannergrabber.sh  hostname [scratchfile]
#   scratchfile is used during processing but removed;
#   default is: "scratch.file" or tempfile-generated name
#

#
function isportopen ()
{
    (( $# < 2 )) && return 1                           
    local host port
    host=$1
    port=$2
    echo >/dev/null 2>&1  < /dev/tcp/${host}/${port}   
    return $?
}

function cleanup ()
{
    rm -f "$SCRATCH"
}

ATHOST="$1"
SCRATCH="$2"
if [[ -z $2 ]]
then
    if [[ -n $(type -p tempfile) ]]
    then
	SCRATCH=$(tempfile)
    else
    	SCRATCH='scratch.file'
    fi
fi

trap cleanup EXIT                                      
touch "$SCRATCH"                                       

if isportopen $ATHOST 21	# FTP                  
then
    # i.e., ftp -n $ATHOST
    exec 3<>/dev/tcp/${ATHOST}/21                      
    echo -e 'quit
' >&3                             
    cat <&3  >> "$SCRATCH"                             
fi

if isportopen $ATHOST 25	# SMTP
then
    # i.e., telnet $ATHOST 25
    exec 3<>/dev/tcp/${ATHOST}/25
    echo -e 'quit
' >&3
    cat <&3  >> "$SCRATCH"
fi

if isportopen $ATHOST 80	# HTTP
then
    curl -LIs "https://${ATHOST}"  >> "$SCRATCH"      
fi

cat "$SCRATCH"                                        

As you saw in Chapter 9, this script, too, will make use of the special filename /dev/tcp to open, or attempt to open, a TCP socket at the host and port number specified as part of that filename (e.g., /dev/tcp/127.0.0.1/631).

We begin the isportopen function with an error check to be sure that we were passed the correct number of parameters. We have not been doing this in most of our scripts, even though it is good programming practice to do so. We avoided such checks to avoid making the scripts overly complicated during the learning process; for real use in production environments, by all means use such error checks. It will also save time if debugging is necessary.

This is the heart of the technique to see whether the port is open. The three redirections may seem odd, but let’s break them down. The echo with no other arguments will echo a newline—and we do not really care about that. We are sending it to /dev/null (discarding it). Any error messages (stderr) will be directed to the same place. The crux of the matter is the input redirection. “But echo doesn’t read anything from stdin!” you might be thinking—true enough. However, bash will attempt to open the file named as the redirection of stdin—and the opening (or failing to open) is what tells us whether the port is (or is not) open. If the redirect fails, the overall command fails, and thus $? will be set to a nonzero value. If the redirect succeeds, then $? be zero.

We set the trap so that when the script exits, we are sure to remove our scratch file (via the cleanup function).

Now we create the file to make sure it’s there and ready for use. It prevents an error, should nothing else write to the file (see ).

This check will use our helper function to see if the FTP port (21) is open at the hostname specified by the user when the user invoked the script.

This use of exec is just to set file descriptor 3 to be open for both reading and writing (<>). The file that it is opening is the standard FTP port, 21.

This writes a short message to the FTP port to avoid leaving it open; we don’t want to perform any file transfers, so we tell it to quit. The -e option tells the echo command to interpret the escape sequences (the ), which are the characters that the TCP socket expects for line termination.

This reads from file descriptor 3, our TCP connection, and writes data returned into the scratch file. Notice the use of >> so that we append rather than rewrite the file. It’s not needed the first time we write to the file, but better to do it this way in case we ever rearrange the code (and the parallel construction—that is, all the uses of redirecting to $SCRATCH look the same).

For the HTTP connection, we don’t need to use /dev/tcp, because we can just use the curl command to much the same effect, appending the output into the scratch file.

The final step is to dump all the output that we found. If none of the ports had been open, nothing would have been written to the scratch file. We intentionally touch the file first thing so that we can cat the file without any File Not Found error.

Summary

Reconnaissance is one of the most important steps in any penetration test. The more information you have about a target, the easier it will be to launch a successful exploit. Be cautious when performing reconnaissance so as to not tip your hand too early. Be aware of which techniques are active (detectable by the target) and which are passive (not detectable by the target).

In the next chapter, we look at methods for obfuscating scripts that make them more difficult to reverse engineer or execute in the event they are captured by network defenders.

Workshop

1. Create a pipeline of commands that uses curl to retrieve a web page and then display any email addresses found on the page to the screen.

2. Modify smtpconnect.sh so that the network port used to connect is specified by a command-line argument (e.g., ./smtpconnect.sh 192.168.0.16 25).

3. Modify bannergrabber.sh so that instead of a single hostname specified on the command line, it reads in a list of multiple target IP addresses from a file.

4. Modify bannergrabber.sh so that it outputs a list of all discovered banners to a single file in the form of an HTML table.

Visit the Cybersecurity Ops website for additional resources and the answers to these questions.