8
IMPLEMENTING THE NETWORK PROTOCOL

Analyzing a network protocol can be an end in itself; however, most likely you’ll want to implement the protocol so you can actually test it for security vulnerabilities. In this chapter, you’ll learn ways to implement a protocol for testing purposes. I’ll cover techniques to repurpose as much existing code as possible to reduce the amount of development effort you’ll need to do.

This chapter uses my SuperFunkyChat application, which provides testing data and clients and servers to test against. Of course, you can use any protocol you like: the fundamentals should be the same.

Replaying Existing Captured Network Traffic

Ideally, we want to do only the minimum necessary to implement a client or server for security testing. One way to reduce the amount of effort required is to capture example network protocol traffic and replay it to real clients or servers. We’ll look at three ways to achieve this goal: using Netcat to send raw binary data, using Python to send UDP packets, and repurposing our analysis code in Chapter 5 to implement a client and a server.

Capturing Traffic with Netcat

Netcat is the simplest way to implement a network client or server. The basic Netcat tool is available on most platforms, although there are multiple versions with different command line options. (Netcat is sometimes called nc or netcat.) We’ll use the BSD version of Netcat, which is used on macOS and is the default on most Linux systems. You might need to adapt commands if you’re on a different operating system.

The first step when using Netcat is to capture some traffic you want to replay. We’ll use the Tshark command line version of Wireshark to capture traffic generated by SuperFunkyChat. (You may need to install Tshark on your platform.)

To limit our capture to packets sent to and received by our ChatServer running on TCP port 12345, we’ll use a Berkeley Packet Filter (BPF) expression to restrict the capture to a very specific set of packets. BPF expressions limit the packets captured, whereas Wireshark’s display filter limits only the display of a much larger set of capture packets.

Run the following command at the console to begin capturing port 12345 traffic and writing the output to the file capture.pcap. Replace INTNAME with the name of the interface you’re capturing from, such as eth0.

$ tshark -i INTNAME -w capture.pcap tcp port 12345

Make a client connection to the server to start the packet capture and then stop the capture by pressing CTRL+C in the console running Tshark. Make sure you’ve captured the correct traffic into the output file by running Tshark with the -r parameter and specifying the capture.pcap file. Listing 8-1 shows example output from Tshark with the addition of the parameters -z conv,tcp to print the list of capture conversations.

$ tshark -r capture.pcap -z conv,tcp
 ➊ 1 0 192.168.56.1 → 192.168.56.100 TCP 66 26082 → 12345 [SYN]
    2 0.000037695 192.168.56.100 → 192.168.56.1 TCP 66 12345 → 26082 [SYN, ACK]
    3 0.000239814 192.168.56.1 → 192.168.56.100 TCP 60 26082 → 12345 [ACK]
    4 0.007160883 192.168.56.1 → 192.168.56.100 TCP 60 26082 → 12345 [PSH, ACK]
    5 0.007225155 192.168.56.100 → 192.168.56.1 TCP 54 12345 → 26082 [ACK]
--snip--

================================================================================
TCP Conversations
Filter:<No Filter>
                                              |       <-      | |       ->      |
                                              | Frames  Bytes | | Frames  Bytes |
192.168.56.1:26082 <-> 192.168.56.100:12345➋   17      1020➌     28      1733➍
================================================================================

Listing 8-1: Verifying the capture of the chat protocol traffic

As you can see in Listing 8-1, Tshark prints the list of raw packets at ➊ and then displays the conversation summary ➋, which shows that we have a connection going from 192.168.56.1 port 26082 to 192.168.56.100 port 12345. The client on 192.168.56.1 has received 17 frames or 1020 bytes of data ➌, and the server received 28 frames or 1733 bytes of data ➍.

Now we use Tshark to export just the raw bytes for one direction of the conversation:

$ tshark -r capture.pcap -T fields -e data 'tcp.srcport==26082' > outbound.txt

This command reads the packet capture and outputs the data from each packet; it doesn’t filter out items like duplicate or out-of-order packets. There are a couple of details to note about this command. First, you should use this command only on captures produced on a reliable network, such as via localhost or a local network connection, or you might see erroneous packets in the output. Second, the data field is only available if the protocol isn’t decoded by a dissector. This is not an issue with the TCP capture, but when we move to UDP, we’ll need to disable dissectors for this command to work correctly.

Recall that at ➋ in Listing 8-1, the client session was using port 26082. The display filter tcp.srcport==26082 removes all traffic from the output that doesn’t have a TCP source port of 26082. This limits the output to traffic from the client to the server. The result is the data in hex format, similar to Listing 8-2.

$ cat outbound.txt
42494e58
0000000d
00000347
00
057573657231044f4e595800
--snip--

Listing 8-2: Example output from dumping raw traffic

Next, we convert this hex output to raw binary. The simplest way to do so is with the xxd tool, which is installed by default on most Unix-like systems. Run the xxd command, as shown in Listing 8-3, to convert the hex dump to a binary file. (The -p parameter converts raw hex dumps rather than the default xxd format of a numbered hex dump.)

$ xxd -p -r outbound.txt > outbound.bin
$ xxd outbound.bin
00000000: 4249 4e58 0000 000d 0000 0347 0005 7573  BINX.......G..us
00000010: 6572 3104 4f4e 5958 0000 0000 1c00 0009  er1.ONYX........
00000020: 7b03 0575 7365 7231 1462 6164 6765 7220  {..user1.badger
--snip--

Listing 8-3: Converting the hex dump to binary data

Finally, we can use Netcat with the binary data file. Run the following netcat command to send the client traffic in outbound.bin to a server at HOSTNAME port 12345. Any traffic sent from the server back to the client will be captured in inbound.bin.

$ netcat HOSTNAME 12345 < outbound.bin > inbound.bin

You can edit outbound.bin with a hex editor to change the session data you’re replaying. You can also use the inbound.bin file (or extract it from a PCAP) to send traffic back to a client by pretending to be the server using the following command:

$ netcat -l 12345 < inbound.bin > new_outbound.bin

Using Python to Resend Captured UDP Traffic

One limitation of using Netcat is that although it’s easy to replay a streaming protocol such as TCP, it’s not as easy to replay UDP traffic. The reason is that UDP traffic needs to maintain packet boundaries, as you saw when we tried to analyze the Chat Application protocol in Chapter 5. However, Netcat will just try to send as much data as it can when sending data from a file or a shell pipeline.

Instead, we’ll write a very simple Python script that will replay the UDP packets to the server and capture any results. First, we need to capture some UDP example chat protocol traffic using the ChatClient’s --udp command line parameter. Then we’ll use Tshark to save the packets to the file udp_capture.pcap, as shown here:

tshark -i INTNAME -w udp_capture.pcap udp port 12345

Next, we’ll again convert all client-to-server packets to hex strings so we can process them in the Python client:

tshark -T fields -e data -r udp_capture.pcap --disable-protocol gvsp/
  "udp.dstport==12345" > udp_outbound.txt

One difference in extracting the data from the UDP capture is that Tshark automatically tries to parse the traffic as the GVSP protocol. This results in the data field not being available. Therefore, we need to disable the GVSP dissector to create the correct output.

With a hex dump of the packets, we can finally create a very simple Python script to send the UDP packets and capture the response. Copy Listing 8-4 into udp_client.py.

udp_client.py

import sys
import binascii
from socket import socket, AF_INET, SOCK_DGRAM

if len(sys.argv) < 3:
    print("Specify destination host and port")
    exit(1)

# Create a UDP socket with a 1sec receive timeout
sock = socket(AF_INET, SOCK_DGRAM)
sock.settimeout(1)
addr = (sys.argv[1], int(sys.argv[2]))

for line in sys.stdin:
    msg = binascii.a2b_hex(line.strip())
    sock.sendto(msg, addr)

    try:
        data, server = sock.recvfrom(1024)
        print(binascii.b2a_hex(data))
    except:
        pass

Listing 8-4: A simple UDP client to send network traffic capture

Run the Python script using following command line (it should work in Python 2 and 3), replacing HOSTNAME with the appropriate host:

python udp_client.py HOSTNAME 12345 < udp_outbound.txt

The server should receive the packets, and any received packets in the client should be printed to the console as binary strings.

Repurposing Our Analysis Proxy

In Chapter 5, we implemented a simple proxy for SuperFunkyChat that captured traffic and implemented some basic traffic parsing. We can use the results of that analysis to implement a network client and a network server to replay and modify traffic, allowing us to reuse much of our existing work developing parsers and associated code rather than having to rewrite it for a different framework or language.

Capturing Example Traffic

Before we can implement a client or a server, we need to capture some traffic. We’ll use the parser.csx script we developed in Chapter 5 and the code in Listing 8-5 to create a proxy to capture the traffic from a connection.

chapter8_capture
_proxy.csx

   #load "parser.csx"
   using static System.Console;
   using static CANAPE.Cli.ConsoleUtils;

   var template = new FixedProxyTemplate();
   // Local port of 4444, destination 127.0.0.1:12345
   template.LocalPort = 4444;
   template.Host = "127.0.0.1";
   template.Port = 12345;
➊ template.AddLayer<Parser>();

   var service = template.Create();
   service.Start();
   WriteLine("Created {0}", service);
   WriteLine("Press Enter to exit...");
   ReadLine();
   service.Stop();

   WriteLine("Writing Outbound Packets to packets.bin");
➋ service.Packets.WriteToFile("packets.bin", "Out");

Listing 8-5: The proxy to capture chat traffic to a file

Listing 8-5 sets up a TCP listener on port 4444, forwards new connections to 127.0.0.1 port 12345, and captures the traffic. Notice that we still add our parsing code to the proxy at ➊ to ensure that the captured data has the data portion of the packet, not the length or checksum information. Also notice that at ➋, we write the packets to a file, which will include all outbound and inbound packets. We’ll need to filter out a specific direction of traffic later to send the capture over the network.

Run a single client connection through this proxy and exercise the client a good bit. Then close the connection in the client and press ENTER in the console to exit the proxy and write the packet data to packets.bin. (Keep a copy of this file; we’ll need it for our client and server.)

Implementing a Simple Network Client

Next, we’ll use the captured traffic to implement a simple network client. To do so, we’ll use the NetClientTemplate class to establish a new connection to the server and provide us with an interface to read and write network packets. Copy Listing 8-6 into a file named chapter8_client.csx.

chapter8
_client.csx

   #load "parser.csx"

   using static System.Console;
   using static CANAPE.Cli.ConsoleUtils;

➊ if (args.Length < 1) {
       WriteLine("Please Specify a Capture File");
       return;
   }
➋ var template = new NetClientTemplate();
   template.Port = 12345;
   template.Host = "127.0.0.1";
   template.AddLayer<Parser>();
➌ template.InitialData = new byte[] { 0x42, 0x49, 0x4E, 0x58 };

➍ var packets = LogPacketCollection.ReadFromFile(args[0]);

➎ using(var adapter = template.Connect()) {
       WriteLine("Connected");
       // Write packets to adapter
    ➏ foreach(var packet in packets.GetPacketsForTag("Out")) {
           adapter.Write(packet.Frame);
       }

       // Set a 1000ms timeout on read so we disconnect
       adapter.ReadTimeout = 1000;
    ➐ DataFrame frame = adapter.Read();
       while(frame != null) {
           WritePacket(frame);
           frame = adapter.Read();
       }
   }

Listing 8-6: A simple client to replace SuperFunkyChat traffic

One new bit in this code is that each script gets a list of command line arguments in the args variable ➊. By using command line arguments, we can specify different packet capture files without having to modify the script.

The NetClientTemplate is configured ➋ similarly to our proxy, making connections to 127.0.0.1:12345 but with a few differences to support the client. For example, because we parse the initial network traffic inside the Parser class, our capture file doesn’t contain the initial magic value that the client sends to the server. We add an InitialData array to the template with the magic bytes ➌ to correctly establish the connection.

We then read the packets from the file ➍ into a packet collection. When everything is configured, we call Connect() to establish a new connection to the server ➎. The Connect() method returns a Data Adapter that allows us to read and write parsed packets on the connection. Any packet we read will also go through the Parser and remove the length and checksum fields.

Next, we filter the loaded packets to only outbound and write them to the network connection ➏. The Parser class again ensures that any data packets we write have the appropriate headers attached before being sent to the server. Finally, we read out packets and print them to the console until the connection is closed or the read times out ➐.

When you run this script, passing the path to the packets we captured earlier, it should connect to the server and replay your session. For example, any message sent in the original capture should be re-sent.

Of course, just replaying the original traffic isn’t necessarily that useful. It would be more useful to modify traffic to test features of the protocol, and now that we have a very simple client, we can modify the traffic by adding some code to our send loop. For example, we might simply change our username in all packets to something else—say from user1 to bobsmith—by replacing the inner code of the send loop (at ➏ in Listing 8-6) with the code shown in Listing 8-7.

➊ string data = packet.Frame.ToDataString();
➋ data = data.Replace("u0005user1", "u0008bobsmith");
   adapter.Write(data.ToDataFrame());

Listing 8-7: A simple packet editor for the client

To edit the username, we first convert the packet into a format we can work with easily. In this case, we convert it to a binary string using the ToDataString() method ➊, which results in a C# string where each byte is converted directly to the same character value. Because the strings in SuperFunkyChat are prefixed with their length, at ➋ we use the uXXXX escape sequence to replace the byte 5 with 8 for the new length of the username. You can replace any nonprintable binary character in the same way, using the escape sequence for the byte values.

When you rerun the client, all instances of user1 should be replaced with bobsmith. (Of course, you can do far more complicated packet modification at this point, but I’ll leave that for you to experiment with.)

Implementing a Simple Server

We’ve implemented a simple client, but security issues can occur in both the client and server applications. So now we’ll implement a custom server similar to what we’ve done for the client.

First, we’ll implement a small class to act as our server code. This class will be created for every new connection. A Run() method in the class will get a Data Adapter object, essentially the same as the one we used for the client. Copy Listing 8-8 into a file called chat_server.csx.

chat_server.csx

   using CANAPE.Nodes;
   using CANAPE.DataAdapters;
   using CANAPE.Net.Templates;

➊ class ChatServerConfig {
       public LogPacketCollection Packets { get; private set; }
       public ChatServerConfig() {
           Packets = new LogPacketCollection();
       }
   }

➋ class ChatServer : BaseDataEndpoint<ChatServerConfig> {
       public override void Run(IDataAdapter adapter, ChatServerConfig config) {
           Console.WriteLine("New Connection");
        ➌ DataFrame frame = adapter.Read();
           // Wait for the client to send us the first packet
           if (frame != null) {

               // Write all packets to client
            ➍ foreach(var packet in config.Packets) {
                   adapter.Write(packet.Frame);
               }
           }
           frame = adapter.Read();
       }
   }

Listing 8-8: A simple server class for chat protocol

The code at ➊ is a configuration class that simply contains a log packet collection. We could have simplified the code by just specifying LogPacketCollection as the configuration type, but doing so with a distinct class demonstrates how you might add your own configuration more easily.

The code at ➋ defines the server class. It contains the Run() function, which takes a data adapter and the server configuration, and allows us to read and write to the data adapter after waiting for the client to send us a packet ➌. Once we’ve received a packet, we immediately send our entire packet list to the client ➍.

Note that we don’t filter the packets at ➍, and we don’t specify that we’re using any particular parser for the network traffic. In fact, this entire class is completely agnostic to the SuperFunkyChat protocol. We configure much of the behavior for the network server inside a template, as shown in Listing 8-9.

chapter8
_example
_server.csx

➊ #load "chat_server.csx"
   #load "parser.csx"
   using static System.Console;

   if (args.Length < 1) {
       WriteLine("Please Specify a Capture File");
       return;
   }
➋ var template = new NetServerTemplate<ChatServer, ChatServerConfig>();
   template.LocalPort = 12345;
   template.AddLayer<Parser>();
➌ var packets = LogPacketCollection.ReadFromFile(args[0])
                                    .GetPacketsForTag("In");
   template.ServerFactoryConfig.Packets.AddRange(packets);

➍ var service = template.Create();
   service.Start();
   WriteLine("Created {0}", service);
   WriteLine("Press Enter to exit...");
   ReadLine();
   service.Stop();

Listing 8-9: A simple example ChatServer

Listing 8-9 might look familiar because it’s very similar to the script we used for the DNS server in Listing 2-11. We begin by loading in the chat_server.csx script to define our ChatServer class ➊. Next, we create a server template at ➋ by specifying the type of the server and the configuration type. Then we load the packets from the file passed on the command line, filtering to capture only inbound packets and adding them to the packet collection in the configuration ➌. Finally, we create a service and start it ➍, just as we do proxies. The server is now listening for new connections on TCP port 12345.

Try the server with the ChatClient application; the captured traffic should be sent back to the client. After all the data has been sent to the client, the server will automatically close the connection. As long as you observe the message we re-sent, don’t worry if you see an error in the ChatClient’s output. Of course, you can add functionality to the server, such as modifying traffic or generating new packets.

Repurposing Existing Executable Code

In this section, we’ll explore various ways to repurpose existing binary executable code to reduce the amount of work involved in implementing a protocol. Once you’ve determined a protocol’s details by reverse engineering the executable (perhaps using some tips from Chapter 6), you’ll quickly realize that if you can reuse the executable code, you’ll avoid having to implement the protocol.

Ideally, you’ll have the source code you’ll need to implement a particular protocol, either because it’s open source or the implementation is in a scripting language like Python. If you do have the source code, you should be able to recompile or directly reuse the code in your own application. However, when the code has been compiled into a binary executable, your options can be more limited. We’ll look at each scenario now.

Managed language platforms, such as .NET and Java, are by far the easiest in which to reuse existing executable code, because they have a well-defined metadata structure in compiled code that allows a new application to be compiled against internal classes and methods. In contrast, in many unmanaged platforms, such as C/C++, the compiler will make no guarantees that any component inside a binary executable can be easily called externally.

Well-defined metadata also supports reflection, which is the ability of an application to support late binding of executable code to inspect data at runtime and to execute arbitrary methods. Although you can easily decompile many managed languages, it may not always be convenient to do so, especially when dealing with obfuscated applications. This is because the obfuscation can prevent reliable decompilation to usable source code.

Of course, the parts of the executable code you’ll need to execute will depend on the application you’re analyzing. In the sections that follow, I’ll detail some coding patterns and techniques to use to call the appropriate parts of the code in .NET and Java applications, the platforms you’re most likely to encounter.

Repurposing Code in .NET Applications

As discussed in Chapter 6, .NET applications are made up of one or more assemblies, which can be either an executable (with an .exe extension) or a library (.dll). When it comes to repurposing existing code, the form of the assembly doesn’t matter because we can call methods in both equally.

Whether we can just compile our code against the assembly’s code will depend on the visibility of the types we’re trying to use. The .NET platform supports different visibility scopes for types and members. The three most important forms of visibility scope are public, private, and internal. Public types or members are available to all callers outside the assembly. Private types or members are limited in scope to the current type (for example, you can have a private class inside a public class). Internal visibility scopes the types or members to only callers inside the same assembly, where they act as if they were public (although an external call cannot compile against them). For example, consider the C# code in Listing 8-10.

➊ public class PublicClass
   {
     private class PrivateClass
     {
    ➋ public PrivatePublicMethod() {}
     }
     internal class InternalClass
     {
    ➌ public void InternalPublicMethod() {}
     }
     private void PrivateMethod() {}
     internal void InternalMethod() {}
   ➍ public void PublicMethod() {}
   }

Listing 8-10: Examples of .NET visibility scopes

Listing 8-10 defines a total of three classes: one public, one private, and one internal. When you compile against the assembly containing these types, only PublicClass can be directly accessed along with the class’s PublicMethod() (indicated by ➊ and ➍); attempting to access any other type or member will generate an error in the compiler. But notice at ➋ and ➌ that public members are defined. Can’t we also access those members? Unfortunately, no, because these members are contained inside the scope of a PrivateClass or InternalClass. The class’s scope takes precedence over the members’ visibility.

Once you’ve determined whether all the types and members you want to use are public, you can add a reference to the assembly when compiling. If you’re using an IDE, you should find a method that allows you to add this reference to your project. But if you’re compiling on the command line using Mono or the Windows .NET framework, you’ll need to specify the -reference:<FILEPATH> option to the appropriate C# compiler, CSC or MCS.

Using the Reflection APIs

If all the types and members are not public, you’ll need to use the .NET framework’s Reflection APIs. You’ll find most of these in the System.Reflection namespace, except for the Type class, which is under the System namespace. Table 8-1 lists the most important classes with respect to reflection functionality.

Table 8-1: .NET Reflection Types

Class name	Description
System.Type	Represents a single type in an assembly and allows access to information about its members
System.Reflection.Assembly	Allows access to loading and inspecting an assembly as well as enumerating available types
System.Reflection.MethodInfo	Represents a method in a type
System.Reflection.FieldInfo	Represents a field in a type
System.Reflection.PropertyInfo	Represents a property in a type
System.Reflection.ConstructorInfo	Represents a class’s constructor

Loading the Assembly

Before you can do anything with the types and members, you’ll need to load the assembly using the Load() or the LoadFrom() method on the Assembly class. The Load() method takes an assembly name, which is an identifier for the assembly that assumes the assembly file can be found in the same location as the calling application. The LoadFrom() method takes the path to the assembly file.

For the sake of simplicity, we’ll use LoadFrom(), which you can use in most cases. Listing 8-11 shows a simple example of how you might load an assembly from a file and extract a type by name.

Assembly asm = Assembly.LoadFrom(@"c:path oassembly.exe");
Type type = asm.GetType("ChatProgram.Connection");

Listing 8-11: A simple assembly loading example

The name of the type is always the fully qualified name including its namespace. For example, in Listing 8-11, the name of the type being accessed is Connection inside the ChatProgram namespace. Each part of the type name is separated by periods.

How do you access classes that are declared inside other classes, such as those shown in Listing 8-10? In C#, you access these by specifying the parent class name and the child class name separated by periods. The framework is able to differentiate between ChatProgram.Connection, where we want the class Connection in namespace ChatProgram, and the child class Connection inside the class ChatProgram by using a plus (+) symbol: ChatProgram+Connection represents a parent/child class relationship.

Listing 8-12 shows a simple example of how we might create an instance of an internal class and call methods on it. We’ll assume that the class is already compiled into its own assembly.

internal class Connection
{
  internal Connection() {}

  public void Connect(string hostname)
  {
    Connect(hostname, 12345);
  }

  private void Connect(string hostname, int port)
  {
    // Implementation...
  }

  public void Send(byte[] packet)
  {
    // Implementation...
  }

  public void Send(string packet)
  {
    // Implementation...
  }

  public byte[] Receive()
  {
    // Implementation...
  }
}

Listing 8-12: A simple C# example class

The first step we need to take is to create an instance of this Connection class. We could do this by calling GetConstructor on the type and calling it manually, but sometimes there’s an easier way. One way would be to use the built-in System.Activator class to handle creating instances of types for us, at least in very simple scenarios. In such a scenario, we call the method CreateInstance(), which takes an instance of the type to create and a Boolean value that indicates whether the constructor is public or not. Because the constructor is not public (it’s internal), we need to pass true to get the activator to find the right constructor.

Listing 8-13 shows how to create a new instance, assuming a nonpublic parameterless constructor.

Type type = asm.GetType("ChatProgram.Connection");
object conn = Activator.CreateInstance(type, true);

Listing 8-13: Constructing a new instance of the Connection object

At this point, we would call the public Connect() method.

In the possible methods of the Type class, you’ll find the GetMethod() method, which just takes the name of the method to look up and returns an instance of a MethodInfo type. If the method cannot be found, null is returned. Listing 8-14 shows how to execute the method by calling the Invoke() method on MethodInfo, passing the instance of the object to execute it on and the parameters to pass to the method.

MethodInfo connect_method = type.GetMethod("Connect");
connect_method.Invoke(conn, new object[] { "host.badgers.com" });

Listing 8-14: Executing a method on a Connection object

The simplest form of GetMethod() takes as a parameter the name of the method to find, but it will look for only public methods. If instead you want to call the private Connect() method to be able to specify an arbitrary TCP port, use one of the various overloads of GetMethod(). These overloads take a BindingFlags enumeration value, which is a set of flags you can pass to reflection functions to determine what sort of information you want to look up. Table 8-2 shows some important flags.

Table 8-2: Important .NET Reflection Binding Flags

Flag name	Description
BindingFlags.Public	Look up public members
BindingFlags.NonPublic	Look up nonpublic members (internal or private)
BindingFlags.Instance	Look up members that can only be used on an instance of the class
BindingFlags.Static	Look up members that can be accessed statically without an instance

To get a MethodInfo for the private method, we can use the overload of GetMethod(), as shown in Listing 8-15, which takes a name and the binding flags. We’ll need to specify both NonPublic and Instance in the flags because we want a nonpublic method that can be called on instances of the type.

MethodInfo connect_method = type.GetMethod("Connect",
                                   BindingFlags.NonPublic | BindingFlags.Instance);
connect_method.Invoke(conn, new object[] { "host.badgers.com", 9999 });

Listing 8-15: Calling a nonpublic Connect() method

So far so good. Now we need to call the Send() method. Because this method is public, we should be able to call the basic GetMethod() method. But calling the basic method generates the exception shown in Listing 8-16, indicating an ambiguous match. What’s gone wrong?

System.Reflection.AmbiguousMatchException: Ambiguous match found.
   at System.RuntimeType.GetMethodImpl(...)

   at System.Type.GetMethod(String name)
   at Program.Main(String[] args)

Listing 8-16: An exception thrown for the Send() method

Notice in Listing 8-12 the Connection class has two Send() methods: one takes an array of bytes and the other takes a string. Because the reflection API doesn’t know which method you want, it doesn’t return a reference to either; instead, it just throws an exception. Contrast this with the Connect() method, which worked because the binding flags disambiguate the call. If you’re looking up a public method with the name Connect(), the reflection APIs will not even inspect the nonpublic overload.

We can get around this error by using yet another overload of GetMethod() that specifies exactly the types we want the method to support. We’ll choose the method that takes a string, as shown in Listing 8-17.

MethodInfo send_method = type.GetMethod("Send", new Type[] { typeof(string) });
send_method.Invoke(conn, new object[] { "data" });

Listing 8-17: Calling the Send(string) method

Finally, we can call the Receive() method. It’s public, so there are no additional overloads and it should be simple. Because Receive() takes no parameters, we can either pass an empty array or null to Invoke(). Because Invoke() returns an object, we need to cast the return value to a byte array to access the bytes directly. Listing 8-18 shows the final implementation.

MethodInfo recv_method = type.GetMethod("Receive");
byte[] packet = (byte[])recv_method.Invoke(conn, null);

Listing 8-18: Calling the Receive() method

Repurposing Code in Java Applications

Java is fairly similar to .NET, so I’ll just focus on the difference between them, which is that Java does not have the concept of an assembly. Instead, each class is represented by a separate .class file. Although you can combine class files into a Java Archive (JAR) file, it is just a convenience feature. For that reason, Java does not have internal classes that can only be accessed by other classes in the same assembly. However, Java does have a somewhat similar feature called package-private scoped classes, which can only be accessed by classes in the same package. (.NET refers to packages as a namespace.)

The upshot of this feature is that if you want to access classes marked as package scoped, you can write some Java code that defines itself in the same package, which can then access the package-scoped classes and members at will. For example, Listing 8-19 shows a package-private class that would be defined in the library you want to call and a simple bridge class you can compile into your own application to create an instance of the class.

// Package-private (PackageClass.java)
package com.example;

class PackageClass {
    PackageClass() {
    }

    PackageClass(String arg) {
    }


    @Override
    public String toString() {
        return "In Package";
    }
}

// Bridge class (BridgeClass.java)
package com.example;

public class BridgeClass {
    public static Object create() {
        return new PackageClass();
    }
}

Listing 8-19: Implementing a bridge class to access a package-private class

You specify the existing class or JAR files by adding their locations to the Java classpath, typically by specifying the -classpath parameter to the Java compiler or Java runtime executable.

If you need to call Java classes by reflection, the core Java reflection types are very similar to those described in the preceding .NET section: Type in .NET is class in Java, MethodInfo is Method, and so on. Table 8-3 contains a short list of Java reflection types.

Table 8-3: Java Reflection Types

Class name	Description
java.lang.Class	Represents a single class and allows access to its members
java.lang.reflect.Method	Represents a method in a type
java.lang.reflect.Field	Represents a field in a type
java.lang.reflect.Constructor	Represents a class’s constructor

You can access a class object by name by calling the Class.forName() method. For example, Listing 8-20 shows how we would get the PackageClass.

Class c = Class.forName("com.example.PackageClass");
System.out.println(c);

Listing 8-20: Getting a class in Java

If we want to create an instance of a public class with a parameterless constructor, the Class instance has a newInstance() method. This won’t work for our package-private class, so instead we’ll get an instance of the Constructor by calling getDeclaredConstructor() on the Class instance. We need to pass a list of Class objects to getDeclaredConstructor() to select the correct Constructor based on the types of parameters the constructor accepts. Listing 8-21 shows how we would choose the constructor, which takes a string, and then create a new instance.

   Constructor con = c.getDeclaredConstructor(String.class);
➊ con.setAccessible(true);
   Object obj = con.newInstance("Hello");

Listing 8-21: Creating a new instance from a private constructor

The code in Listing 8-21 should be fairly self-explanatory except perhaps for the line at ➊. In Java, any nonpublic member, whether a constructor, field, or method, must be set as accessible before you use it. If you don’t call setAccessible() with the value true, then calling newInstance() will throw an exception.

Unmanaged Executables

Calling arbitrary code in most unmanaged executables is much more difficult than in managed platforms. Although you can call a pointer to an internal function, there’s a reasonable chance that doing so could crash your application. However, you can reasonably call the unmanaged implementation when it’s explicitly exposed through a dynamic library. This section offers a brief overview of using the built-in Python library ctypes to call an unmanaged library on a Unix-like platform and Microsoft Windows.

Calling Dynamic Libraries

Linux, macOS, and Windows support dynamic libraries. Linux calls them object files (.so), macOS calls them dynamic libraries (.dylib), and Windows calls them dynamic link libraries (.dll). The Python ctypes library provides a mostly generic way to load all of these libraries into memory and a consistent syntax for defining how to call the exported function. Listing 8-22 shows a simple library written in C, which we’ll use as an example throughout the rest of the section.

#include <stdio.h>
#include <wchar.h>

void say_hello(void) {
  printf("Hello ");
}

void say_string(const char* str) {
  printf("%s ", str);
}

void say_unicode_string(const wchar_t* ustr) {
  printf("%ls ", ustr);
}

const char* get_hello(void) {
  return "Hello from C";
}

int add_numbers(int a, int b) {
  return a + b;
}

long add_longs(long a, long b) {
  return a + b;
}

void add_numbers_result(int a, int b, int* c) {
  *c = a + b;
}

struct SimpleStruct
{
  const char* str;
  int num;
};

void say_struct(const struct SimpleStruct* s) {
  printf("%s %d ", s->str, s->num);
}

Listing 8-22: The example C library lib.c

You can compile the code in Listing 8-22 into an appropriate dynamic library for the platform you’re testing. For example, on Linux you can compile the library by installing a C compiler, such as GCC, and executing the following command in the shell, which will generate a shared library lib.so:

gcc -shared -fPIC -o lib.so lib.c

Loading a Library with Python

Moving to Python, we can load our library using the ctypes.cdll.LoadLibrary() method, which returns an instance of a loaded library with the exported functions attached to the instance as named methods. For example, Listing 8-23 shows how to call the say_hello() method from the library compiled in Listing 8-22.

listing8-23.py

from ctypes import *

# On Linux
lib = cdll.LoadLibrary("./lib.so")
# On macOS
#lib = cdll.LoadLibrary("lib.dylib")
# On Windows
#lib = cdll.LoadLibrary("lib.dll")
# Or we can do the following on Windows
#lib = cdll.lib

lib.say_hello()
>>> Hello

Listing 8-23: A simple Python example for calling a dynamic library

Note that in order to load the library on Linux, you need to specify a path. Linux by default does not include the current directory in the library search order, so loading lib.so would fail. That is not the case on macOS or on Windows. On Windows, you can simply specify the name of the library after cdll and it will automatically add the .dll extension and load the library.

Let’s do some exploring. Load Listing 8-23 into a Python shell, for example, by running execfile("listing8-23.py"), and you’ll see that Hello is returned. Keep the interactive session open for the next section.

Calling More Complicated Functions

It’s easy enough to call a simple method, such as say_hello(), as in Listing 8-23. But in this section, we’ll look at how to call slightly more complicated functions including unmanaged functions, which take multiple different arguments.

Wherever possible, ctypes will attempt to determine what parameters are passed to the function automatically based on the parameters you pass in the Python script. Also, the library will always assume that the return type of a method is a C integer. For example, Listing 8-24 shows how to call the add_numbers() or say_string() methods along with the expected output from the interactive session.

print lib.add_numbers(1, 2)
>>> 3

lib.say_string("Hello from Python");
>>> Hello from Python

Listing 8-24: Calling simple methods

More complex methods require the use of ctypes data types to explicitly specify what types we want to use as defined in the ctypes namespace. Table 8-4 shows some of the more common data types.

Table 8-4: Python ctypes and Their Native C Type Equivalent

Python ctypes	Native C types
c_char, c_wchar	char, wchar_t
c_byte, c_ubyte	char, unsigned char
c_short, c_ushort	short, unsigned short
c_int, c_uint	int, unsigned int
c_long, c_ulong	long, unsigned long
c_longlong, c_ulonglong	long long, unsigned long long (typically 64 bit)
c_float, c_double	float, double
c_char_p, c_wchar_p	char*, wchar_t* (NUL terminated strings)
c_void_p	void* (generic pointer)

To specify the return type, we can assign a data type to the lib.name.restype property. For example, Listing 8-25 shows how to call get_hello(), which returns a pointer to a string.

# Before setting return type
print lib.get_hello()
>>> -1686370079

# After setting return type
lib.get_hello.restype = c_char_p
print lib.get_hello()
>>> Hello from C

Listing 8-25: Calling a method that returns a C string

If instead you want to specify the arguments to be passed to a method, you can set an array of data types to the argtypes property. For example, Listing 8-26 shows how to call add_longs() correctly.

# Before argtypes
lib.add_longs.restype = c_long
print lib.add_longs(0x100000000, 1)
>>> 1

# After argtypes
lib.add_longs.argtypes = [c_long, c_long]

print lib.add_longs(0x100000000, 1)
>>> 4294967297

Listing 8-26: Specifying argtypes for a method call

To pass a parameter via a pointer, use the byref helper. For example, add_numbers_result() returns the value as a pointer to an integer, as shown in Listing 8-27.

i = c_int()
lib.add_numbers_result(1, 2, byref(i))
print i.value
>>> 3

Listing 8-27: Calling a method with a reference parameter

Calling a Function with a Structure Parameter

We can define a structure for ctypes by creating a class derived from the Structure class and assigning the _fields_ property, and then pass the structure to the imported method. Listing 8-28 shows how to do this for the say_struct() function, which takes a pointer to a structure containing a string and a number.

class SimpleStruct(Structure):
  _fields_ = [("str", c_char_p),
              ("num", c_int)]

s = SimpleStruct()
s.str = "Hello from Struct"
s.num = 100
lib.say_struct(byref(s))
>>> Hello from Struct 100

Listing 8-28: Calling a method taking a structure

Calling Functions with Python on Microsoft Windows

In this section, information on calling unmanaged libraries on Windows is specific to 32-bit Windows. As discussed in Chapter 6, Windows API calls can specify a number of different calling conventions, the most common being stdcall and cdecl. By using cdll, all calls assume that the function is cdecl, but the property windll defaults instead to stdcall. If a DLL exports both cdecl and stdcall methods, you can mix calls through cdll and windll as necessary.

Encryption and Dealing with TLS

Encryption on network protocols can make it difficult for you to perform protocol analysis and reimplement the protocol to test for security issues. Fortunately, most applications don’t roll their own cryptography. Instead, they utilize a version of TLS, as described at the end of Chapter 7. Because TLS is a known quantity, we can often remove it from a protocol or reimplement it using standard tools and libraries.

Learning About the Encryption In Use

Perhaps unsurprisingly, SuperFunkyChat has support for a TLS endpoint, although you need to configure it by passing the path to a server certificate. The binary distribution of SuperFunkyChat comes with a server.pfx for this purpose. Restart the ChatServer application with the --server_cert parameter, as shown in Listing 8-29, and observe the output to ensure that TLS has been enabled.

$ ChatServer  --server_cert ChatServer/server.pfx
ChatServer (c) 2017 James Forshaw
WARNING: Don't use this for a real chat system!!!
Loaded certificate, Subject=CN=ExampleChatServer➊
Running server on port 12345 Global Bind False
Running TLS server on port 12346➋ Global Bind False

Listing 8-29: Running ChatServer with a TLS certificate

Two indications in the output of Listing 8-29 show that TLS has been enabled. First, the subject name of the server certificate is shown at ➊. Second, you can see that TLS server is listening on port 12346 ➋.

There’s no need to specify the port number when connecting the client using TLS with the --tls parameter: the client will automatically increment the port number to match. Listing 8-30 shows how when you add the --tls command line parameter to the client, it displays basic information about the connection to the console.

   $ ChatClient -–tls user1 127.0.0.1
   Connecting to 127.0.0.1:12346
➊ TLS Protocol: TLS v1.2
➋ TLS KeyEx   : RsaKeyX
➌ TLS Cipher  : Aes256
➍ TLS Hash    : Sha384
➎ Cert Subject: CN=ExampleChatServer
➏ Cert Issuer : CN=ExampleChatServer

Listing 8-30: A normal client connection

In this output, the TLS protocol in use is shown at ➊ as TLS 1.2. We can also see the key exchange ➋, cipher ➌, and hash algorithms ➍ negotiated. At ➎, we see some information about the server certificate, including the name of the Cert Subject, which typically represents the certificate owner. The Cert Issuer ➏ is the authority that signed the server’s certificate, and it’s the next certificate in the chain, as described in “Public Key Infrastructure” on page 169. In this case, the Cert Subject and Cert Issuer are the same, which typically means the certificate is self-signed.

Decrypting the TLS Traffic

A common technique to decrypt the TLS traffic is to actively use a man-in-the-middle attack on the network traffic so you can decrypt the TLS from the client and reencrypt it when sending it to the server. Of course, in the middle, you can manipulate and observe the traffic all you like. But aren’t man-in-the-middle attacks exactly what TLS is supposed to protect against? Yes, but as long as we control the client application sufficiently well, we can usually perform this attack for testing purposes.

Adding TLS support to a proxy (and therefore to servers and clients, as discussed earlier in this chapter) can be a simple matter of adding a single line or two to the proxy script to add a TLS decryption and encryption layer. Figure 8-1 shows a simple example of such a proxy.

Figure 8-1: An example MITM TLS proxy

We can implement the attack shown in Figure 8-1 by replacing the template initialization in Listing 8-5 with the code in Listing 8-31.

   var template = new FixedProxyTemplate();
   // Local port of 4445, destination 127.0.0.1:12346
➊ template.LocalPort = 4445;
   template.Host = "127.0.0.1";
   template.Port = 12346;

   var tls = new TlsNetworkLayerFactory();
➋ template.AddLayer(tls);
   template.AddLayer<Parser>();

Listing 8-31: Adding TLS support to capture a proxy

We make two important changes to the template initialization. At ➊, we increment port numbers because the client automatically adds 1 to the port when trying to connect over TLS. Then at ➋, we add a TLS network layer to the proxy template. (Be sure to add the TLS layer before the parser layer, or the parser layer will try to parse the TLS network traffic, which won’t work so well.)

With the proxy in place, let’s repeat our test with the client from Listing 8-31 to see the differences. Listing 8-32 shows the output.

   C:> ChatClient user1 127.0.0.1 --port 4444 -l
   Connecting to 127.0.0.1:4445
➊ TLS Protocol: TLS v1.0
➋ TLS KeyEx   : ECDH
   TLS Cipher  : Aes256
   TLS Hash    : Sha1
   Cert Subject: CN=ExampleChatServer
➌ Cert Issuer : CN=BrokenCA_PleaseFix

Listing 8-32: ChatClient connecting through a proxy

Notice some clear changes in Listing 8-32. One is that the TLS protocol is now TLS v1.0 ➊ instead of TLS v1.2. Another is that the Cipher and Hash algorithms differ from those in Listing 8-30, although the key exchange algorithm is using Elliptic Curve Diffie–Hellman (ECDH) for forward secrecy ➋. The final change is shown in the Cert Issuer ➌. The proxy libraries will autogenerate a valid certificate based on the original one from the server, but it will be signed with the library’s Certificate Authority (CA) certificate. If a CA certificate isn’t configured, one will be generated on first use.

Forcing TLS 1.2

The changes to the negotiated encryption settings shown in Listing 8-32 can interfere with your successfully proxying applications because some applications will check the version of TLS negotiated. If the client will only connect to a TLS 1.2 service, you can force that version by adding this line to the script:

tls.Config.ServerProtocol = System.Security.Authentication.SslProtocols.Tls12;

Replacing the Certificate with Our Own

Replacing the certificate chain involves ensuring that the client accepts the certificate that you generate as a valid root CA. Run the script in Listing 8-33 in CANAPE.Cli to generate a new CA certificate, output it and key to a PFX file, and output the public certificate in PEM format.

generate_ca
_cert.csx

using System.IO;

// Generate a 4096 bit RSA key with SHA512 hash
var ca = CertificateUtils.GenerateCACert("CN=MyTestCA",
    4096, CertificateHashAlgorithm.Sha512);
// Export to PFX with no password
File.WriteAllBytes("ca.pfx", ca.ExportToPFX());

// Export public certificate to a PEM file
File.WriteAllText("ca.crt", ca.ExportToPEM());

Listing 8-33: Generating a new root CA certificate for a proxy

On disk, you should now find a ca.pfx file and a ca.crt file. Copy the ca.pfx file into the same directory where your proxy script files are located, and add the following line before initializing the TLS layer as in Listing 8-31.

CertificateManager.SetRootCert("ca.pfx");

All generated certificates should now use your CA certificate as the root certificate.

You can now import ca.crt as a trusted root for your application. The method you use to import the certificate will depend on many factors, for example, the type of device the client application is running on (mobile devices are typically more difficult to compromise). Then there’s the question of where the application’s trusted root is stored. For example, is it in an application binary? I’ll show just one example of importing the certificate on Microsoft Windows.

Because it’s common for Windows applications to refer to the system trusted root store to get their root CAs, we can import our own certificate into this store and SuperFunkyChat will trust it. To do so, first run certmgr.msc either from the Run dialog or a command prompt. You should see the application window shown in Figure 8-2.

Figure 8-2: The Windows certificate manager

Choose Trusted Root Certification Authorities ▸ Certificates and then select Action ▸ All Tasks ▸ Import. An import Wizard should appear. Click Next and you should see a dialog similar to Figure 8-3.

Figure 8-3: Using the Certificate Import Wizard file import

Enter the path to ca.crt or browse to it and click Next again.

Next, make sure that Trusted Root Certification Authorities is shown in the Certificate Store box (see Figure 8-4) and click Next.

Figure 8-4: The certificate store location

On the final screen, click Finish; you should see the warning dialog box shown in Figure 8-5. Obviously, heed its warning, but click Yes all the same.

Figure 8-5: A warning about importing a root CA certificate

As long as your application uses the system root store, your TLS proxy connection will be trusted. We can test this quickly with SuperFunkyChat using --verify with the ChatClient to enable server certificate verification. Verification is off by default to allow you to use a self-signed certificate for the server. But when you run the client against the proxy server with --verify, the connection should fail, and you should see the following output:

SSL Policy Errors: RemoteCertificateNameMismatch
Error: The remote certificate is invalid according to the validation procedure.

The problem is that although we added the CA certificate as a trusted root, the server name, which is in many cases specified as the subject of the certificate, is invalid for the target. As we’re proxying the connection, the server hostname is, for example, 127.0.0.1, but the generated certificate is based on the original server’s certificate.

To fix this, add the following lines to specify the subject name for the generated certificate:

tls.Config.SpecifyServerCert = true;
tls.Config.ServerCertificateSubject = "CN=127.0.0.1";

When you retry the client, it should successfully connect to the proxy and then on to the real server, and all traffic should be unencrypted inside the proxy.

We can apply the same code changes to the network client and server code in Listing 8-6 and Listing 8-8. The framework will take care of ensuring that only specific TLS connections are established. (You can even specify TLS client certificates in the configuration for use in performing mutual authentication, but that’s an advanced topic that’s beyond the scope of this book.)

You should now have some ideas about how to man-in-the-middle TLS connections. The techniques you’ve learned will enable you to decrypt and encrypt the traffic from many applications to perform analysis and security testing.

Final Words

This chapter demonstrated some approaches you can take to reimplement your application protocol based on the results of either doing on-the-wire inspection or reverse engineering the implementation. I’ve only scratched the surface of this complex topic—many interesting challenges await you as you investigate security issues in network protocols.