20
NETWORK PROGRAMMING WITH BOOST ASIO

Anyone who has lost track of time when using a computer knows the propensity to dream, the urge to make dreams come true, and the tendency to miss lunch.
—Tim Berners-Lee

Boost Asio is a library for low-level I/O programming. In this chapter, you’ll learn about Boost Asio’s basic networking facilities, which enable programs to interact easily and efficiently with network resources. Unfortunately, the stdlib doesn’t contain a network-programming library as of C++17. For this reason, Boost Asio plays a central role in many C++ programs with a networking component.

Although Boost Asio is the primary choice for C++ developers who want to incorporate cross-platform, high-performance I/O into their programs, it’s a notoriously complicated library. This complication combined with an unfamiliarity with low-level network programming might be too overwhelming for newcomers. If you find this chapter obtuse or if you don’t need information on network programming, you can skip this chapter.

The Boost Asio Programming Model

In the Boost programming model, an I/O context object abstracts the operating system interfaces that handle asynchronous data processing. This object is a registry for I/O objects, which initiate asynchronous operations. Each object knows its corresponding service, and the context object mediates the connection.

Boost Asio defines a single service object, boost::asio::io_context. Its constructor takes an optional integer argument called the concurrency hint, which is the number of threads the io_context should allow to run concurrently. For example, on an eight-core machine, you might construct an io_context as follows:

boost::asio::io_context io_context{ 8 };

You’ll pass the same io_context object into the constructors of your I/O objects. Once you’ve set up all your I/O objects, you’ll call the run method on the io_context, which will block until all pending I/O operations complete.

One of the simplest I/O objects is the boost::asio::steady_timer, which you can use to schedule tasks. Its constructor accepts an io_context object and an optional std::chrono::time_point or std::chrono_duration. For example, the following constructs a steady_timer that expires in three seconds:

boost::asio::steady_timer timer{
  io_context, std::chrono::steady_clock::now() + std::chrono::seconds{ 3 }
};

You can wait on the timer with a blocking or a non-blocking call. To block the current thread, you use the timer’s wait method. The result is essentially similar to using std::this_thread::sleep_for, which you learned about in “Chrono” on page 387. To wait asynchronously, you use the timer’s async_wait method. This accepts a function object referred to as a callback. The operating system will invoke the function object once it’s time for the thread to wake up. Due to complications arising from modern operating systems, this might or might not be due to the timer’s expiring.

Once a timer expires, you can create another timer if you want to perform an additional wait. If you wait on an expired timer, it will return immediately. This is probably not what you intend to do, so make sure you wait only on unexpired timers.

To check whether the timer has expired, the function object must accept a boost::system::error_code. The error_code class is a simple class that represents operating system–specific errors. It converts implicitly to bool (true if it represents an error condition; false otherwise). If the callback’s error_code evaluates to false, the timer expired.

Once you enqueue an asynchronous operation using async_wait, you’ll call the run method on your io_context object because this method blocks until all asynchronous operations are complete.

Listing 20-1 illustrates how to construct and use timers for blocking and non-blocking waits.

#include <iostream>
#include <boost/asio.hpp>
#include <chrono>

boost::asio::steady_timer make_timer(boost::asio::io_context& io_context) { ➊
  return boost::asio::steady_timer{
          io_context,
          std::chrono::steady_clock::now() + std::chrono::seconds{ 3 }
  };
}

int main() {
  boost::asio::io_context io_context; ➋

  auto timer1 = make_timer(io_context); ➌
  std::cout << "entering steady_timer::wait
";
  timer1.wait(); ➍
  std::cout << "exited steady_timer::wait
";

  auto timer2 = make_timer(io_context); ➎
  std::cout << "entering steady_timer::async_wait
";
  timer2.async_wait([] (const boost::system::error_code& error) { ➏
    if (!error) std::cout << "<<callback function>>
";
  });
  std::cout << "exited steady_timer::async_wait
";
  std::cout << "entering io_context::run
";
  io_context.run(); ➐
  std::cout << "exited io_context::run
";
}
-----------------------------------------------------------------------
entering steady_timer::wait
exited steady_timer::wait
entering steady_timer::async_wait
exited steady_timer::async_wait
entering io_context::run
<<callback function>>
exited io_context::run

Listing 20-1: A program using boost::asio::steady_timer for synchronous and asynchronous waiting

You define the make_timer function for building a steady_timer that expires in three seconds ➊. Within main, you initialize your program’s io_context ➋ and construct your first timer from make_timer ➌. When you call wait on this timer ➍, the thread blocks for three seconds before proceeding. Next, you construct another timer with make_timer ➎, and then you invoke async_wait with a lambda that prints <<callback_function>> when the timer expires ➏. Finally, you invoke run on your io_context to begin processing operations ➐.

Network Programming with Asio

Boost Asio contains facilities for performing network-based I/O over several important network protocols. Now that you know the basic usage of io_context and how to enqueue asynchronous I/O operations, you can explore how to perform more involved kinds of I/O. In this section, you’ll extend what you learned about waiting for timers and employ Boost Asio’s network I/O facilities. By the end of this chapter, you’ll know how to build programs that communicate over a network.

The Internet Protocol Suite

The Internet Protocol (IP) is the primary protocol for ferrying data across networks. Each participant in an IP network is called a host, and each host gets an IP address to identify it. IP addresses come in two versions: IPv4 and IPv6. An IPv4 address is 32 bits, and an IPv6 address is 128 bits.

The Internet Control Message Protocol (ICMP) is used by network devices to send information that supports operation of an IP network. The ping and traceroute programs use ICMP messages to query a network. Typically, end user applications don’t need to interface with ICMP directly.

To send data across an IP network, you typically use either the Transmission Control Protocol (TCP) or User Datagram Protocol (UDP). In general, you use TCP when you need to be sure that data arrives at its destination, and you use UDP when you need to be sure that data transits quickly. TCP is a connection-oriented protocol where receivers acknowledge that they’ve received messages intended for them. UDP is a simple, connectionless protocol that has no built-in reliability.

With TCP and UDP, network devices connect to each other using ports. A port is an integer ranging from 0 to 65,535 (2 bytes) that specifies a particular service running on a given network device. This way, a single device can run multiple services and each can be addressed separately. When one device, called a client, initiates communication with another device, called a server, the client specifies which port it wants to connect to. When you pair a device’s IP address with a port number, the result is called a socket.

For example, a device with IP address 10.10.10.100 could serve a web page by binding a web server application to port 80. This creates a server socket at 10.10.10.100:80. Next, a device with IP address 10.10.10.200 launches a web browser, which opens a “random high port,” such as 55123. This creates a client socket at 10.10.10.200:55123. The client then connects to the server by creating a TCP connection between the client socket and the server socket. Many other processes could be running on either or both devices with many other network connections simultaneously.

The Internet Assigned Numbers Authority (IANA) maintains a list of assigned numbers to standardize the ports that certain kinds of services use (the list is available at https://www.iana.org/). Table 20-1 provides a few commonly used protocols on this list.

Table 20-1: Well-Known Protocols Assigned by IANA

Port	TCP	UDP	Keyword	Description
7	✓	✓	echo	Echo Protocol
13	✓	✓	daytime	Daytime Protocol
21	✓		ftp	File Transfer Protocol
22	✓		ssh	Secure Shell Protocol
23	✓		telnet	Telnet Protocol
25	✓		smtp	Simple Mail Transfer Protocol
53	✓	✓	domain	Domain Name System
80	✓		http	Hypertext Transfer Protocol
110	✓		pop3	Post Office Protocol
123		✓	ntp	Network Time Protocol
143	✓		imap	Internet Message Access Protocol
179	✓		bgp	Border Gateway Protocol
194	✓		irc	Internet Relay Chat
443	✓		https	Hypertext Transfer Protocol (Secure)

Boost Asio supports network I/O over ICMP, TCP, and UDP. For brevity, this chapter only discusses TCP because the Asio classes involved in all three protocols are so similar.

Hostname Resolution

When a client wants to connect to a server, it needs the server’s IP address. In some scenarios, the client might already have this information. In others, the client might have only a service name. The process of converting a service name to an IP address is called hostname resolution. Boost Asio contains the boost::asio::ip::tcp::resolver class to perform hostname resolution. To construct a resolver, you pass an io_context instance as the only constructor parameter, as in the following:

boost::asio::ip::tcp::resolver my_resolver{ my_io_context };

To perform hostname resolution, you use the resolve method, which accepts at least two string_view arguments: the hostname and the service. You can provide either a keyword or a port number for service (refer to Table 20-1 for some example keywords). The resolve method returns a range of boost::asio::ip::tcp::resolver::basic_resolver_entry objects, which expose several useful methods:

• endpoint gets the IP address and port.
• host_name gets the hostname.
• service_name gets the name of the service associated with this port.

If the resolution fails, resolve throws a boost::system::system_error. Alternatively, you can pass a boost::system::error_code reference, which receives the error in lieu of throwing an exception. For example, Listing 20-2 determines the IP address and port for the No Starch Press web server using Boost Asio.

#include <iostream>
#include <boost/asio.hpp>

int main() {
  boost::asio::io_context io_context; ➊
  boost::asio::ip::tcp::resolver resolver{ io_context }; ➋
  boost::system::error_code ec;
  for(auto&& result : resolver.resolve("www.nostarch.com", "http", ec)) { ➌
    std::cout << result.service_name() << " " ➍
              << result.host_name() << " " ➎
              << result.endpoint() ➏
              << std::endl;
  }
  if(ec) std::cout << "Error code: " << ec << std::endl; ➐
}
-----------------------------------------------------------------------
http www.nostarch.com 104.20.209.3:80
http www.nostarch.com 104.20.208.3:80

Listing 20-2: Blocking hostname resolution with Boost Asio

You initialize an io_context ➊ and a boost::asio::ip::tcp::resolver ➋. Within a range-based for loop, you iterate over each result ➌ and extract the service_name ➍, the host_name ➎, and the endpoint ➏. If resolve encounters an error, you print it to stdout ➐.

You can perform asynchronous hostname resolution using the async_resolve method. As with resolve, you pass a hostname and a service as the first two arguments. Additionally, you provide a callback function object that accepts two arguments: a system_error_code and a range of basic_resolver_entry objects. Listing 20-3 illustrates how to refactor Listing 20-2 to use asynchronous hostname resolution instead.

#include <iostream>
#include <boost/asio.hpp>

int main() {
  boost::asio::io_context io_context;
  boost::asio::ip::tcp::resolver resolver{ io_context };
  resolver.async_resolve("www.nostarch.com", "http", ➊
    [](boost::system::error_code ec, const auto& results) { ➋
      if (ec) { ➌
        std::cerr << "Error:" << ec << std::endl;
        return; ➍
      }
      for (auto&& result : results) { ➎
        std::cout << result.service_name() << " "
                  << result.host_name() << " "
                  << result.endpoint() << " "
                  << std::endl; ➏
      }
    }
  );
  io_context.run(); ➐
}
-----------------------------------------------------------------------
http www.nostarch.com 104.20.209.3:80
http www.nostarch.com 104.20.208.3:80

Listing 20-3: Refactoring Listing 20-2 to use async_resolve

The setup is identical to Listing 20-2 until you invoke async_resolve on your resolver ➊. You pass the same hostname and service as before, but you add a callback argument that accepts the obligatory parameters ➋. Within the body of the callback lambda, you check for an error condition ➌. If one exists, you print a friendly error message and return ➍. In the error-free case, you iterate over the results as before ➎, printing the service_name, host_name, and endpoint ➏. As with the timer, you need to invoke run on the io_context to give the asynchronous operations the opportunity to complete ➐.

Connecting

Once you’ve obtained a range of endpoints either through hostname resolution or through constructing one on your own, you’re ready to make a connection.

First, you’ll need a boost::asio::ip::tcp::socket, a class that abstracts the underlying operating system’s socket and presents it for use in Asio. The socket takes an io_context as an argument.

Second, you’ll need to make a call to the boost::asio::connect function, which accepts a socket representing the endpoint you want to connect with as its first argument and an endpoint range as its second argument. You can provide an error_code reference as an optional third argument; otherwise, connect will throw a system_error exception if an error occurs. If successful, connect returns a single endpoint, the endpoint in the input range to which it successfully connected. After this point, the socket object represents a real socket in your system’s environment.

Listing 20-4 illustrates how to connect to No Starch Press’s web server.

#include <iostream>
#include <boost/asio.hpp>

int main() {
  boost::asio::io_context io_context;
  boost::asio::ip::tcp::resolver resolver{ io_context }; ➊
  boost::asio::ip::tcp::socket socket{ io_context }; ➊
  try  {
    auto endpoints = resolver.resolve("www.nostarch.com", "http"); ➌
    const auto connected_endpoint = boost::asio::connect(socket, endpoints); ➍
    std::cout << connected_endpoint; ➎
  } catch(boost::system::system_error& se) {
    std::cerr << "Error: " << se.what() << std::endl; ➏
  }
}
-----------------------------------------------------------------------
104.20.209.3:80 ➎

Listing 20-4: Connecting to the No Starch web server

You construct a resolver ➊ as in Listing 20-3. In addition, you initialize a socket with the same io_context ➋. Next, you invoke the resolve method to obtain every endpoint associated with www.nostarch.com at port 80 ➌. Recall that each endpoint is an IP address and a port corresponding to the host you resolved. In this case, resolve used the domain name system to determine that www.nostarch.com at port 80 resides at the IP address 104.20.209.3. You then invoke connect using your socket and endpoints ➍, which returns the endpoint to which connect successfully connected ➎. In the event of an error, resolve or connect would throw an exception, which you would catch and print to stderr ➏.

You can also connect asynchronously with boost::asio::async_connect, which accepts the same two arguments as connect: a socket and an endpoint range. The third argument is a function object acting as the callback, which must accept an error_code as its first argument and an endpoint as its second argument. Listing 20-5 illustrates how to connect asynchronously.

#include <iostream>
#include <boost/asio.hpp>

int main() {
  boost::asio::io_context io_context;
  boost::asio::ip::tcp::resolver resolver{ io_context };
  boost::asio::ip::tcp::socket socket{ io_context };
  boost::asio::async_connect(socket, ➊
    resolver.resolve("www.nostarch.com", "http"), ➋
    [] (boost::system::error_code ec, const auto& endpoint){ ➌
      std::cout << endpoint; ➍
  });
  io_context.run(); ➎
}
-----------------------------------------------------------------------
104.20.209.3:80 ➍

Listing 20-5: Connecting to the No Starch web server asynchronously

The setup is exactly as in Listing 20-4 except you replace connect with async_connect and pass the same first ➊ and second ➋ arguments. The third argument is your callback function object ➌ inside of which you print the endpoint to stdout ➍. As with all asynchronous Asio programs, you make a call to run on your io_context ➎.

Buffers

Boost Asio provides several buffer classes. A buffer (or data buffer) is memory that stores transient data. The Boost Asio buffer classes form the interface for all I/O operations. Before you can do anything with the network connections you make, you’ll need an interface for reading and writing data. For this, you’ll need just three buffer types:

• boost::asio::const_buffer holds a buffer that cannot be modified once you’ve constructed it.
• boost::asio::mutable_buffer holds a buffer that can be modified after construction.
• boost::asio::streambuf holds an automatically resizable buffer based on std::streambuf.

All three buffer classes provide two important methods for accessing their underlying data: data and size.

The mutable_buffer and const_buffer classes’ data methods return a pointer to the first element in the underlying data sequence, and their size methods return the number of elements in that sequence. The elements are contiguous. Both buffers provide default constructors, which initialize an empty buffer, as Listing 20-6 illustrates.

#include <boost/asio.hpp>

TEST_CASE("const_buffer default constructor") {
  boost::asio::const_buffer cb; ➊
  REQUIRE(cb.size() == 0); ➋
}

TEST_CASE("mutable_buffer default constructor") {
  boost::asio::mutable_buffer mb; ➌
  REQUIRE(mb.size() == 0); ➍
}

Listing 20-6: Default constructing const_buffer and mutable_buffer yields empty buffers.

Using the default constructors ➊➌, you build empty buffers that have zero size ➋➍.

Both mutable_buffer and const_buffer provide constructors that accept a void* and a size_t corresponding to the data you want to wrap. Note that these constructors don’t take ownership of the pointed-to memory, so you must ensure that the storage duration of that memory is at least as long as the lifetime of the buffer you’re constructing. This is a design decision that gives you, as the Boost Asio user, maximum flexibility. Unfortunately, it also leads to potentially nasty errors. Failure to properly manage the lifetimes of buffers and the objects they point to will result in undefined behavior.

Listing 20-7 illustrates how to construct buffers using the pointer-based constructor.

#include <boost/asio.hpp>
#include <string>

TEST_CASE("const_buffer constructor") {
  boost::asio::const_buffer cb{ "Blessed are the cheesemakers.", 7 }; ➊

  REQUIRE(cb.size() == 7); ➋
  REQUIRE(*static_cast<const char*>(cb.data()) == 'B'); ➌
}

TEST_CASE("mutable_buffer constructor") {
  std::string proposition{ "Charity for an ex-leper?" };
  boost::asio::mutable_buffer mb{ proposition.data(), proposition.size() }; ➍

  REQUIRE(mb.data() == proposition.data()); ➎
  REQUIRE(mb.size() == proposition.size()); ➏
}

Listing 20-7: Constructing a const_buffer and a mutable_buffer using the pointer-based constructor

In the first test, you construct a const_buffer using a C-style string and a fixed length of 7 ➊. This fixed length is smaller than the length of the string literal Blessed are the cheesemakers., so this buffer refers to Blessed rather than the entire string. This illustrates that you can select a subset of an array (just as with std::string_view, which you learned about in “String View” on page 500). The resulting buffer has size 7 ➋, and if you cast the pointer from data to a const char*, you’ll see that it points to the character B from your C-style string ➌.

In the second test, you construct a mutable_buffer using a string by invoking its data and size members within the buffer’s constructor ➍. The resulting buffer’s data ➎ and size ➏ methods return identical data to your original string.

The boost::asio::streambuf class accepts two optional constructor arguments: a size_t maximum size and an allocator. By default, the maximum size is std::numeric_limits<std::size_t> and the allocator is similar to the default allocator for stdlib containers. The streambuf input sequence’s initial size is always zero, which Listing 20-8 illustrates.

#include <boost/asio.hpp>

TEST_CASE("streambuf constructor") {
  boost::asio::streambuf sb; ➊
  REQUIRE(sb.size() == 0); ➋
}

Listing 20-8: Default constructing a streambuf

You default construct a streambuf ➊, and when you invoke its size method, it returns 0 ➋.

You can pass a pointer to a streambuf into a std::istream or std::ostream constructor. Recall from “Stream Classes” on page 524 that these are specializations of basic_istream and basic_ostream that expose stream operations to an underlying sync or source. Listing 20-9 illustrates how to write into and subsequently read from a streambuf using these classes.

TEST_CASE("streambuf input/output") {
  boost::asio::streambuf sb; ➊
  std::ostream os{ &sb }; ➋
  os << "Welease Wodger!"; ➌

  std::istream is{ &sb }; ➍
  std::string command; ➎
  is >> command; ➏

  REQUIRE(command == "Welease"); ➐
}

Listing 20-9: Writing to and reading from a streambuf

You again construct an empty streambuf ➊, and you pass its address into the constructor of an ostream ➋. You then write the string Welease Wodger! into the ostream, which in turn writes the string into the underlying streambuf ➌.

Next, you create an istream again using the address of the streambuf ➍. You then create a string ➎ and write the istream into the string ➏. Recall from “Special Formatting for Fundamental Types” on page 529 that this operation will skip any leading whitespace and then read the following string until the next whitespace. This yields the first word of the string, Welease ➐.

Boost Asio also offers the convenience function template boost::asio::buffer, which accepts a std::array or std::vector of POD elements or a std::string. For example, you can create the std::string backed mutable_buffer in Listing 20-7 using the following construction instead:

std::string proposition{ "Charity for an ex-leper?" };
auto mb = boost::asio::buffer(proposition);

The buffer template is specialized so if you provide a const argument, it will return a const_buffer instead. In other words, to make a const_buffer out of proposition, simply make it const:

const std::string proposition{ "Charity for an ex-leper?" };
auto cb = boost::asio::buffer(proposition);

You’ve now created a const_buffer cb.

Additionally, you can create a dynamic buffer, which is a dynamically resizable buffer backed by a std::string or a std::vector. You can create one by using the boost::asio::dynamic_buffer function template, which accepts either a string or a vector and returns a boost::asio::dynamic_string_buffer or boost::asio::dynamic_vector_buffer as appropriate. For example, you can make a dynamic buffer using the following construction:

std::string proposition{ "Charity for an ex-leper?" };
auto db = boost::asio::dynamic_buffer(proposition);

Although a dynamic buffer is dynamically resizable, recall that the vector and string classes use an allocator and that allocation can be a relatively slow operation. So, if you know how much data you’ll write into a buffer, you might have better performance using a non-dynamic buffer. As always, measuring and experimenting will help you decide which approach to take.

Reading and Writing Data with Buffers

With your new knowledge of how to store and retrieve data using buffers, you can learn how to pull data off a socket. You can read data from active socket objects into buffer objects using built-in Boost Asio functions. For blocking reads, Boost Asio offers three functions:

• boost::asio::read attempts to read a fixed-size data chunk.
• boost::asio::read_at attempts to read a fixed-size data chunk beginning at an offset.
• boost::asio::read_until attempts to read until a delimiter, regular expression, or arbitrary predicate matches.

All three methods take a socket as their first argument and a buffer object as their second argument. The remaining arguments are optional and depend on which function you’re using:

• A completion condition is a function object that accepts an error_code and a size_t argument. The error_code will be set if the Asio function encountered an error, and the size_t argument corresponds with the number of bytes transferred so far. The function object returns a size_t corresponding to the number of bytes remaining to be transferred, and it returns 0 if the operation is complete.
• A match condition is a function object that accepts a range specified by a begin and end iterator. It must return a std::pair, where the first element is an iterator indicating the starting point for the next attempt at matching and the second element is a bool representing whether the range contains a match.
• boost::system::error_code reference, which the function will set if it encounters an error condition.

Table 20-2 lists many of the ways you can invoke one of the read functions.

Table 20-2: Arguments for read, read_at, and read_until

Invocation	Description
read(s, b, [cmp], [ec])	Reads a certain amount of data from socket s into a mutable buffer b according to completion condition cmp. Sets the error_code ec if an error condition is encountered; otherwise, throws a system_error.
read_at(s, off, b, [cmp], [ec])	Reads a certain amount of data starting from socket s, starting from size_t offset off, into a mutable buffer b according to completion condition cmp. Sets the error_code ec if an error condition is encountered; otherwise, throws a system_error.
read_until(s, b, x, [ec])	Reads data from socket s into a mutable buffer b until it meets a condition represented by x, which can be one of the following: a char, a string_view, a boost::regex, or a match condition. Sets the error_code ec if an error condition is encountered; otherwise, throws a system_error.

You can also write data to an active socket object from a buffer. For blocking writes, Boost Asio offers two functions:

• boost::asio::write attempts to write a fixed-size data chunk.
• boost::asio::write_at attempts to write a fixed-size data chunk beginning at an offset.

Table 20-3 shows how to invoke these two methods. Their arguments are analogous to those for the reading methods.

Table 20-3: Arguments for write and write_at

Invocation	Description
write(s, b, [cmp], [ec])	Writes a certain amount of data into socket s from a const buffer b according to completion condition cmp. Sets the error_code ec if an error condition is encountered; otherwise, throws a system_error.
write_at(s, off, b, [cmp], [ec])	Writes a certain amount of data from const buffer b, starting from size_t offset off, into socket s according to completion condition cmp. Sets the error_code ec if an error condition is encountered; otherwise, throws a system_error.

The Hypertext Transfer Protocol (HTTP)

HTTP is the 30-year-old protocol undergirding the web. Although it’s a very complicated protocol to use to introduce networking, its ubiquity makes it one of the most relevant choices. In the next section, you’ll use Boost Asio to make very simple HTTP requests. It’s not strictly necessary that you have a solid foundation in HTTP, so you can skip this section on first reading. However, the information here adds some color to the examples in the next section and provides references for further study.

HTTP sessions have two parties: a client and a server. An HTTP client sends a plaintext request over TCP containing one or more lines separated by a carriage return and a line feed (a “CR-LF newline”).

The first line is the request line, which contains three tokens: an HTTP method, a uniform resource locator (URL), and the HTTP version of the request. For example, if a client wants a file called index.htm, the status line might be GET /index.htm HTTP/1.1.

Directly following the request line are one or more headers, which define the parameters of an HTTP transaction. Each header contains a key and a value. The key must be composed of alphanumeric characters and dashes. A colon plus a space delimits the key from the value. A CR-LF newline terminates the header. The following headers are especially common in requests:

• Host specifies the domain of the service requested. Optionally, you can include a port. For example, Host: www.google.com specifies www.google.com as the host for the requested service.
• Accept specifies the acceptable media types in MIME format for the response. For example, Accept: text/plain specifies that the requester can process plaintext.
• Accept-Language specifies the acceptable human languages for the response. For example, Accept-Language: en-US specifies that the requester can process American English.
• Accept-Encoding specifies the acceptable encodings for the response. For example, Accept-Encoding: identity specifies that the requester can process contents without any encoding.
• Connection specifies control options for the current connection. For example, Connection: close specifies that the connection will be closed after completion of the response.

You terminate the headers with an additional CR-LF newline. For certain kinds of HTTP requests, you’ll also include a body following the headers. If you do, you’ll also include Content-Length and Content-Type headers. The Content-Length value specifies the length of the request body in bytes, and the Content-Type value specifies the MIME format of the body.

An HTTP response’s first line is the status line, which includes the HTTP version of the response, a status code, and a reason message. For example, the status line HTTP/1.1 200 OK indicates a successful (“OK”) request. Status codes are always three digits. The leading digit indicates the status group of the code:

1** (Informational) The request was received.

2** (Successful) The request was received and accepted.

3** (Redirection) Further action is required.

4** (Client Error) The request was bad.

5** (Server Error) The request seems okay, but the server encountered an internal error.

After the status line, the response contains any number of headers in the same format as the response. Many of the same request headers are also common response headers. For example, if the HTTP response contains a body, the response headers will include Content-Length and Content-Type.

If you need to program HTTP applications, you should absolutely refer to the Boost Beast library, which provides high-performance, low-level HTTP and WebSockets facilities. It’s built atop Asio and works seamlessly with it.

Implementing a Simple Boost Asio HTTP Client

In this section, you’ll implement a (very) simple HTTP client. You’ll build an HTTP request, resolve an endpoint, connect to a web server, write the request, and read the response. Listing 20-10 illustrates one possible implementation.

#include <boost/asio.hpp>
#include <iostream>
#include <istream>
#include <ostream>
#include <string>

std::string request(std::string host, boost::asio::io_context& io_context) { ➊
  std::stringstream request_stream;
  request_stream << "GET / HTTP/1.1
"
                    "Host: " << host << "
"
                    "Accept: text/html
"
                    "Accept-Language: en-us
"
                    "Accept-Encoding: identity
"
                    "Connection: close

";
  const auto request = request_stream.str(); ➋
  boost::asio::ip::tcp::resolver resolver{ io_context };
  const auto endpoints = resolver.resolve(host, "http"); ➌
  boost::asio::ip::tcp::socket socket{ io_context };
  const auto connected_endpoint = boost::asio::connect(socket, endpoints); ➍
  boost::asio::write(socket, boost::asio::buffer(request)); ➎
  std::string response;
  boost::system::error_code ec;
  boost::asio::read(socket, boost::asio::dynamic_buffer(response), ec); ➏
  if (ec && ec.value() != 2) throw boost::system::system_error{ ec }; ➐
  return response;
}

int main() {
  boost::asio::io_context io_context;
  try  {
    const auto response = request("www.arcyber.army.mil", io_context); ➑
    std::cout << response << "
"; ➒
  } catch(boost::system::system_error& se) {
    std::cerr << "Error: " << se.what() << std::endl;
  }
}
-----------------------------------------------------------------------
HTTP/1.1 200 OK
Pragma: no-cache
Content-Type: text/html; charset=utf-8
X-UA-Compatible: IE=edge
pw_value: 3ce3af822980b849665e8c5400e1b45b
Access-Control-Allow-Origin: *
X-Powered-By:
Server:
X-ASPNET-VERSION:
X-FRAME-OPTIONS: SAMEORIGIN
Content-Length: 76199
Cache-Control: private, no-cache
Expires: Mon, 22 Oct 2018 14:21:09 GMT
Date: Mon, 22 Oct 2018 14:21:09 GMT
Connection: close
<!DOCTYPE html>
<html  lang="en-US">
<head id="Head">
--snip--
</body>
</html>

Listing 20-10: Completing a simple request to the United States Army Cyber Command web server

You first define a request function, which accepts a host and an io_context and returns an HTTP response ➊. First, you use a std::stringstream to build a std::string containing an HTTP request ➋. Next, you resolve the host using a boost::asio::ip::tcp::resolver ➌ and connect a boost::asio::ip::tcp::socket to the resulting endpoint range ➍. (This matches the approach in Listing 20-4.)

Then you write your HTTP request to the server you’ve connected to. You use boost::asio::write, passing in your connected socket and your request. Because write accepts Asio buffers, you use boost::asio::buffer to create a mutable_buffer from your request (which is a std::string) ➎.

Next, you read the HTTP response from the server. Because you don’t know the length of the response in advance, you create a std::string called response to receive the response. Eventually, you’ll use this to back a dynamic buffer. For simplicity, the HTTP request contains a Connection: close header that causes the server to terminate the connection immediately after it sends its response. This will result in Asio returning an “end of file” error code (value 2). Because you expect this behavior, you declare a boost::system::error_code to receive this error.

Next, you invoke boost::asio::read with the connected socket, a dynamic buffer that will receive the response, and the error_condition ➏. You use boost::asio_dynamic_buffer to construct your dynamic buffer from response. Immediately after read returns, you check for an error_condition other than end of file (which you throw) ➐. Otherwise, you return the response.

Within main, you invoke your request function with the www.arcyber.army.mil host and an io_context object ➑. Finally, you print the response to stdout ➒.

Asynchronous Reading and Writing

You can also read and write asynchronously with Boost Asio. The corresponding asynchronous functions are analogous to their blocking corollaries. For asynchronous reads, Boost Asio offers three functions:

• boost::asio::async_read attempts to read a fixed-size data chunk.
• boost::asio::async_read_at attempts to read a fixed-size data chunk beginning at an offset.
• boost::asio::async_read_until attempts to read until a delimiter, regular expression, or arbitrary predicate matches.

Boost Asio also offers two asynchronous write functions:

• boost::asio::async_write attempts to write a fixed-size data chunk.
• boost::asio::async_write_at attempts to write a fixed-size data chunk beginning at an offset.

All five of these asynchronous functions accept the same arguments as their blocking counterparts, except their final argument is always a callback function object that accepts two arguments: a boost::system::error_code indicating whether the function met an error and a size_t indicating the number of bytes it transferred. For the asynchronous write functions, you need to determine whether Asio wrote the entire payload. Because these calls are asynchronous, your thread doesn’t block while it’s waiting for I/O to complete. Instead, the operating system calls your thread back whenever a portion of your I/O request completes.

Because the callback’s second argument is a size_t corresponding to the number of transferred bytes, you can do the arithmetic to figure out whether you have anything left to write. If there is, you must invoke another asynchronous write function by passing the remaining data.

Listing 20-11 contains an asynchronous version of the simple web client in Listing 20-10. Note that using the asynchronous functions is a bit more complicated. But there’s a pattern with callbacks and handlers that’s consistent across the request’s lifetime.

#include <boost/asio.hpp>
#include <iostream>
#include <string>
#include <sstream>

using ResolveResult = boost::asio::ip::tcp::resolver::results_type;
using Endpoint = boost::asio::ip::tcp::endpoint;

struct Request {
  explicit Request(boost::asio::io_context& io_context, std::string host)
      : resolver{ io_context },
        socket{ io_context },
        host{ std::move(host) } { ➊
    std::stringstream request_stream;
    request_stream << "GET / HTTP/1.1
"
                      "Host: " << this->host << "
"
                      "Accept: text/plain
"
                      "Accept-Language: en-us
"
                      "Accept-Encoding: identity
"
                      "Connection: close
"
                      "User-Agent: C++ Crash Course Client

";
    request = request_stream.str(); ➋
    resolver.async_resolve(this->host, "http",
       [this] (boost::system::error_code ec, const ResolveResult& results) {
         resolution_handler(ec, results); ➌
       });
  }
 void resolution_handler(boost::system::error_code ec,
                          const ResolveResult& results) {
    if (ec) { ➍
      std::cerr << "Error resolving " << host << ": " << ec << std::endl;
      return;
    }
    boost::asio::async_connect(socket, results,
            [this] (boost::system::error_code ec, const Endpoint& endpoint){
              connection_handler(ec, endpoint); ➎
            });
  }

  void connection_handler(boost::system::error_code ec,
                          const Endpoint& endpoint) { ➏
    if (ec) {
      std::cerr << "Error connecting to " << host << ": "
                << ec.message() << std::endl;
      return;
    }
    boost::asio::async_write(socket, boost::asio::buffer(request),
            [this] (boost::system::error_code ec, size_t transferred){
              write_handler(ec, transferred);
            });
  }

  void write_handler(boost::system::error_code ec, size_t transferred) { ➐
    if (ec) {
      std::cerr << "Error writing to " << host << ": " << ec.message()
                << std::endl;
    } else if (request.size() != transferred) {
      request.erase(0, transferred);
      boost::asio::async_write(socket, boost::asio::buffer(request),
                               [this] (boost::system::error_code ec,
                                       size_t transferred){
                                 write_handler(ec, transferred);
                               });
    } else {
      boost::asio::async_read(socket, boost::asio::dynamic_buffer(response),
                              [this] (boost::system::error_code ec,
                                      size_t transferred){
                                read_handler(ec, transferred);
                              });
    }
  }

  void read_handler(boost::system::error_code ec, size_t transferred) { ➑
    if (ec && ec.value() != 2)
      std::cerr << "Error reading from " << host << ": "
                << ec.message() << std::endl;
  }

  const std::string& get_response() const noexcept {
    return response;
  }
private:
  boost::asio::ip::tcp::resolver resolver;
  boost::asio::ip::tcp::socket socket;
  std::string request, response;
  const std::string host;
};

int main() {
  boost::asio::io_context io_context;
  Request request{ io_context, "www.arcyber.army.mil" }; ➒
  io_context.run(); ➓
  std::cout << request.get_response();
}
-----------------------------------------------------------------------
HTTP/1.1 200 OK
Pragma: no-cache
Content-Type: text/html; charset=utf-8
X-UA-Compatible: IE=edge
pw_value: 3ce3af822980b849665e8c5400e1b45b
Access-Control-Allow-Origin: *
X-Powered-By:
Server:
X-ASPNET-VERSION:
X-FRAME-OPTIONS: SAMEORIGIN
Content-Length: 76199
Cache-Control: private, no-cache
Expires: Mon, 22 Oct 2018 14:21:09 GMT
Date: Mon, 22 Oct 2018 14:21:09 GMT
Connection: close

<!DOCTYPE html>
<html  lang="en-US">
<head id="Head">
--snip--
</body>
</html>

Listing 20-11: An asynchronous refactor of Listing 20-9

You first declare a Request class that will handle a web request. It has a single constructor that takes an io_context and a string containing the host you want to connect with ➊. Just as in Listing 20-9, you create an HTTP GET request using a std::stringstream and save the resulting string into the request field ➋. Next, you use async_resolve to request the endpoints corresponding to the requested host. Within the callback, you invoke the resolution_handler method on the current Request ➌.

The resolution_handler receives the callback from async_resolve. It first checks for an error condition, printing to stderr and returning if it finds one ➍. If async_resolve didn’t pass an error, resolution_handler invokes async_connect using the endpoints contained in its results variable. It also passes the socket field of the current Request, which will store the connection that async_connect is about to create. Finally, it passes a connection callback as the third parameter. Within the callback, you invoke the connection_handler method of the current request ➎.

The connection_handler ➏ follows a similar pattern to the resolution_handler method. It checks for an error condition, and if one exists, it prints to stderr and returns; otherwise, it proceeds to process the request by invoking async_write, which takes three parameters: the active socket, a mutable buffer-wrapping request, and a callback function. The callback function, in turn, invokes the write_handler method on the current request.

Are you seeing a pattern here in these handler functions? The write_handler ➐ checks for an error and proceeds to determine whether the entire request has been sent. If it hasn’t, you still need to write some of the request, so you adjust the request accordingly and invoke async_write again. If async_write has written the entire request into socket, it’s time to read the response. For this, you invoke async_read using your socket, a dynamic buffer wrapping the response field, and a callback function that invokes the read_handler method on the current request.

The read_handler ➑ first checks for an error. Because your request used the Connection: close header, you expect an end-of-file error (value 2) as in Listing 20-10 and so ignore it. If it encounters a different kind of error, you print it to stderr and return. Your request is complete at this point. (Phew.)

Within main, you declare your io_context and initialize a Request to www.arcyber.army.mil ➒. Because you’re using asynchronous functions, you invoke the run method on io_context ➓. After io_context returns, you know that no asynchronous operations are pending, so you print the contents of the response on your Request object to stdout.

Serving

Building a server atop Boost Asio is essentially similar to building a client. To accept TCP connections, you use the boost::asio::ip::tcp::acceptor class, which takes a boost::asio::io_context object as its only constructor argument.

To accept a TCP connection using a blocking approach, you use the acceptor object’s accept method, which takes a boost::asio::ip::tcp::socket reference, which will hold the client’s socket, and an optional boost::error_code reference, which will hold any error conditions that arise. If you don’t provide a boost::error_code and an error arises, accept will throw a boost::system_error instead. Once accept returns without error, you can use the socket you passed in to read and write with the same read and write methods you used with the client in the previous sections.

For example, Listing 20-12 illustrates how to build an echo server that receives a message and sends it back uppercased to the client.

#include <iostream>
#include <string>
#include <boost/asio.hpp>
#include <boost/algorithm/string/case_conv.hpp>

using namespace boost::asio;

void handle(ip::tcp::socket& socket) { ➊
  boost::system::error_code ec;
  std::string message;
  do {
    boost::asio::read_until(socket, dynamic_buffer(message), "
"); ➋
    boost::algorithm::to_upper(message); ➌
    boost::asio::write(socket, buffer(message), ec); ➍
    if (message == "
") return; ➎
    message.clear();
  } while(!ec); ➏
}

int main()  {
  try {
    io_context io_context;
    ip::tcp::acceptor acceptor{ io_context,
                                ip::tcp::endpoint(ip::tcp::v4(), 1895) }; ➐
    while (true) {
      ip::tcp::socket socket{ io_context };
      acceptor.accept(socket); ➑
      handle(socket); ➒
    }
  } catch (std::exception& e) {
    std::cerr << e.what() << std::endl;
  }
}

Listing 20-12: An uppercasing echo server

You declare the handle function that accepts a socket reference corresponding to a client and handles messages from it ➊. Within a do-while loop, you read a line of text from the client into a string called message ➋, you convert it to uppercase using the to_upper function illustrated in Listing 15-31 ➌, and write it back to the client ➍. If the client sent a blank line, you exit from handle ➎; otherwise, you clear the contents of the message and loop if no error condition occurred ➏.

Within main, you initialize an io_context and an acceptor so that the program binds to the localhost:1895 socket ➐. Within an infinite loop, you create a socket and call accept on the acceptor ➑. As long as this doesn’t throw an exception, the socket will represent a new client, and you can pass this socket to handle to service the request ➒.

To interact with the server in Listing 20-12, you can use GNU Netcat, a network utility that allows you to create inbound and outbound TCP and UDP connections and then read and write data. If you’re using a Unix-like system, you probably have it installed. If you don’t, see https://nmap.org/ncat/. Listing 20-13 shows a sample session that connects to the uppercasing echo server.

$ ncat localhost 1895 ➊
The 300 ➋
THE 300
This is Blasphemy! ➋
THIS IS BLASPHEMY!
This is madness! ➋
THIS IS MADNESS!
Madness...? ➋
MADNESS...?
This is Sparta! ➋
THIS IS SPARTA!
➌
Ncat: Broken pipe. ➍

Listing 20-13: Interacting with the uppercasing echo server using Netcat

Netcat (ncat) takes two arguments: a host and a port ➊. Once you’ve invoked the program, each line you enter results in an uppercased result from the server. When you type text into stdin, Netcat sends it to the server ➋, which responds in uppercase. Once you send it an empty line ➌, the server terminates the socket and you get a Broken pipe ➍.

To accept connections using an asynchronous approach, you use the async_accept method on the acceptor, which takes a single argument: a callback object that accepts an error_code and a socket. If an error occurs, the error_code contains an error condition; otherwise, the socket represents the successfully connected client. From there, you can use the socket in the same way you did in the blocking approach.

A common pattern for asynchronous, connection-oriented servers is to use the std::enable_shared_from_this template discussed in “Advanced Patterns” on page 362. The idea is to create a shared pointer to a session object for each connection. When you register callbacks for reading and writing within the session object, you capture a shared pointer “from this” within the callback object so that while I/O is pending, the session stays alive. Once no I/O is pending, the session object dies along with all the shared pointers. Listing 20-14 illustrates how to reimplement the upper-casing echo server using asynchronous I/O.

#include <iostream>
#include <string>
#include <boost/asio.hpp>
#include <boost/algorithm/string/case_conv.hpp>
#include <memory>
using namespace boost::asio;

struct Session : std::enable_shared_from_this<Session> {
  explicit Session(ip::tcp::socket socket) : socket{ std::move(socket) } { } ➊
  void read() {
    async_read_until(socket, dynamic_buffer(message), '
', ➋
            [self=shared_from_this()] (boost::system::error_code ec,
                                       std::size_t length) {
              if (ec || self->message == "
") return; ➌
              boost::algorithm::to_upper(self->message);
              self->write();
            });
  }
  void write() {
    async_write(socket, buffer(message), ➍
                [self=shared_from_this()] (boost::system::error_code ec,
                                           std::size_t length) {
                  if (ec) return; ➎
                  self->message.clear();
                  self->read();
                });
  }
private:
  ip::tcp::socket socket;
  std::string message;
};

void serve(ip::tcp::acceptor& acceptor) {
  acceptor.async_accept([&acceptor](boost::system::error_code ec, ➏
                                    ip::tcp::socket socket) {
    serve(acceptor); ➐
    if (ec) return;
    auto session = std::make_shared<Session>(std::move(socket)); ➑
    session->read();
  });
}

int main()  {
  try {
    io_context io_context;
    ip::tcp::acceptor acceptor{ io_context,
                                ip::tcp::endpoint(ip::tcp::v4(), 1895) };
    serve(acceptor);
    io_context.run(); ➒
  } catch (std::exception& e) {
    std::cerr << e.what() << std::endl;
  }
}

Listing 20-14: An asynchronous version of Listing 20-12

You first define a Session class to manage connections. Within the constructor, you take ownership of the socket corresponding to the connecting client and store it as a member ➊.

Next, you declare a read method that invokes async_read_until on the socket so it reads into a dynamic_buffer wrapping the message member string up to the next newline character ➋. The callback object captures this as a shared_ptr using the shared_from_this method. When invoked, the function checks for either an error condition or an empty line, in which case it returns ➌. Otherwise, the callback converts message to uppercase and invokes the write method.

The write method follows a similar pattern as the read method. It invokes async_read, passing the socket, the message (now uppercase), and a callback function ➍. Within the callback function, you check for an error condition and return immediately if one exists ➎. Otherwise, you know that Asio successfully sent your uppercased message to the client, so you invoke clear on it to prepare for the next message from the client. Then you invoke the read method, which starts the process over.

Next, you define a serve function that accepts an acceptor object. Within the function, you invoke async_accept on the acceptor object and pass a callback function to handle connections ➏. The callback function first invokes serve again using the acceptor so your program can handle new connections immediately ➐. This is the secret sauce that makes the asynchronous handling so powerful on the server side: you can handle many connections at once because the running thread doesn’t need to service one client before handling another. Next, you check for an error condition and exit if one exists; otherwise, you create a shared_ptr owning a new Session object ➑. This Session object will own the socket that the acceptor just set up for you. You invoke the read method on the new Session object, which creates a second reference within the shared_ptr thanks to the shared_from_this capture. Now you’re all set! Once the read and write cycle ends due to an empty line from the client or some error condition, the shared_ptr reference will go to zero and the Session object will destruct.

Finally, within main you construct an io_context and an acceptor as in Listing 20-12. You then pass the acceptor to your serve function to begin the service loop and invoke run on the io_context to start servicing asynchronous operations ➒.

Multithreading Boost Asio

To make your Boost Asio program multithreaded, you can simply spawn tasks that invoke run on your io_context object. Of course, this doesn’t make your program safe, and all the admonitions in “Sharing and Coordinating” on page 647 are in full effect. Listing 20-15 illustrates how to multithread your server from Listing 20-14.

#include <iostream>
#include <string>
#include <boost/asio.hpp>
#include <boost/algorithm/string/case_conv.hpp>
#include <memory>
#include <future>
struct Session : std::enable_shared_from_this<Session> {
--snip--
};

void serve(ip::tcp::acceptor& acceptor) {
--snip--
}

int main()  {
  const int n_threads{ 4 };
  boost::asio::io_context io_context{ n_threads };
  ip::tcp::acceptor acceptor{ io_context,
                              ip::tcp::endpoint(ip::tcp::v4(), 1895) }; ➊
  serve(acceptor); ➋

  std::vector<std::future<void>> futures;
  std::generate_n(std::back_inserter(futures), n_threads, ➌
                  [&io_context] {
                    return std::async(std::launch::async,
                                      [&io_context] { io_context.run(); }); ➍
                  });

  for(auto& future : futures) { ➎
    try {
      future.get(); ➏
    } catch (const std::exception& e) {
      std::cerr << e.what() << std::endl;
    }
  }
}

Listing 20-15: Multithreading your asynchronous echo server

Your Session and serve definitions are identical. Within main, you declare n_threads constant representing the number of threads you’ll use to serve, an io_context, and an acceptor with parameters identical to those in Listing 12-12 ➊. Next, you invoke serve to begin the async_accept loop ➋.

More or less, main is almost identical to Listing 12-12. The difference is that you’ll dedicate multiple threads to running the io_context rather than just one. First, you initialize a vector to store each future corresponding to the tasks you’ll launch. Second, you use a similar approach with std::generate_n to create tasks ➌. As the generative function object, you pass a lambda that invokes std::async ➍. Within the std::async call, you pass the execution policy std::launch::async and a function object that invokes run on your io_context.

Boost Asio is off to the races now that you’ve assigned some tasks to running your io_context. You’ll want to wait for all asynchronous operations to complete, so you call get on each future you stored in futures ➎. Once this loop completes, each Request has finished and you’re ready to print a summary of the resulting responses ➏.

Sometimes it makes sense to create additional threads and assign them to processing I/O. Often, one thread will suffice. You must measure whether the optimization (and attendant difficulties arising from concurrent code) are worth it.

Summary

This chapter covered Boost Asio, a library for low-level I/O programming. You learned the basics of queuing asynchronous tasks and providing a thread pool in Asio, as well as how to interact with its basic networking facilities. You built several programs, including a simple HTTP client using synchronous and asynchronous approaches and an echo server.