7
THE NETWORK
FreeBSD is famous for its network performance. The TCP/IP network protocol suite was first developed on BSD, and BSD, in turn, included the first major implementation of TCP/IP. While competing network protocols were considered more exciting in the 1980s, the wide availability, flexibility, and liberal licensing of the BSD TCP/IP stack made it the de facto standard. This isn’t just a historical curiosity; today, Facebook is actively looking for engineers who can make Linux’s network performance match that of FreeBSD. The project is expected to take several years.
Many system administrators today have a vague familiarity with the basics of networking but don’t really understand how it all hangs together. Good sysadmins understand the network, however. Knowing what an IP address really is, how a netmask works, and how a port number differs from a protocol number is a necessary step toward mastering your profession. We’ll cover some of these issues in this chapter. For a start, you must understand the network layers.
While this chapter gives a decent overview of TCP/IP, it won’t cover many of the numerous details, gotchas, and caveats. If you need to learn more about TCP/IP, pick up a book on the subject. For an overview, check out my book Networking for Systems Administrators (Tilted Windmill Press, 2015). Eventually you’ll need a deep dive into networking; proceed directly to Charles M. Kozierok’s The TCP/IP Guide (No Starch Press, 2005).
The dominant internet protocol is TCP/IP (Transmission Control Protocol over Internet Protocol). TCP is a transport protocol, while IP is a network protocol, but they’re so tightly intertwined that they’re generally referred to as a single entity. We’ll look at how the network works, then discuss IP versions 4 and 6, and proceed to TCP and UDP.
Network Layers
Each layer of the network handles a specific task within the network process and interacts only with the layers above and below it. People learning TCP/IP often laugh when they hear that all these layers simplify the network process, but this is really true. The important thing to remember right now is that each layer communicates only with the layer directly above it and the layer directly beneath it.
The classic Open System Interconnection (OSI) network protocol stack has seven layers, is exhaustively complete, and covers almost any situation with any network protocol and any application. The internet, however, is just one such situation, and this isn’t a book about networking or networked applications in general. We’re limiting our discussion to TCP/IP networks, such as the internet and almost all corporate networks, so we need to consider only four layers of the network stack.
The Physical Layer
At the very bottom, we have the physical layer: the network card and the wire, fiber, or radio waves leaving it. This layer includes the physical switch; the hub, or base station; the cables attaching that device to the router; and the fiber that runs from your office to the telephone company. The telephone company switch is part of the physical layer, as are transcontinental fiber-optic cables. If it can be tripped over, dropped, or chainsawed, it’s part of the physical layer. From this point on, we’ll refer to the physical layer as the wire, although this layer can be just about any sort of medium.
This is the easiest layer to understand—it’s as simple as having intact hardware. If your wire meets the requirements of the physical protocol, you’re in business. If not, you’re bankrupt. Without a physical layer, the rest of the network can’t work—period. End of story. One of the functions of internet routers is to connect one sort of physical layer to another—for example, converting local Ethernet into optical fiber. The physical layer has no decision-making abilities and no intelligence; everything that runs over it is dictated by the datalink layer.
Datalink: The Physical Protocol
The datalink layer, or the physical protocol, is where things get interesting. This layer transforms information into the actual ones and zeros that are sent over the physical layer in the appropriate encoding for that physical protocol. For example, IP version 4 (IPv4) over Ethernet uses Media Access Control (MAC) addresses and the Address Resolution Protocol (ARP); IP version 6 (IPv6) over Ethernet uses Neighbor Discovery Protocol (NDP or sometimes ND). In addition to the popular Ethernet datalink layers, FreeBSD supports others, including Point-to-Point Protocol (PPP) and High-Level Data Link Control (HDLC), as well as combinations such as the PPP over Ethernet (PPPoE) used by some home broadband vendors. While FreeBSD supports all of these datalink protocols, it doesn’t support every datalink protocol ever used. If you have unusual network requirements, check the documentation for your version of FreeBSD to see whether it’s supported.
Some physical protocols have been implemented over many different physical layers. Ethernet, for instance, has been transmitted over twinax, coax, CAT3, CAT5, CAT6, CAT7, optical fiber, HDMI, and radio waves. With minor changes in the device drivers, the datalink layer can address any sort of physical layer. This is one of the ways in which layers simplify the network. We’ll discuss Ethernet in detail in “Understanding Ethernet” on page 140 at the end of this chapter, as it’s the most common network type FreeBSD systems use. By understanding Ethernet on FreeBSD, you’ll be able to manage other protocols on FreeBSD as well—once you understand those protocols, of course!
In addition to exchanging information with the physical layer, the datalink layer communicates with the network layer.
The Network Layer
The network layer? Isn’t the whole thing a network?
Yes, but the network layer is more specific. It maps connectivity between network nodes, answering questions like, “Where are other hosts?” and “Can you reach this particular host?” This logical protocol provides a consistent interface to programs that run over the network, no matter what sort of physical layer you’re using. The network layer used on the internet is Internet Protocol (IP). IP provides each host with a unique1 address, known as an IP address, so that any other host on the network can find it. You need to understand IP, both version 4 and version 6.
The network layer is where we truly abstract away the underlying physical media. Is IP running over Ethernet? ATM? Carrier pigeon? Who cares? It’s got an IP address, so we can talk to it. Move on.
The network layer talks to the datalink layer below it and the transport layer above it.
Heavy Lifting: The Transport Layer
The transport layer deals with real data for real applications and perhaps even real human beings. The three common transport layer protocols are ICMP, TCP, and UDP.
Internet Control Message Protocol (ICMP) manages basic connectivity messages between hosts with IP addresses. If IP provides a road and addresses, ICMP provides traffic lights and highway exit signs. Most of the time, ICMP just runs in the background and you never have to think about it.
The other well-known transport protocols are User Datagram Protocol (UDP) and Transmission Control Protocol (TCP). How common are these?
Well, the Internet Protocol suite is generally called TCP/IP. These protocols provide services such as multiplexing via port numbers and transmitting user data. UDP is a bare-bones transport protocol, offering the minimum services needed to transfer data over the network. TCP provides more sophisticated features, such as congestion control and integrity checking.
In addition to these three, many other protocols run above IP. The file /etc/protocols contains a fairly comprehensive list of transport protocols that use IP as an underlying mechanism. You won’t find non-IP protocols here, such as Digital’s LAT, but it contains many more protocols than you’ll ever see in the real world. For example, here are the entries for IP and ICMP, the network-layer protocols commonly used on the internet:
➊ip ➋0 ➌IP ➍# Internet protocol, pseudo protocol number
icmp 1 ICMP # Internet control message protocol
Each entry in /etc/protocols has three key fields: an unofficial name ➊, a protocol number ➋, and any aliases ➌. The protocol number is used within network requests to identify traffic. You’ll see it if you ever fire up a packet sniffer or start digging deeper into your network for any reason. As you can see, IP is protocol 0 and ICMP is protocol 1—if that’s not the groundwork for everything else, it’s hard to see what could be! TCP is protocol 6, and UDP is protocol 17. You’ll also see comments ➍ giving slightly more detail about each protocol.
The transport layer speaks to the network layer below and to the applications above it.
Applications
Applications are definitely a part of the network. Applications open requests for network connectivity, send data over the network, receive data from the network, and process that data. Web browsers, email clients, JSP servers, and so on are all network-aware applications. Applications have to communicate only with the network protocol and the user. Problems with the user layer are beyond the scope of this book.2
The Network in Practice
So, you understand how everything hooks together and are ready to move on, right? Don’t think so. Let’s see how this works in the real world. Some of this explanation touches on stuff that we’ll cover later in this chapter, but if you’re reading this book, you’re probably conversant enough with networks to be able to follow it. If you’re having trouble, reread this section after reading the remainder of this chapter. (Just buy a second copy of this book, cut these pages out of the second copy, and glue them in at the end of this chapter.)
Suppose a user connected to the internet via your network wants to look at Yahoo! The user accesses his web browser and enters the URL.
The browser application knows how to talk to the next layer down in the network, which is the transport layer. After kneading the user’s request into an appropriate form, the browser asks the transport layer for a TCP connection to a particular IP address on port 80. (Purists will note that we’re skipping the DNS request part of the process, but it’s quite similar to what’s being described and would only confuse our example.)
The transport layer examines the browser’s request. Since the application has requested a TCP connection, the transport layer allocates the appropriate system resources for that sort of connection. The request is broken up into digestible chunks and handed down to the network layer.
The network layer doesn’t care about the actual request. It’s been handed a lump of data to be carried over the internet. Much like your mail carrier delivers letters without caring about the contents, the network layer just bundles the TCP data with the proper addressing information. The resulting mass of data is called a packet. The network layer hands these packets down to the datalink layer.
The datalink layer doesn’t care about the contents of the packet. It certainly doesn’t care about IP addressing or routing. It’s been given a lump of zeros and ones, and it has the job of transmitting those zeros and ones across the network. All it knows is how to perform that transmission. The datalink layer may add the appropriate header and/or footer information to the packet for the physical medium used, creating a frame. Finally, it hands the frame off to the physical layer for transmission on the local wire, wave, or other media.
The physical layer has no intelligence at all. The datalink layer hands it a bunch of zeros and ones, and the physical layer transmits them to another physical device. It has no idea what protocol is being spoken or how those digits might be echoed through a switch, hub, or repeater, but one of the hosts on this network is presumably the router of the network.
When the router receives the zeros and ones, it hands them up to the datalink layer. The datalink layer strips its framing information and hands the resulting packet up to the network layer within the router. The router’s network layer examines the packet and decides what to do with it based on its routing tables. It then hands the packet down to the appropriate datalink layer. This might be another Ethernet interface or perhaps a PPP interface out of a T1.
Your wire can go through many physical changes as the data travels. Your cable internet line could be aggregated into an optical fiber DS3, which is then transformed into an OC192 cross-country link. Thanks to the wonders of layering and abstraction, neither your computer nor your user needs to know anything about any of these.
When the request reaches its destination, the computer at the other end of the transaction accepts the frame and sends it all the way back up the protocol stack. The physical wire accepts the zeros and ones and sends them up to the datalink layer. The datalink layer strips the Ethernet headers off the frame and hands the resulting packet up to the network. The network layer strips off the packet header and shuffles the remaining segments up to the transport layer. The transport layer reassembles the segments into a data stream, which it then hands to an application—in this case, a web server. The application processes the request and returns an answer, which descends the protocol stack and travels across the network, bouncing up and down through various datalink layers on the way as necessary. This is an awful lot of work to make the machine go through just so you can get your “404 Page Not Found” error.
This example shows why layering is so important. Each layer knows only what it absolutely must about the layers above and below it, making it possible to swap out the innards of layers if desired. When a new datalink protocol is created, the other layers don’t have to change; the network protocol just hands a properly formatted request to the datalink layer and lets that layer do its thing. When you have a new network card, you only need a driver that interfaces with the datalink layer and the physical layer; you don’t have to change anything higher in the network stack, including your application. Imagine a device driver that had to be installed in your web browser, your email client, and every other application you had on your computer, including the custom-built ones. You would quickly give up on computing and take up something sane and sensible, like skydiving with anvils.
Getting Bits and Hexes
As a system administrator, you’ll frequently come across terms like 48-bit address and 18-bit netmask. I’ve seen a surprising number of sysadmins who just nod and smile when they hear this, all the while thinking, “Yeah, whatever, just tell me what I need to know to do my job.” Unfortunately, math is a real part of the job, and you must understand bits. While this math is not immediately intuitive, understanding it is one of the things that separates amateurs from professionals. You don’t read a book like this if you want to stay an amateur.
Maybe you’re muttering, “But I already know this!” Then skip it. But don’t cheat yourself if you don’t.
You probably already know that a computer treats all data as zeros and ones, and that a single zero or one is a bit. When a protocol specifies a number of bits, it’s talking about the number as seen by the computer. A 32-bit number has 32 digits, each being either zero or one. You were probably introduced to binary math, or base 2, back in elementary school and remembered it just long enough to pass the test. It’s time to dust off those memories. Binary math is simply a different way to work with the numbers we see every day.
We use decimal math, or base 10, every day to pay the pizza guy and balance the checkbook. Digits run from 0 to 9. When you want to go above the highest digit you have, you add a digit on the left and set your current digit to zero. This is the whole “carry the one” thing you learned many years ago and now probably do without conscious thought. In binary math, the digits run from 0 to 1, and when you want to go above the highest digit you have, you add a digit on the left and set your current digit to 0. It’s exactly the same as decimal math with eight fingers missing. As an example, Table 7-1 shows the first few decimal numbers converted to binary.
Table 7-1: Decimal and Binary Numbers
Decimal |
Binary |
0 |
0 |
1 |
1 |
2 |
10 |
3 |
11 |
4 |
100 |
5 |
101 |
6 |
110 |
7 |
111 |
8 |
1000 |
When you have a 32-bit number, such as an IP address, you have a string of 32 ones and zeros. Ethernet MAC addresses are 48-bit numbers and have 48 ones and zeros.
Just for fun, Unix also uses hexadecimal numbers in some cases, such as MAC addresses and netmasks. Hexadecimal digits are 4 bits long. The binary number 1111, the full 4 bits, is equivalent to 15; this means that the digits in hexadecimal math run from 0 to 15. At this point, a few of you are looking at the 2-digit number 15 that’s supposed to be a single digit and wondering what I’m smoking and where you can get your own supply. Hexadecimal math uses the letters A through F as digits for the numbers 10 through 15. When you count up to the last digit and want to add one, you set the current digit to zero and add a digit to the left of the number. For example, to count to 17 in hexadecimal, you say, “1, 2, 3, 4, 5, 6, 7, 8, 9, A, B, C, D, E, F, 10, 11.” Take off a shoe and count along once or twice until you get the idea.
Hexadecimal numbers are usually marked with a 0x in front. The number 0x12 is the hexadecimal equivalent of decimal 18, while the number 18 is plain old 18. If a hex number is not marked by a leading 0x, it’s in a place where the output is always in hexadecimal, such as MAC addresses. The letters A to F are also a dead giveaway, but not entirely reliable; many hex numbers have no letters at all, just as many decimal numbers have no odd digits.
When you’re working with hexadecimal, decimal, and binary numbers, the simplest thing to do is break out a scientific calculator. Today’s medium-end or better calculators have functions to convert between the three systems, as do most software calculators.
Network Stacks
A network stack is the software that lets a host communicate with other hosts over the network. A host can run with an IPv4-only network stack, an IPv6-only network stack, or a dual-stacked setup. FreeBSD enables both by default.
You’re probably familiar with an IPv4-only stack. Most hosts have run on IPv4 for much of the past 30 years. An IPv4-only stack can communicate only over IPv4. Today, an IPv4-only stack gets you access to most of the internet, with a few deliberate exceptions. That will not be true in a few years.
Likewise, an IPv6-only stack can communicate only with IPv6 hosts. The majority of large internet sites support IPv6, but you’ll find a few annoying exceptions.3 Using only IPv6 will cut you off from some popular internet sites.
The most common server configuration these days is a dual-stack setup. Client hosts try to use both IPv4 and IPv6, preferring one over the other. The last few versions of Microsoft Windows have preferred IPv6.
We’ll look at the more familiar IPv4 first and then use IPv4 as a reference to discuss IPv6.
IPv4 Addresses and Netmasks
An IP address is a unique 32-bit number assigned to a particular node on a network. Some IP addresses are more or less permanent, such as those assigned to vital servers. Others change as required by the network, such as those used by dial-up clients. Individual machines on a shared network get adjoining IP addresses.
Rather than expressing that 32-bit number as a single number, an IP address is broken up into four 8-bit numbers, usually shown as decimal numbers. While 203.0.113.1 is the same as 11001011.00000000.01110001.00000001 or the single number 11001011000000000111000100000001, the four decimal numbers are easiest for our weak minds to deal with.
IP addresses are issued in chunks by internet service providers. Frequently these chunks are very small—say, 16 or 32 IP addresses. If your system is colocated on a server farm, you might only get a few IP addresses out of a block.
A netmask, which might also be called a prefix length or slash, is a label indicating the size of the block of IP addresses assigned to your local network. The size of your IP block determines your netmask—or, your netmask determines how many IP addresses you have. If you’ve done networking for any length of time, you’ve seen the netmask 255.255.255.0 and know that it’s associated with a block of 256 IP addresses. You might even know that the wrong netmask prevents your system from working. In today’s world, however, that simple netmask is becoming less and less common. Netmasks made up of 255s and 0s are easy to look at but waste IP addresses.4 And IPv4 addresses are an extremely scarce resource.
When you get a block of IP addresses for your server, it’ll probably look something like 203.0.113.128/25. This isn’t a class in binary math, so I won’t make you draw it out and do the conversion, but think of an IP address as a string of binary numbers. On your network, you can change the bits on the far right, but not the ones on the far left. The only question is, “Where is the line that separates right from left?” There’s no reason for that boundary to be on one of those convenient 8-bit lines that separate the decimal versions of the address. A prefix length is simply the number of fixed bits on your network. A /25 means that you have 25 fixed bits. You can play with 7 bits. You get a decimal netmask by setting the fixed bits to 1 and your network bits to 0, as in the following example of a /25 netmask:
11111111.11111111.11111111.10000000
A binary 11111111 is a decimal 255, while 1000000 is 128. Your netmask is 255.255.255.128. It’s very simple, if you think in binary. You won’t have to work with this every day, but if you don’t understand the underlying binary concepts, the decimal conversion looks deranged. With practice, you’ll learn to recognize certain strings of decimals as legitimate binary conversions.
What does all this mean in practice? First off, blocks of IP addresses are issued in multiples of 2. If you have 4 bits to play with, you have 16 IP addresses (2 × 2 × 2 × 2 = 16). If you have 8 bits to play with, you have 256 addresses (28 = 256). If someone says that you have exactly 19 IP addresses, you’re either sharing an Ethernet with other people or they’re wrong.
It’s not uncommon to see a host’s IP address with its netmask attached—for example, 198.51.100.4/26. This gives you everything you need to get the host on the local network. (Finding the default gateway is another problem, but by convention, it’s most often the top or bottom address in the block.)
Computing Netmasks in Decimal
You probably don’t want to repeatedly convert between decimal and binary. Not only is it uncomfortable; it also increases your chances of making an error. Here’s a trick to calculate your netmask while remaining in decimal land.
You need to find how many IP addresses you have on your network. This will be a multiple of 2 almost certainly smaller than 256. Subtract the number of IP addresses you have from 256. This is the last number of your netmask. You’ll still need to recognize legitimate network sizes. If your IP address is 203.0.113.100/26, you’ll need to know that a /26 is 26 fixed bits, or 64 IP addresses. Look at the last number of your IP address, 100. It certainly isn’t between 0 and 63, but it’s between 64 and 127. The other hosts on your IP block have IP addresses ranging from 203.0.113.64 to 203.0.113.127, and your netmask is 255.255.255.192 (256 – 64 = 192).
At this point, I should mention that netmasks frequently appear as hex numbers. You might feel like giving up the whole thing as a bad job, but to simplify your life, Table 7-2 shows netmasks, IP information, and related goodness for /24 and smaller networks.
Table 7-2: Netmask and IP Address Conversions
Prefix |
Binary mask |
Decimal mask |
Hex mask |
Available IPs |
/24 |
00000000 |
255.255.255.0 |
0xffffff00 |
256 |
/25 |
10000000 |
255.255.255.128 |
0xffffff80 |
128 |
/26 |
11000000 |
255.255.255.192 |
0xffffffc0 |
64 |
/27 |
11100000 |
255.255.255.224 |
0xffffffe0 |
32 |
/28 |
11110000 |
255.255.255.240 |
0xfffffff0 |
16 |
/29 |
11111000 |
255.255.255.248 |
0xfffffff8 |
8 |
/30 |
11111100 |
255.255.255.252 |
0xfffffffc |
4 |
/31 |
11111110 |
255.255.255.254 |
0xfffffffe |
2 |
/32 |
11111111 |
255.255.255.255 |
0xffffffff |
1 |
Unusable IP Addresses
You now understand how slashes, netmasks, and IP address assignments work together and how, for example, a /28 has 16 IP addresses. Unfortunately, you can’t use all the IP addresses in a block. The first IP address in a block is the network number, which is used for internal bookkeeping.
Traditionally, the last number in any block of IP addresses was called the broadcast address. According to the original IP specifications, every machine on a network was supposed to respond to a request for this address. This allowed you to ping the broadcast address to quickly determine which IP addresses were in use. For example, on a typical /24 network, the broadcast address was x .y .z .255. In the late 1990s, however, this feature was transformed into an attack technique and was disabled by default on almost every operating system and most network appliances.5 If you need this feature, set the sysctl net.inet.icmp.bmcastecho to 1. In most environments, the broadcast address is a waste of an IP address. In any case, you can’t assign the first or the last IP address in a network to a device without causing network problems. (Yes, this makes /31 networks useless.) Some systems fail gracefully; others fail gracelessly. Go ahead and try it sometime—preferably after hours, unless you want a good story to tell at your next job.
Assigning IPv4 Addresses
You might think that each computer on a network has an IP address, but this isn’t strictly true. Every network interface has an IP address. Most computers have only one network interface, so for them, the difference is nonexistent. If you have multiple network cards, however, each card has a separate IP address. You can also have multiple IP addresses on a single card through aliasing. On the other hand, with special configuration you can bond multiple cards into a single network interface, giving the computer one virtual interface despite the many cards. While these distinctions are small, remember them when troubleshooting.
The IP address 127.0.0.1 is always attached to every host’s loopback interface. It can be reached only from the local machine.
IPv6 Addresses and Subnets
The original engineers of IPv4 thought that 4.29 billion IP addresses would be enough for the whole world. Computers were expensive, after all, and only military and educational systems connected to the internet. It’s not as if every person in the world would one day own multiple networked devices.
Oops.
Unused IPv4 addresses are no longer available. The prices for used IPv4 addresses are increasing. Eventually, IPv4 addresses will be priced beyond reach for most people. The world is unwillingly groaning toward IPv4’s replacement—IP version 6.
Telecom networks and parts of the world outside North America already use IPv6 pretty widely. Even if your network doesn’t use IPv6 today, one day you’ll unexpectedly discover that you needed it the week before.
IPv6 Basics
Like IPv4, IPv6 is a network-layer protocol. TCP, UDP, ICMP, and other protocols run atop it. Recall that IPv4 uses 32-bit addresses, usually expressed as four groups of decimal numbers from 0 to 255—for example, 203.0.113.13. IPv6 uses 128-bit addresses, expressed as eight groups of four hexadecimal characters separated by colons—for example, 2001:db8:5c00:0:90ff:bad:c0de:cafe.
A 128-bit address space is unimaginably huge, but let’s try to imagine it. Count every human being that’s ever lived. Now count the number of cells in each of them—not just in their body but also all the bacterial cells in their bodies. IPv6 is roomy enough to assign each of those cells an address space larger than the entirety of IPv4.
The good news is that you don’t need to relearn the basics of networking. Hosts need an IP address, a netmask, and a default gateway. You can almost—almost—substitute an IPv6 address for an IPv4 address and watch everything just work. A web server doesn’t care whether it binds to port 80 on 203.0.113.13 or 2001:db8:5c00:0:90ff:bad:c0de:cafe. The server accepts requests it receives and responds appropriately. That said, software does need to change slightly because our web server must be able to log connections from both IPv4 and IPv6 addresses. These changes have wide-reaching repercussions, and we’ll be discovering new edge cases for decades. But, in general, once you understand the new rules for IPv6, all of your networking knowledge is applicable.
Understanding IPv6 Addresses
As noted, IPv6 addresses are 128 bits, expressed as eight colon-delimited groups of four hexadecimal characters each. As with decimal IP addresses, you don’t need to display leading zeros in each group. The address 2001:db8:5c00:0:90ff:bad:c0de:cafe could be written as 2001:0db8:5c00:0000:90ff:0bad:c0de:cafe, but just as we wouldn’t write 203.000.113.013, we strip out the leading zeros in an IPv6 address.
IPv6 addresses often contain long strings of zeros because of the way IPv6 subnets. As I write this, the IPv6 address of Sprint’s website is 2600:0:0:0:0:0:0:0. When consecutive groups contain only zeros, they’re replaced with two colons. You can display this IP address as 2600::. You can do the double-colon substitution only once per address, however. Addresses like 2001::a::1 would be ambiguous. Does 2001::a::1 represent 2001:0:0:0:0:a:0:1, 2001:0:0:0:a:0:0:1, 2001:0:0: a:0:0:0:1, or 2001:0:a:0:0:0:0:0:1? No way to tell.
You’ve probably seen a port number added to an IPv4 address, such as 203.0.113.13:80. Using this terminology with IPv6 addresses would make them even uglier and confuse everyone. An IP address and port combination like 2001:db8:5c00:0:90ff:bad:c0de:cafe:80 is not ambiguous, but unless you read it very carefully, you might think it’s an IP address ending in 80. If you’re expressing an IP and port combination, enclose the address in square brackets, as in [2001:db8:5c00:0:90ff:bad:c0de:cafe]:80.
IPv6 Subnets
IPv6 addresses have colons every 16 bits, so the obvious and natural ways to divide networks are at the /16, /32, /48, /64, /80, /96, and /112. The original IPv6 standards recommend subnetting only on these boundaries (repeating one of IPv4’s greatest mistakes), but that’s increasingly being rejected in favor of IPv4-style subnetting anywhere. IPv6 subnets are always expressed as a slash, also known as a prefix length, so you won’t see a netmask like ffff:ffff:ffff:ffff:: analogous to IPv4.
ISPs are usually issued a /32 or /48 and are expected to issue end-user networks, such as a typical client, a /64 network. A /64 has 264 subnets, or 18,446,744,073,709,551,616 addresses. If your home or office runs out of IP addresses, you need to stop networking individual blueberries.
When you subnet at 16-bit boundaries, each network has 65,536 subnets of the next smaller size. A /32 contains 65,536 /48 networks, and a /48 contains 65,536 /64 networks.
This is a long-winded way of explaining why I don’t provide handy charts of IPv6 subnets and network size. Do an internet search for “IPv6 subnet calculator” to use one of the many on the internet.
Link-Local Addresses
Addresses beginning with fe8x: (where x is any hexadecimal character) are local to their interface. Every interface has such link-local addresses that are valid only on a specific local network. Even if an IPv6 network has no router, hosts on the local directly attached network can find each other and communicate using these local addresses. Link-local networks are always /64 subnets. You’ll see identical IPv6 subnets on other interfaces and on networks completely disconnected from your network. That’s okay. These addresses are local to the link. For example, here’s a link-local address from a test machine.
fe80::bad:c0de:cafe%vtnet0
The link-local address of this interface is fe80::bad:c0de:cafe. The trailing %vtnet0 indicates that this address is local to the interface vtnet0 and isn’t usable on any other interface on the machine. If your machine has an interface vtnet1, and a host on that network tries to reach the address fe80::bad:c0de:cafe, this machine will not respond. This particular address is valid only for hosts on the network segment directly attached to interface vtnet0.
You might note that the link-local address has a section in common with the public IPv6 address on this interface. That’s because an autoconfigured IPv6 address is usually calculated from the interface’s physical address; it doesn’t matter whether that autoconfigured address is public or local to the link.
Assigning IPv6 Addresses
IPv6 clients on a /64 or larger network can normally autoconfigure their network through router discovery. Router discovery resembles a stripped-down DHCP service. The router broadcasts gateway and subnet information, and the hosts configure themselves to use it.
Modern versions of router discovery include very basic DHCP-style options, such as DNS servers. Not all IPv6 providers include these options in their router discovery configuration, however. If you want to provide sophisticated autoconfiguration for phones or diskless hosts, or if your provider doesn’t offer DNS information in their configuration, you’ll need to set up an IPv6 DHCP server.
Servers should not use IPv6 autoconfiguration. A server usually needs a static IP, even in IPv6.
Hosts on a network smaller than /64 must be manually configured.
The address ::1 always represents the local host and is assigned to the loopback address.
TCP/IP Basics
Now that you have a simple overview of how the IP system works, let’s consider the most common network protocols in more depth. The dominant transport protocol on the internet is the Transmission Control Protocol over Internet Protocol, or TCP/IP. Although TCP is a transport protocol and IP is a network protocol, the two are so tightly intertwined that they’re generally referred to as a single entity.
We’ll start with the simplest, ICMP, and proceed to UDP and TCP. All of these protocols run over both IPv4 and IPv6. While the versions of each protocol vary according to the underlying IP protocol, they behave essentially the same.
ICMP
The Internet Control Message Protocol (ICMP) is the standard for transmitting routing and availability messages across the network. Tools such as ping(8) and traceroute(8) use ICMP to gather their results. IPv4 and IPv6 have slightly different versions of ICMP, sometimes called ICMPv4 and ICMPv6.
While some people claim that you must block ICMP for security reasons, ICMP is just as diverse as the better-understood protocols TCP and UDP. Proper IPv4 network performance requires large chunks of ICMPv4. If you feel you must block ICMP, do so selectively. For example, blocking source quench messages breaks path maximum transmission unit (pMTU) discovery, which is like faceplanting into a crate of broken glass and rusty nails. If you don’t understand that last sentence, don’t block ICMP.
IPv6 dies without ICMPv6, as IPv6 doesn’t support packet fragmentation. If you use IPv6, never block ICMPv6 as a whole. Blocking parts of ICMPv6 without destroying your network requires careful research and testing.
UDP
The User Datagram Protocol (UDP) is the most bare-bones data transfer protocol that runs over IP. It has no error handling, minimal integrity verification, and no defense whatsoever against data loss. Despite these drawbacks, UDP can be a good choice for particular sorts of data transfer, and many vital internet services rely on it.
When a host transmits data via UDP, the sender has no way of knowing whether the data ever reached its destination. Programs that receive UDP data simply listen to the network and accept what happens to arrive. When a program receives data via UDP, it cannot verify the source of that data—while a UDP packet includes a source address, this address is easily faked. This is why UDP is called connectionless, or stateless.
With all of these drawbacks, why use UDP at all? Applications that use UDP most often have their own error-correction handling methods that don’t mesh well with the defaults provided by protocols such as TCP. For example, simple client DNS queries must time out within just a few seconds or the user will call the helpdesk and whine. TCP connections time out only after two minutes. Since the computer wants to handle its failed DNS requests much more quickly, simple DNS queries use UDP. In cases where DNS must transfer larger amounts of data (for example, for zone transfers), it intelligently switches to TCP. Real-time streaming data, such as video conferencing, also uses UDP. If you miss a few pixels of the picture in a real-time video conference, retransmitting that data would simply add congestion. You can’t go back in time to fill in those missing chunks of the picture, after all! You’ll find similar reasoning behind almost all other network applications that use UDP.
Because the UDP protocol itself doesn’t return anything when you connect to a port, there’s no reliable way to remotely test whether a UDP port is reachable (although tools like nmap try to do so).
UDP is also a datagram protocol, meaning that each network transmission is complete, self-contained, and received as a single integral unit. While the application might not consider a single UDP packet a complete request, the network does. TCP is entirely different.
TCP
The Transmission Control Protocol (TCP) includes such nifty features as error correction and recovery. The receiver must acknowledge every packet it gets; otherwise, the sender will retransmit any unacknowledged packets. Applications that use TCP can expect reliable data transmission. This makes TCP a connected, or stateful, protocol, unlike UDP.
TCP is also a streaming protocol, meaning that a single request can be split amongst several network packets. While the sender might transmit several chunks of data one after the other, the recipient could receive them out of order or fragmented. The recipient must keep track of these chunks and assemble them properly to complete the network transaction.
For two hosts to exchange TCP data, they must set up a channel for that data to flow across. One host requests a connection, the other host responds to the request, and then the first host starts transmitting. This setup process is known as the three-way handshake. The specifics are not important right now, but you should know that this process happens. Similarly, once transmission is complete, the systems must do a certain amount of work to tear down the connections.
TCP is commonly used by applications—such as email programs, FTP clients, and web browsers—for its fairly generic set of timeouts and transmission features.
How Protocols Fit Together
You can compare the network stack to sitting with your family at a holiday dinner. The datalink layer (ARP, in the case of IPv4 over Ethernet) lets you see everyone else at the table. IP gives every person at the table a unique chair, except for the three young nephews using piano bench NAT. ICMP provides basic routing information, such as, “The quickest way to the peas is to ask Uncle Chris to hand them to you.” TCP is where you hand someone a dish and the other person must say, “Thanks,” before you let go. Finally, UDP is like tossing a roll at Aunt Betty; she might catch it, it might bounce off her forehead, or it could be snatched out of midair by the dog who has watched for her opportunity since the meal began.
Transport Protocol Ports
Have you ever noticed that computers have too many ports? We’re going to add TCP and UDP ports into the stew. Transport protocol ports permit one server to serve many different services over a single transport protocol, multiplexing connections between machines.
When a network server program starts, it attaches, or binds, to one or more logical ports. A logical port is just an arbitrary number ranging from 1 to 65535. For example, internet mail servers bind to TCP port 25. Each TCP or UDP packet arriving at a system has a field indicating its desired destination port. Each incoming request is flagged with a desired destination port number. If an incoming request asks for port 25, it’s connected to the mail server listening on that port. This means that other programs can run on different ports, clients can talk to those different ports, and nobody except the sysadmin gets confused.
The /etc/services file contains a list of port numbers and the services that they’re commonly associated with. It’s possible to run almost any service on any port, but by doing so, you’ll confuse other internet hosts that try to connect to your system. If someone tries to send you email, their mail program automatically connects to port 25 on your system. If you run email on port 77 and you have a web server on port 25, you’ll never get your email and your web server will start receiving spam. The /etc/services file has a very simple five-column format.
➊qotd ➋17/➌tcp ➍quote ➎#Quote of the Day
This is the entry for the qotd service ➊, which runs on port 17 ➋ in the TCP protocol ➌. It’s also known as the quote service ➍. Finally, we have a comment ➎ that provides more detail; apparently qotd stands for quote of the day. Services are assigned the same port number in both TCP and UDP, even though they usually run only on one and not the other—for example, qotd has ports 17/tcp and 17/udp.
Many server programs read /etc/services to learn which port to bind to on startup, while client programs read /etc/services to learn which port they should try to connect to. If you run servers on unusual ports, you might have to edit this file to tell the server where to attach to.
As in all standards, there are often good reasons for breaking the rules. The SSH daemon, sshd, normally listens on port 22/tcp, but I’ve run it on ports 23 (telnet), 80 (HTTP), and 443 (HTTPS) for various reasons. Configuring this depends on the server program you’re using.
Reserved Ports
Ports below 1024 in both TCP and UDP are called reserved ports. These ports are assigned only to core internet infrastructure and important services such as DNS, SSH, HTTP, LDAP, and so on—services that should legitimately be offered only by a system or network administrator. Only programs with root-level privileges can bind to low-numbered ports. A user can provide, say, a game server on a high-numbered port if the system policy allows—but that’s a little different from setting up an official-looking web page that’s visible to everyone and states that the main purpose of the machine is to be a game server! The port assignment for these core protocols is generally carved in stone.
You can view and change the reserved ports with the sysctls net.inet.ip.portrange.reservedhigh and net.inet.ip.portrange.reservedlow.
Every so often, someone thinks that they can disable this “bind-only-by-root” feature and increase their system’s security—after all, if your application can be run as a regular user instead of root, wouldn’t that increase system security? Most programs that run on reserved ports actually start as root, bind to the port, and then drop privileges to a special restricted user that has even less privilege than a regular user. These programs are designed to start as root and frequently behave differently when run as a regular user. A few programs, such as the Apache web server, are written so they can be started safely by a non-root user, but others are not.
Understanding Ethernet
Ethernet is extremely popular in corporate and home networks and is the most common connection media for FreeBSD systems. Ethernet is a shared network; many different machines can connect to the same Ethernet and can communicate directly with each other. This gives Ethernet a great advantage over other network protocols, but Ethernet has physical distance limitations that make it practical only for offices, co-location facilities, and other comparatively small networks.6
Many different physical media have supported Ethernet over the years. Once upon a time, most Ethernet cables were thick chunks of coaxial cable. Today, most are comparatively thin CAT6 cables with eight strands of very thin wire inside them. You might also encounter Ethernet over optical fiber or radio. For purposes of our discussion, we’ll assume that you’re working with CAT6 or better cable, today’s most popular choice. No matter what physical media you use, the theory of Ethernet doesn’t change—remember, the physical layer is abstracted away.
Protocol and Hardware
Ethernet is a broadcast protocol, which means that every packet you send on the network can be sent to every workstation on the network. (Note that I said can be; some Ethernet hardware limits recipients of these broadcasts.) Either your network card or its device driver separates the data intended for your computer from the data meant for other computers. One side effect of Ethernet’s broadcast nature is that you can eavesdrop on other computers’ network traffic. While this can be very useful when diagnosing problems, it’s also a security issue. Capturing clear-text passwords is trivial on an old-fashioned Ethernet. A section of Ethernet where all hosts can communicate directly with all other hosts without involving a router is called a broadcast domain, or segment.
Ethernet segments are connected via hubs or switches. An Ethernet hub is a central piece of hardware to physically connect many other Ethernet devices. Hubs simply forward all received Ethernet frames to every other device attached to the network. Hubs broadcast all Ethernet traffic that they receive to every attached host and other attached hubs. Each host is responsible for filtering out the traffic it doesn’t want. Hubs are old-school Ethernet and rarely seen today.
Switches have largely supplanted hubs. A switch is like a hub, but it filters which traffic it sends to each host. It identifies the physical addresses of attached devices and, for the most part, forwards frames only to the devices they are meant for. Since each Ethernet host has a finite amount of bandwidth, switching reduces the load on individual systems by decreasing the amount of traffic each host must sort through.
Switch Failure
Switches fail, despite what Cisco would have you believe. Some failures are obvious, such as those where the magic black smoke is leaking out of the back of the box. When a switch loses its magic smoke, it stops working. Others are more subtle and make it appear that the switch is still working.
Every switch manufacturer must decide how to handle subtle errors. Either the switch can shut down until it is attended to, or it can attempt to alert its manager and continue forwarding packets to the best of its ability. If you’re a vendor, the choice is obvious—you stumble along as best you can so your customers don’t think your switches are crap. This means your switch can start to act like a hub and you might not know about it. The bad news is that if you were relying on the switch to prevent leakage of secure information, you’re fated for disappointment. More than one switch has failed on me in this way, so don’t be too surprised when it happens to you.
Installing a syslog server (see Chapter 21) and having your switches log to it can mitigate this risk. While logging won’t prevent switch failure, it will simplify listening to a dying switch when it tries to complain.
Ethernet Speed
Ethernet originally supported only a couple of megabits per second but has expanded to handle tens-of-gigabits speeds. Most Ethernet cards are gigabit speed, meaning they can handle a gigabit per second, but you’ll find a few 10Gbs or 100Gbs cards in high-speed applications. If a card is labeled gigabit, it doesn’t mean it can actually push that much traffic—I’ve seen gigabit cards choke on a tenth that much bandwidth. Card quality is important when you want to push bandwidth, and the quality of the entire computer is important when pushing serious bandwidth.
Let the switch and the card negotiate their settings on their own through autonegotiation. While some old hands might remember disabling autonegotiation on older Ethernet cards, gigabit and faster Ethernet requires autonegotiation to function.
MAC Addresses
Every Ethernet card has a unique identifier, a Media Access Control (MAC) address. This 48-bit number is sometimes called an Ethernet address or physical address. When a system transmits data to another host on the Ethernet, it first broadcasts an Ethernet request asking, “Which MAC address is responsible for this IP address?” If a host responds with its MAC address, further data for that IP is transmitted to that MAC address.
IPv4 uses the Address Resolution Protocol (ARP) to map IP addresses to hosts. Use arp(8) to view your FreeBSD system’s knowledge of the ARP table. The most common usage is the arp -a command, which shows all of the MAC addresses and hostnames that your computer knows of.
# arp -a
gw.blackhelicopters.org (198.51.100.1) at 00:00:93:34:4e:78 on igb0 [ethernet]
sipura.blackhelicopters.org (198.51.100.5) at 00:00:93:c2:0f:8c on igb0 [ethernet]
This full listing of ARP information is known as the ARP table, or MAC table. (The terms MAC and ARP are frequently used interchangeably, so don’t worry about it too much.) Here we see that the host gw.blackhelicopters.org has an IP address of 198.51.100.1 and a MAC address of 00:00:93:34:4e:78, and that you can reach these hosts on the local system’s interface, igb0.
If a MAC address shows up as incomplete, the host cannot be contacted on the local Ethernet. In this case, check your physical layer (the wire), the remote system, and the configuration of both systems.
IPv6 uses Neighbor Discovery Protocol (NDP) to map IPv6 addresses to MAC addresses. It’s a separate protocol from ARP to encompass router discovery. Use ndp(8) to view the host’s MAC table and corresponding IPv6 addresses. The output deliberately resembles that of arp(1).
# ndp -a
Neighbor Linklayer Address Netif Expire S Flags
fe80::fc25:90ff:fee8:1270%vtnet0 fe:25:90:e8:12:70 vtnet0 4s R R
www.michaelwlucas.com 00:25:90:e8:12:70 vtnet0 permanent R
fe80::225:90ff:fee8:1270%vtnet0 00:25:90:e8:12:70 vtnet0 permanent R
The output deliberately resembles that of arp(8) but is slightly more tabular. The Neighbor column shows either the IPv6 address, the hostname, or the link-local address of each neighbor. The Linklayer Address column shows the MAC address of the neighbor. The Netif column displays the network interface this host is attached to, while the Expire column shows when the cached entry will expire. The S (state) column shows further information about the entry. A state of R means the host is reachable, while an I (incomplete) means the host is unreachable. The only Flags entry you’re likely to see is R, indicating this host is advertising itself as a router. For more states and flags, see ndp(8).
Why two separate commands? Both arp(8) and ndp(8) exist to map IP addresses to MAC addresses. Some hosts might be available only via one protocol or the other. IPv6-only hosts will not show up in your ARP table, and IPv4-only hosts will not appear in the NDP table.
For both arp(8) and ndp(8), the -n flag turns off hostname lookups. This is highly useful when you debug network issues and can’t get DNS resolution.
Now that you know how the network works, configuring an internet connection is pretty straightforward.