INTRODUCTION
Welcome to Absolute FreeBSD! This book is a one-stop shop for system administrators who want to build, configure, and manage FreeBSD servers. It will also be useful for those folks who want to run FreeBSD on their desktops, embedded devices, server farms, and so on. By the time you finish this book, you should be able to use FreeBSD to provide network services. You should also understand how to manage, patch, and maintain your FreeBSD systems and have a basic understanding of networking, system security, and software management. We’ll discuss FreeBSD versions 11 and 12, which are the most recent versions at the time this book is being released; however, most of this book applies to earlier and later versions as well.
What Is FreeBSD?
FreeBSD is a freely available Unix-like operating system popular with internet service providers, in appliances and embedded systems, and anywhere that reliability on commodity hardware is paramount. One day last week, FreeBSD miraculously appeared on the internet, fully formed, extruded directly from the mutant brain of its heroic creator’s lofty intellect. Just kidding—the truth is far more impressive. FreeBSD is a result of almost four decades of continuous development, research, and refinement. The story of FreeBSD begins in 1979, with BSD.
BSD: FreeBSD’s Granddaddy
Many years ago, AT&T needed a lot of specialized, custom-written computer software to run its business. It wasn’t allowed to compete in the computer industry, however, so it couldn’t sell its software. Instead, AT&T licensed various pieces of software and the source code for that software to universities at low, low prices. The universities could save money by using this software instead of commercial equivalents with pricey licenses, and university students with access to this nifty technology could read the source code to see how everything worked. In return, AT&T got exposure, some pocket change, and a generation of computer scientists who had cut their teeth on AT&T technology. Everyone got something out of the deal. The best-known software distributed under this licensing plan was Unix.
Compared with modern operating systems, the original Unix had a lot of problems. Thousands of students had access to its source code, however, and hundreds of teachers needed interesting projects for their students. If a program behaved oddly, or if the operating system itself had a problem, the people who lived with the system on a day-to-day basis had the tools and the motivation to fix it. Their efforts quickly improved Unix and created many features we now take for granted. Students added the ability to control running processes, also known as job control. The Unix S51K filesystem made system administrators bawl like exhausted toddlers, so they replaced it with the Fast File System (FFS), whose features have spread into every modern filesystem. Many small, useful programs were written over the years, gradually replacing entire swaths of Unix.
The Computer Systems Research Group (CSRG) at the University of California, Berkeley, participated in these improvements and also acted as a central clearinghouse for Unix code improvements. CSRG collected changes from other universities, evaluated them, packaged them, and distributed the compilation for free to anyone with a valid AT&T UNIX license. The CSRG also contracted with the Defense Advanced Research Projects Agency (DARPA) to implement various features in Unix, such as TCP/IP. The resulting collection of software came to be known as the Berkeley Software Distribution, or BSD.
BSD users took the software, improved it further, and then fed their enhancements back into BSD. Today, we consider this to be a fairly standard way for an open source project to run, but in 1979 it was revolutionary. BSD was also quite successful; if you check the copyright statement on an old BSD system, you’ll see this:
Copyright 1979, 1980, 1983, 1986, 1988, 1989, 1991, 1992, 1993, 1994
The Regents of the University of California. All rights reserved.
Yep, 15 years of work—a lifetime in software development. How many other pieces of software are not only still in use, but still in active development, 15 years after work began? In fact, so many enhancements and improvements went into BSD that the CSRG found that over the years, it had replaced almost all of the original Unix with code created by the CSRG and its contributors. You had to look hard to find any original AT&T code.
Eventually, the CSRG’s funding ebbed, and it became clear that the BSD project would end. After some political wrangling within the University of California, in 1992 the BSD code was released to the general public under what became known as the BSD license.
The BSD License
BSD code is available for anyone to use under what is probably the most liberal license in the history of software development. The license can be summarized as follows:
- Don’t claim you wrote this.
- Don’t blame us if it breaks.
- Don’t use our name to promote your product.
This means that you can do almost anything you want with BSD code. (The original BSD license did require that users be notified if a software product included BSD-licensed code, but that requirement was later dropped.) There’s not even a requirement that you share your changes with the original authors! People were free to take BSD and include it in proprietary products, open source products, or free products—they could even print it out on punch cards and cover the lawn with it. You want to run off 10,000 BSD CDs and distribute them to your friends? Enjoy. Instead of copyright, the BSD license is sometimes referred to as copycenter, as in Take this down to the copy center and run off a few for yourself. Not surprisingly, companies such as Sun Microsystems jumped right on it: it was free, it worked, and plenty of new graduates had experience with the technology—including Bill Joy, one of Sun’s founders. One company, BSDi, was formed specifically to take advantage of BSD Unix.
The AT&T/CSRG/BSDi Iron Cage Match
At AT&T, UNIX work continued apace even as the CSRG went on its merry way. AT&T took parts of the BSD Unix distribution, integrated them with its UNIX, and then relicensed the result back to the universities that provided those improvements. This worked well for AT&T until the company was broken up and the resulting companies were permitted to compete in the computer software business. AT&T had one particularly valuable property: a high-end operating system that had been extensively debugged by thousands of people. This operating system had many useful features, such as a variety of small but powerful commands, a modern filesystem, job control, and TCP/IP. AT&T started a subsidiary, Unix Systems Laboratories (USL), which happily started selling Unix to enterprises and charging very high fees for it, all the while maintaining the university relationship that had given it such an advanced operating system in the first place.
Berkeley’s public release of the BSD code in 1992 was met with great displeasure from USL. Almost immediately, USL sued the university and the software companies that had taken advantage of the software, particularly BSDi. The University of California claimed that the CSRG had compiled BSD from thousands of third-party contributors unrelated to AT&T, and so it was the CSRG’s intellectual property to dispose of as it saw fit.
This lawsuit motivated many people to grab a copy of BSD to see what all the fuss was about, while others started building products on top of it. One of these products was 386BSD, which would eventually be used as the core of FreeBSD 1.0.
In 1994, after two years of legal wrangling, the University of California lawyers proved that the majority of AT&T UNIX was actually taken in its entirety from BSD, rather than the other way around. To add insult to injury, AT&T had actually violated the BSD license by stripping the CSRG copyright from files it had assimilated. (Only a very special company can violate the world’s most generous software license!) A half-dozen files were the only sources of contention, and to resolve these outstanding issues, USL donated some of them to BSD while retaining some as proprietary information.
Once the dust settled, a new version of BSD Unix was released to the world as BSD 4.4-Lite. A subsequent update, BSD 4.4-Lite2, is the grandfather of the current FreeBSD, as well as ancestor to every other BSD variant in use today.
The Birth of FreeBSD
One early result of BSD was 386BSD, a version of BSD designed to run on the cheap 386 processor.1 The 386BSD project successfully ported BSD to Intel’s 386 processor, but it stalled. After a period of neglect, a group of 386BSD users decided to branch out on their own and create FreeBSD so they could keep the operating system up to date. (Several other groups started their own branches off of 386BSD around the same time, of which only NetBSD remains.)
386BSD and FreeBSD 1 were derived from 1992’s BSD release, the subject of AT&T’s wrath. As a result of the lawsuit, all users of the original BSD were requested to base any further work on BSD 4.4-Lite2. BSD 4.4-Lite2 was not a complete operating system—in particular, those few files AT&T had retained as proprietary were vital to the system’s function. (After all, if those files hadn’t been vital, AT&T wouldn’t have bothered!) The FreeBSD development team worked frantically to replace those missing files, and FreeBSD 2.0 was released shortly afterward. Development has continued ever since.
Today, FreeBSD is used across the internet by some of the most vital and visible internet-oriented companies. Netflix’s content delivery system runs entirely on FreeBSD. IBM, Dell/EMC, Juniper, NetApp, Sony and many other hardware companies use FreeBSD in embedded systems where you’d never even know it unless someone told you. The fact is, if a company needs to pump serious internet bandwidth, it’s probably running FreeBSD or one of its BSD relatives.
FreeBSD also finds its way into all sorts of embedded and dedicated-purpose devices. Do you have a PlayStation 4? Congratulations, you’re running FreeBSD. I hear a root shell is hard to get on one of them, though.
Like smog, spiders, and corn syrup, FreeBSD is all around you; you simply don’t see it because FreeBSD just works. The key to FreeBSD’s reliability is the development team and user community—which are really the same thing.
FreeBSD Development
There’s an old saying that managing programmers is like herding cats. Despite the fact that the FreeBSD development team is scattered across the world and speaks dozens of languages, for the most part, the members work well together as parts of the FreeBSD community. They’re more like a pride of lions than a collection of house cats. Unlike some other projects, all FreeBSD development happens in public. Three groups of people are responsible for FreeBSD’s progress: committers, contributors, and users.
Committers
FreeBSD has about 500 developers, or committers. Committers have read-and-write access to the FreeBSD master source code repository and can develop, debug, or enhance any piece of the system. (The term committer comes from their ability to commit changes to the source code.) Because these commits can break the operating system in both subtle and obvious ways, committers carry a heavy responsibility. Committers are responsible for keeping FreeBSD working or, at worst, not breaking it as they add new features and evaluate patches from contributors. Most of these developers are volunteers; only a handful are actually paid to do this painstaking work, and most of those people are paid only as it relates to other work. For example, Intel employs committers to ensure that FreeBSD properly supports its network cards. FreeBSD has a high profile in the internet’s heavy-lifting crowd, so Intel needs its cards to work on FreeBSD.
To plug yourself into the beehive of FreeBSD development, consider subscribing to the mailing list [email protected], which contains most of the technical discussion. Some of the technical talk is broken out into more specific mailing lists—for example, fine details of the networking implementation are discussed in [email protected].
Every few years, the committer team elects a small number of its members to serve as a core team, or Core. Core’s work is simultaneously vital, underrated, and misunderstood. Core is theoretically responsible for the overall management of FreeBSD, but in practice, it manages little other than resolving personality disputes and procedural conflicts among committers. Core also approves new committers and delegates responsibility for large parts of FreeBSD to individuals or groups. For example, it delegates authority over the ports and packages system to the ports management team. Core does not set architectural direction for FreeBSD, nor does it dictate processes or procedures; that’s up to the committers, who must agree en masse. Core does suggest, cajole, mediate, and inspire, however.
Core also experiences the worst part of management. Some of the key functions of management in a company are oversight, motivation, and handling problems between people. Oversight is provided by the millions of users who will complain loudly when anything breaks or behaves unexpectedly, and FreeBSD committers are self-motivated. The ugly part of management is settling squabbles between two people, and that’s the part Core gets stuck with. The status one gets from saying “I’m in Core” is an insufficient reward for having to manage the occasional argument between two talented developers who’ve gotten on each other’s nerves. Fortunately such disagreements are rare and usually resolved quickly.
Contributors
In addition to the committer team, FreeBSD has thousands of contributors. Contributors don’t have to worry about breaking the main operating system source code repository; they submit their patches for consideration by committers. Committers evaluate contributor submissions and decide what to accept and what to reject. A contributor who submits many high-quality patches is often asked to become a committer themselves.
For example, I spent several years contributing to FreeBSD whenever the urge struck me. Any time I feel that I’ve wasted my life, I can look at the FreeBSD website and see where my work was accepted by the committers and distributed to thousands of people. After I submitted the first edition of this book to the publisher, I spent my spare time submitting patches to the FreeBSD FAQ. Eventually, some members of the FreeBSD Documentation Project approached me and asked me to become a committer. As a reward, I got an email address and the opportunity to humiliate myself before thousands of people, once again demonstrating that no good deed goes unpunished.
If I had never contributed anything, I’d remain a user. Nothing’s wrong with that, either.
Users
Users are the people who run FreeBSD systems. It’s impossible to realistically estimate the number of FreeBSD users. While organizations such as the BSDstats Project (http://www.bsdstats.org/) make an effort, these projects are opt-in. They measure only folks who have installed FreeBSD and then installed the software that adds their system to the count. Most users download the whole of FreeBSD for free and never register, upgrade, or email a mailing list. We have no idea how many FreeBSD users are in the world.
Since FreeBSD is by far the most popular open source BSD, that’s not an inconsiderable number of machines. And since one FreeBSD server can handle hundreds of thousands of internet domains, a disproportionate number of sites use FreeBSD as their supporting operating system. This means that there are hundreds of thousands, if not millions, of FreeBSD system administrators out in the world today.
Other BSDs
FreeBSD might be the most popular BSD, but it’s not the only one. BSD 4.4-Lite2 spawned several different projects, each with its own focus and purpose. Those projects in turn had their own offspring, several of which thrive today.
NetBSD
NetBSD is similar to FreeBSD in many ways, and NetBSD and FreeBSD share developers and code. NetBSD’s main goal is to provide a secure and reliable operating system that can be ported to any hardware platform with minimal effort. As such, NetBSD runs on Vixens, PocketPC devices, and high-end SPARC and Alpha servers. I ran NetBSD on my HP Jornada handheld computer.2
OpenBSD
OpenBSD branched off from NetBSD in 1996 with the goal of becoming the most secure BSD. OpenBSD was the first to support hardware-accelerated cryptography, and its developers are rightfully proud of the fact that their default installation was largely immune to remote exploits for several years. The OpenBSD team has contributed several valuable pieces of software to the world, including the LibreSSL TLS library and the OpenSSH suite used by almost everyone from Linux to Microsoft.
DragonFly BSD
DragonFly BSD forked from FreeBSD 4 in 2003. It developed in a different direction than FreeBSD, with a new kernel messaging system. DragonFly BSD has very high performance and its HAMMER filesystem supports snapshots and fine-grained history. Check out http://www.dragonflybsd.org/ for more information.
macOS
Apple’s macOS? That’s right. Apple incorporates large chunks of FreeBSD into its macOS on an ongoing basis. If you’re looking for a stable operating system with a friendly face and a powerful core, macOS is unquestionably for you. While FreeBSD makes an excellent desktop for a computer professional, I wouldn’t put it in front of a random user. I would put macOS in front of that same random user without a second thought, however, and I’d even feel that I was doing the right thing. But macOS includes many things that aren’t at all necessary for an internet server, and it runs only on Apple hardware, so I don’t recommend it as an inexpensive general-purpose server.
FreeBSD’s Children
Several projects have taken FreeBSD and built other projects or products on top of it. The award-winning FreeNAS transforms a commodity system into a network fileserver. The pfSense project transforms your system into a firewall with a nice web management interface. TrueOS gives FreeBSD a friendly face while supporting resource-intensive advanced features, like ZFS, while GhostBSD puts a friendly face on equipment with less computing oomph. Other projects like this appear from time to time; while not all are successful, I’m sure by the time this book comes out, we’ll have one or two more solid members of this group.
Other Unixes
Several other operating systems derive from or emulate primordial Unix in one way or another. This list is by no means exhaustive, but I’ll touch on the high points.
Solaris
The best-known Unix might be Oracle Solaris. Solaris runs on high-end hardware that supports dozens of processors and gobs of disk. (Yes, gobs is a technical term, meaning more than you could possibly ever need, and I know very well that you need more disk than I think you need.) Solaris, especially early versions of Solaris, had strong BSD roots. Many enterprise-level applications run on Solaris. Solaris runs mainly on the SPARC hardware platform manufactured by Sun, which allows Sun to support interesting features, such as hot-swappable memory and mainboards.
The Oracle Corporation acquired Solaris when they bought Sun Microsystems in 2009. Oracle ceased Solaris development in 2016. While there’s still an extensive installed base of Solaris systems and you can still get Solaris from Oracle, as of today, Oracle Solaris has no future.
illumos
Several years before Oracle purchased Sun Microsystems, Sun open sourced the majority of Solaris and sponsored the OpenSolaris project to improve that codebase. OpenSolaris ran successfully until Oracle shut down source access and reclaimed all of the OpenSolaris resources.
The OpenSolaris code was still available, though. The OpenSolaris community forked OpenSolaris into illumos (http://illumos.org/). If you miss Solaris, you can still use a free, modern, Solaris-like operating system. FreeBSD includes two important features from OpenSolaris, the Zetabyte Filesystem (ZFS) and DTrace, a full-system tracing system.
AIX
Another Unix contender is IBM’s entry, AIX. AIX’s main claim to fame is its journaling filesystem, which records all disk transactions as they happen and allows for fast recovery from a crash. It was also IBM’s standard Unix for many years, and anything backed by Big Blue shows up all over the place. AIX started life based on BSD, but AT&T has twiddled just about everything so that you won’t find much BSD today.
Linux
Linux is a close cousin of Unix, written from the ground up. Linux is similar to FreeBSD in many ways, though FreeBSD has a much longer heritage and is friendlier to commercial use than Linux. Linux includes a requirement that any user who distributes Linux must make his or her changes available to the end user, while BSD has no such restriction. Of course, a Linux fan would say, “FreeBSD is more vulnerable to commercial exploitation than Linux.” Linux developers believe in share-and-share-alike, while BSD developers offer a no-strings-attached gift to everyone. It all depends on what’s important to you.
Many new Unix users have a perception of conflict between the BSD and Linux camps. If you dig a little deeper, however, you’ll find that most of the developers of these operating systems communicate and cooperate in a friendly and open manner. It’s just a hard fringe of users and developers that generate friction, much like different soccer teams’ hooligans or different Star Trek series’ fans.3
Other Unixes
Many Unixes have come and gone, while others stagger on. Past contenders include Silicon Graphics’ IRIX, Hewlett-Packard’s HP/UX, Tru64 Unix, and the suicidal SCO Group’s UnixWare. Dig further and you’ll find older castoffs, including Apple’s A/UX and Microsoft’s Xenix. (Yes, Microsoft was a licensed Unix vendor, back in that age when dinosaurs watched the skies nervously and my dad hunted mammoth for all the tribal rituals.) Many high-end applications are designed to run best on one particular flavor of Unix. All modern Unixes have learned lessons from these older operating systems, and today’s Unixes and Unix-like operating systems are remarkably similar.
FreeBSD’s Strengths
After all this, what makes FreeBSD unique?
Portability
The FreeBSD Project’s goal is to provide a freely redistributable, stable, and secure operating system that runs on the computer hardware that people are most likely to have access to. People have ported FreeBSD to a variety of less popular platforms as well.
The best supported FreeBSD platform is the common 64-bit hardware developed by AMD, used by almost everyone, and even copied by Intel. FreeBSD also fully supports the older 32-bit computers, such as 486s and all the flavors of Pentiums. This book uses 64-bit commodity hardware, or amd64, as a reference platform.
FreeBSD runs well on several other hardware architectures but is not completely supported yet. These include 32-bit ARM processors and PowerPC. While these other platforms are not afterthoughts, they don’t receive the same level of attention that x86 and amd64 do. The 64-bit ARM platform is expected to become Tier 1 shortly after this book comes out, however.
You can also load FreeBSD on certain older architectures, such as 64-bit SPARC. These platforms were once well supported but are on their way out.
Power
Since FreeBSD runs adequately on 486 processors, it runs extremely well on modern computers. It’s rather nice to have an operating system that doesn’t demand 8 cores and 12 gigs of RAM just to run the user interface. As a result, you can actually dedicate your hardware to accomplishing real work rather than tasks you don’t care about. If you choose to run a pretty graphical interface with all sorts of spinning gewgaws and fancy whistles, FreeBSD will support you, and it won’t penalize you if you choose otherwise. FreeBSD will also support you on the latest n -CPU hardware.
Simplified Software Management
FreeBSD also simplifies software management through the packaging system and the Ports Collection. Traditionally, running software on a Unix-like system required a great deal of expertise. Packages and ports simplify this considerably by automating and documenting the install, uninstall, and configuration processes for thousands of software packages.
We discuss packages in Chapter 15 and ports in Chapter 16.
Customizable Builds
FreeBSD provides a painless upgrade procedure, but it also lets you precisely customize the operating system for your hardware. Companies like Apple do exactly this, but they control both the hardware and the software; FreeBSD pulls off the same trick on commodity hardware.
Advanced Filesystems
A filesystem is how information is stored on the physical disk—it’s what maps the file My Resume to a series of zeros and ones on a hard drive. FreeBSD includes two well-supported filesystems, UFS (Chapter 11) and ZFS (Chapter 12). UFS has been around for multiple decades and is highly damage-resistant. ZFS is younger but includes features such as network replication and self-healing.
Who Should Use FreeBSD?
While FreeBSD can be used as a powerful desktop or development machine, its history shows a strong bias toward network services: web, mail, file, and ancillary applications. FreeBSD is most famous for its strengths as an internet server, and it’s an excellent choice as an underlying platform for any network service. If major firms such as Netflix count on FreeBSD to provide reliable service, it will work as well for you.
If you’re thinking of running FreeBSD (or any Unix) on your desktop, you’ll need to understand how your computer works. FreeBSD is not your best choice if you need point-and-click simplicity. If that’s your goal, get a Mac so you can use the power of Unix when you need it and not worry about it the rest of the time. If you want to learn FreeBSD, though, running it on your desktop is the best way—as we’ll discuss later.
Who Should Run Another BSD?
NetBSD and OpenBSD are FreeBSD’s closest competitors. Unlike competitors in the commercial world, this competition is mostly friendly. FreeBSD, NetBSD, and OpenBSD freely share code and developers; some people even maintain the same subsystems in multiple operating systems.
If you want to use old or oddball hardware, NetBSD is a good choice for you. For several years, I ran NetBSD on an ancient SGI workstation that I used as a Domain Name System (DNS) and fileserver. It did the job well until the hardware finally released a cloud of smoke and stopped working.
OpenBSD has implemented an impressive variety of security features. Some of the tools are eventually integrated into FreeBSD, but that takes months or years. Some of the tools can never be duplicated in FreeBSD, however. If you have real security concerns and can use a Unix-like system without the feature set FreeBSD provides, consider OpenBSD. Take a look at my book Absolute OpenBSD (No Starch Press, 2013) for an introduction.
If you’re just experimenting to see what’s out there, any BSD is good!
Who Should Run a Proprietary Operating System?
Operating systems such as macOS, Windows, AIX, and their ilk are still quite popular, despite the open source operating systems gnawing at their market share. High-end enterprises are pretty tightly shackled to commercial operating systems. While this is slowly changing, you’re probably stuck with commercial operating systems in such environments. But slipping in an occasional FreeBSD machine to handle basic services, such as monitoring and department file serving, can make your life much easier at much lower cost. Companies like Dell/EMC/Isilon have built entire businesses using FreeBSD instead of commercial operating systems.
Of course, if the software you need runs only on a proprietary operating system, your choice is pretty clear. Still, always ask a vendor whether a FreeBSD version is available; you might be pleasantly surprised.
How to Read This Book
Many computer books are thick and heavy enough to stun an ox, if you have the strength to lift them high enough. Plus, they’re either encyclopedic in scope or so painfully detailed that they’re difficult to actually read. Do you really need to reference a screenshot when you’re told to click OK or accept the license agreement? And when was the last time you actually sat down to read the encyclopedia?
Absolute FreeBSD is a little different. It’s designed to be read once, from front to back. You can skip around if you want to, but each chapter builds on what comes before it. While this isn’t a small book, it’s smaller than many popular computer books. After you’ve read it once, it makes a decent reference.
If you’re a frequent buyer of computer books, please feel free to insert all that usual crud about “read a chapter at a time for best learning” and so on. I’m not going to coddle you—if you picked up this book, you either have two brain cells to rub together or you’re visiting someone who does. (If it’s the latter, hopefully your host is smart enough to take this book away from you before you learn enough to become dangerous.)
What Must You Know?
This book is aimed at the new Unix administrator. Three decades ago, the average Unix administrator had kernel programming experience and was working on their master’s degree in computer science. Even a decade ago, they were already a skilled Unix user with real programming skills and most of a bachelor’s degree in comp sci. Today, Unix-like operating systems are freely available, computers are cheaper than food, and even 12-year-old children can run Unix, read the source code, and learn enough to intimidate older folks. As such, I don’t expect you to know a huge amount about Unix before firing it up.
To use this book to its full potential, you need to have familiarity with some basic tasks, such as how to change directories, list files in a directory, and log in with a username and password. If you’re not familiar with basic commands and the Unix shell, I recommend you begin with a book like UNIX System Administration Handbook by Evi Nemeth and friends (Prentice Hall PTR, 2017). To make things easier on newer system administrators, I include the exact commands needed to produce the desired results. If you learn best by example, you should have everything you need right here.
You’ll also need to know something about computer hardware—not a huge amount, mind you, but something. It helps to know how to recognize a SATA cable. Your need for this knowledge depends on the hardware you’re using, but if you’re interested enough to pick up this book and read this far, you probably know enough.
For the New System Administrator
If you’re new to Unix, the best way to learn is to eat your own dog food. No, I’m not suggesting that you dine with Rover. If you ran a dog food company, you’d want to make a product that your own dog eats happily. If your dog turns his nose up at your latest recipe, you have a problem. The point here is that if you work with a tool or create something, you should actually use it. The same thing applies to any Unix-like operating system, including FreeBSD.
Desktop FreeBSD
If you’re serious about learning FreeBSD, I suggest wiping out the operating system on your main computer and running FreeBSD instead. No, not a desktop-oriented FreeBSD derivative like TrueOS or GhostBSD: run raw FreeBSD. Yes, I know, now that dog food doesn’t sound so bad. But learning an operating system is like learning a language; total immersion is the quickest and most powerful way to learn. That’s what I did, and today I can make a Unix-like system do anything I want. I’ve written entire books on a FreeBSD laptop, using the open source text editor XEmacs and the LibreOffice.org business suite. I’ve also used FreeBSD to watch movies, rip and listen to MP3s, balance my bank accounts, process my email, and surf the web. The desktop in my lab has a dozen animated BSD daemons running around the window manager, and I occasionally take a break to zap them with my mouse. If this doesn’t count as a Stupid Desktop Trick, I don’t know what does.4
Many Unix system administrators these days come from a Windows background. They’re beavering away in their little world when their manager swoops by and says, “You can handle one more system, can’t you? Glad to hear it! It’s a Unix box, by the way,” and then vanishes into the managerial ether. Once the new Unix administrator decides not to quit her job and start a fresh and exciting career as a whale necropsy technician, she tentatively pokes at the system. She learns that ls is like dir and that cd is the same on both platforms. She can learn the commands by rote, reading, and experience. What she can’t learn, coming from this background, is how a Unix machine thinks. Unix will not adjust to you; you must adjust to it. Windows and macOS require similar adjustments but hide them behind a glittering facade. With that in mind, let’s spend a little time learning how to think about Unix.
How to Think About Unix
These days, most Unix systems come with pretty GUIs out of the box, but they’re just eye candy. No matter how graphically delicious the desktop looks, the real work happens on the command line. The Unix command line is actually one of Unix’s strengths, and it’s responsible for its unparalleled flexibility.
Unix’s underlying philosophy is many small tools, each of which does a single job well. My mail server’s local programs directory (/usr/local/bin) has 262 programs in it. I installed every one of them, either directly or indirectly. Most are small, simple programs that do only one task. This array of small tools makes Unix extremely flexible and adaptable. Many commercial software packages try to do everything; they wind up with all sorts of capabilities but only mediocre performance in their core functions. Remember, at one time you needed to be a programmer to use a Unix system, let alone run one. Programmers don’t mind building their own tools. The Unix concept of pipes encouraged this.
Pipes
People used to GUI environments, such as Windows and macOS, are probably unfamiliar with how Unix handles output and input. They’re used to clicking something and seeing either an OK message, an error, nothing, or (all too often) a pretty blue screen with nifty high-tech letters explaining in the language called Geek why the system crashed. Unix does things a little differently.
Unix programs have three channels of communication, or pipes: standard input, standard output, and standard error. Once you understand how each of these pipes works, you’re a good way along to understanding the whole system.
Standard input is the source of information. When you’re at the console typing a command, the standard input is the data coming from the keyboard. If a program is listening to the network, the standard input is the network. Many programs can rearrange standard input to accept data from the network, a file, another program, the keyboard, or any other source.
The standard output is where the program’s output is displayed. This is frequently the console (screen). Network programs usually return their output to the network. Programs might send their output to a file, to another program, over the network, or anywhere else available to the computer.
Finally, standard error is where the program sends its error messages. Frequently, console programs return their errors to the console; others log errors in a file. If you set up a program incorrectly, it just might discard all error information.
These three pipes can be arbitrarily arranged, a concept that’s perhaps the biggest hurdle for new Unix users and administrators. For example, if you don’t like the error messages appearing on the terminal, you can redirect them to a file. If you don’t want to repeatedly type a lot of information into a command, you can put the information into a file (so you can reuse it) and dump the file into the command’s standard input. Or, better still, you can run a command to generate that information and put it in a file, or just pipe (send) the output of the first command directly to the second, without even bothering with a file.
Small Programs, Pipes, and the Command Line
Taken to their logical extreme, these input/output pipes and the variety of tools seem overwhelming. When I saw a sysadmin type something like the following during my initial Unix training session, I gave serious consideration to changing careers.
$ tail -f /var/log/messages | grep -v popper | grep -v named &
Lines of incomprehensible text began spilling across the screen, and they kept coming. And worse still, my mentor kept typing as gibberish poured out! If you’re from a point-and-click computing environment, a long string of commands like this is definitely intimidating. What do all those funky words mean? And an ampersand? You want me to learn what?
Think of learning to use the command line as learning a language. When learning a language, we start with simple words. As we increase our vocabulary, we also learn how to string the words together. We learn that placing words in a certain order makes sense, and that a different order makes no sense at all. You didn’t speak that well at three years old—give yourself some slack and you’ll get there.
Small, simple programs and pipes provide almost unlimited flexibility. Have you ever wished you could use a function from one program in another program? By using a variety of smaller programs and arranging the inputs and outputs as you like, you can make a Unix system behave in any manner that amuses you. Eventually, you’ll feel positively hogtied if you can’t just run a command’s output through | sort -rnk 6 | less.5
Everything Is a File
You can’t be around Unix for very long before hearing that everything is a file. Programs, account information, and system configuration are all stored in files. Unix has no Windows-style registry; if you back up the files, you have the whole system.
What’s more, the system identifies system hardware as files! Your CD-ROM drive is a file, /dev/cd0. Serial ports appear as files like /dev/cuaa0. Even virtual devices, such as packet sniffers and partitions on hard drives, are files.
When you have a problem, keep this fact in mind. Everything is a file, or is in a file, somewhere on your system. All you have to do is find it!
Notes on the Third Edition
Absolute BSD (No Starch Press, 2002) was my first technology book and was written when the various BSD operating systems had more in common than they wanted to admit. The second edition, Absolute FreeBSD (No Starch Press, 2007), came out after the BSDs had diverged, and detailed FreeBSD’s advances in the previous five years. With another decade of growth, FreeBSD has evolved to compete with the best commercial operating systems. You’ll find multiple top-tier filesystems. Disk management has changed to accommodate new partitioning methods. Virtualization is now a thing, and FreeBSD supports it as either a client or a host.
This growth has driven changes in this book.
We won’t discuss configuring mail, DNS, or web servers. You have more software choices for these tasks than ever before. Entire books have been written about those choices and how to use them. I’ve written some of those books. Those topics have been dropped to make space for FreeBSD-specific material, like ZFS and jails.
Some of these new features are hugely complex. Complete coverage of ZFS would fill entire books—I know, because I’ve written those books, too. FreeBSD supports a whole bunch of special-purpose filesystems, each incredibly useful to the folks who need them and totally irrelevant to those who don’t. Rather than write a monster tome that nobody would actually read, I’ve elected to cover the material that every FreeBSD sysadmin must know. If you’re interested in deeper coverage of a particular topic, it’s available.
Some subsystems are undergoing radical revision. I could wait to write this book until every FreeBSD subsystem has a stable interface, but then it would come out about . . . never. As I write this, the bhyve developers are actively rototilling their entire configuration system. Given the choice between glossing over a topic and providing flat-out wrong material, I’ve chosen to skip detail on bhyve. I hope to be able to delete this paragraph before this book goes to press.
I’ve ruthlessly excised obsolete information from this edition. For example, modern disk drives don’t generally have to worry about write caching. If you discover that a piece of advice you remember using doesn’t appear in this book, please check FreeBSD’s information resources to see whether that advice is still applicable.
Contents of This Book
Absolute FreeBSD, 3rd Edition contains the following chapters.
This chapter discusses the information resources the FreeBSD Project and its devotees provide for users. No one book can cover everything, but knowing how to use the many FreeBSD resources on the internet helps fill any gaps you find here.
Getting FreeBSD installed isn’t that hard. Make poor choices during the install, though, and you’ll have a system that isn’t suited for your needs. The best way to avoid reinstalling is to think about your requirements and make all the decisions beforehand so that the actual install doesn’t require any thought.
This chapter gives you an overview of installing FreeBSD using different partitioning schemes and filesystems.
Chapter 4: Start Me Up! The Boot Process
This chapter teaches you about the FreeBSD boot process and how to make your system start, stop, and reboot in different configurations.
Chapter 5: Read This Before You Break Something Else! (Backup and Recovery)
Here we discuss how to back up your data on both a system-wide and a file-by-file level, and how to make your changes so that they can be easily undone.
This chapter describes configuring the FreeBSD kernel. Unlike some other operating systems, you’re expected to tune FreeBSD’s kernel to best suit your purposes. This gives you tremendous flexibility and lets you optimize your hardware’s potential.
Here we discuss the TCP/IP protocol that underlies the modern internet, both version 4 and version 6.
Chapter 8: Configuring the Network
FreeBSD doesn’t only shuffle packets crazy fast, but it also supports virtual LANs, link aggregation, and more. We’ll configure all of that here.
Chapter 9: Securing Your System
This chapter teaches you how to make your computer resist attackers and intruders.
Chapter 10: Disks, Partitioning, and GEOM
This chapter covers some of the details of working with hard drives in FreeBSD. Working with modern hardware means understanding multiple partitioning schemes, disk alignment, and FreeBSD’s disk management infrastructure.
Chapter 11: The Unix File System
UFS has been FreeBSD’s standard filesystem for decades, and the concepts of UFS pervade the whole operating system. Whether you intend to use UFS or not, you must understand its essentials.
ZFS is a newer filesystem very popular on larger systems. If you’re managing large amounts of data, you’ll want ZFS.
Chapter 13: Foreign Filesystems
Every sysadmin needs to mount disks over the network or use ISOs without burning them to CD. This chapter takes you through those duties, as well as introducing FreeBSD-specific filesystems like devfs.
This chapter describes the many configuration files in FreeBSD and how they operate.
Chapter 15: Making Your System Useful
Here I describe the packages system that FreeBSD uses to manage add-on software.
Chapter 16: Customizing Software with Ports
Sometimes the prebuilt packages won’t cover everything you need. You can leverage FreeBSD’s package-building system to create your own software packages, tuned to meet your exact needs.
Chapter 17: Advanced Software Management
This chapter discusses some of the finer points of running software on FreeBSD systems.
This chapter teaches you how to use FreeBSD’s upgrade process. The upgrade system is among the most remarkable and smooth of any operating system.
Chapter 19: Advanced Security Features
Here we discuss some of the more interesting security features found in FreeBSD.
Chapter 20: Small System Services
Here we discuss some of the small programs you’ll need to manage in order to use FreeBSD properly.
Chapter 21: System Performance and Monitoring
This chapter covers some of FreeBSD’s performance-testing and troubleshooting tools and shows you how to interpret the results. We also discuss logging and FreeBSD’s SNMP implementation.
FreeBSD has a process-isolation subsystem, much like Linux and Solaris containers, called jails. We’ll cover the jail system and how you can leverage it for system security.
Chapter 23: The Fringe of FreeBSD
This chapter teaches you some of the more interesting tricks you can do with FreeBSD, such as running systems without disks and with tiny disks, as well as cloud-friendly features, like libxo.
Chapter 24: Problem Reports and Panics
This chapter teaches you how to deal with those rare occasions when a FreeBSD system fails, how to debug problems, and how to create a useful problem report.
You’ll also find an annotated bibliography, an afterword, and a really spiffy professionally prepared index.
Okay, enough introductory stuff. Onward!