23
THE FRINGE OF FREEBSD
If you hang around the FreeBSD community for any length of time, you’ll hear mention of all sorts of things that can be done if you know how. People build embedded FreeBSD devices and ship them to customers all over the world, who don’t even know that they have a Unix-like server inside the little box running their air conditioner or radio relay station. People run FreeBSD on machines without hard drives, supporting hundreds or thousands of diskless workstations from a single server. You’ll find bootable CDs and USB devices that contain complete FreeBSD systems, including all the installed software you could ever want. These things aren’t difficult to do, once you know the tricks.
In this chapter, we’re headed into the fringes of FreeBSD—the really cool things that are done by FreeBSD users but aren’t necessarily supported by the mainstream FreeBSD Project. While you can find support and assistance through the usual channels, you must be prepared to debug and troubleshoot everything in this chapter even more than usual.
Terminals
A terminal is the device that people can log in on. The keyboard, video, and mouse make up a terminal that’s also called a console. When you SSH into your host, it provides a virtual terminal. Terminal configuration is overwhelmingly automatic, but you might need to tweak it.
The file /etc/ttys controls how and where users may log into your FreeBSD system. Do console logins work? How about virtual terminals? What about logging in over serial lines? FreeBSD systems offer four standard terminals: the console, virtual terminals, dial-up terminals, and pseudoterminals.
The console is the only device available in single-user mode. On most FreeBSD systems, this is either a video console that includes the monitor and keyboard or a serial console accessed from another system. Once the system hits multiuser mode, the console is usually attached to a virtual terminal instead. The console device is /dev/console.
A virtual terminal is attached to the physical monitor and keyboard. You can have multiple terminals on your one physical terminal. Switch between them with ALT and the function keys. The next time you’re at the keyboard, hit ALT-F2. You’ll see a fresh login screen, with ttyv1 after the hostname. This is the second virtual terminal. Hitting ALT-F1 takes you back to the main virtual terminal. By default, FreeBSD has eight virtual terminals and reserves a ninth for X Windows. You can use the eight virtual text terminals even when you’re in X, and some X desktops provide multiple X virtual terminals. The virtual terminals are the /dev/ttyv devices.
A dial-up terminal is connected via serial line. You can attach modems directly to your serial ports and let users dial into your server. This isn’t so common these days, but the same functionality supports logging in over a serial console. Dial-up terminals are the /dev/ttyu devices.
Finally, a pseudoterminal is implemented entirely in software. When you SSH into your server, you don’t need any actual hardware, but the software still needs a device node for your session. Pseudoterminals are the device nodes in /dev/pts/. You don’t configure pseudoterminals; they’re automatically negotiated when you log in.
Configure access to the console, virtual terminals, and dial-up terminals in /etc/ttys. You can enable serial access, require or disable passwords, and more.
/etc/ttys Format
A typical entry in /etc/ttys looks like this:
ttyv0 "/usr/libexec/getty Pc" xterm on secure
The first field is the terminal’s device node. In this case, ttyv0 is the first virtual terminal on the system.
The second field is the program that’s spawned to process login requests on this terminal. FreeBSD uses getty(8), but if you have a preferred terminal management program, you can use it instead. You’ll find several in packages. This field takes one argument, the terminal configuration. The file /etc/gettytab contains all the terminal configurations.
The third entry is the terminal type. The file /etc/termcap describes all the innumerable terminal types FreeBSD supports. For really small systems, FreeBSD provides /etc/termcap.small with only the most vital entries. Almost everything modern works with either xterm or vt100.
The fourth entry determines whether the terminal is available for logins or not. This could be on for accepting logins or off for not allowing them. The onifconsole setting permits logins on a serial port if the kernel configured the port as a console.
Last, we have the options. This example has the option secure set, which tells getty(8) that root may log into this console.
Offering terminals is a low-level system task handled directly by init(8). Changes to /etc/ttys don’t take effect until you tell init(8) to reread its configuration file. Init is always PID 1.
# kill -1 1
Insecure Console
When you boot FreeBSD in single-user mode, you get a root command prompt. This is fine for your laptop and works nicely for servers in your corporate data center, but what about machines in untrusted facilities? If you have a server in a colocation center, for example, you probably don’t want just anyone to be able to get root-level access to a machine. You can tell FreeBSD that the physical console is insecure and make it require the root password to enter single-user mode. The system will then boot from power-on to multiuser mode without requiring a password, but it’ll require the password when you explicitly boot in single-user mode.
Requiring a password in single-user mode doesn’t completely protect your data, but it does raise the bar considerably. A lone tech working late, when nobody’s looking, could boot your system into single-user mode and add an account for himself in only 15 minutes or so. Dismantling your machine, removing the hard drives, mounting them into another machine, making changes, and bringing your server back online requires much more time, is far more intrusive, and is much more likely to be noticed by colocation management.
Find the console entry in /etc/ttys:
console none unknown off secure
You’ll see that the console terminal isn’t as full-featured as other terminals; it doesn’t run getty(8) and uses the generic unknown terminal type. The console is intended for use only in single-user mode and when attached to a physical terminal, however, so that’s fine.
To make the console require a root login when booted into single-user mode, change secure to insecure.
console none unknown off insecure
Password-protecting the console dissuades casual mischief. It won’t even slow a knowledgeable intruder with physical access to the machine.
Managing Cloudy FreeBSD
Clusters of hundreds or thousands of servers are growing increasingly common. Automation systems like Ansible and Puppet somehow let us maintain these systems in a semblance of order. Unix wasn’t designed to be operated that way, however. Primordial UNIX was written to be administered by a highly skilled operator who had no problem handling the vagaries of countless different command output formats and even more configuration file styles.
FreeBSD is attacking the problems of cloud-scale management with libXo and universal configuration language (UCL).
LibXo
While automated monitoring is a necessity and can alert you to issues, when it comes to in-depth troubleshooting, nothing replaces logging into a host, running a command, and interpreting the result. I’ve lost track of how many scripts I’ve written to parse the output of some obscure combination of ps(1) flags so that I could feed a number to the monitoring software. I’ve also lost track of how many hours I’ve spent debugging those scripts or explaining why the script I wrote to process one netstat(1) flag is irrelevant to the flags we’re interested in right now.1 Multiply this by those hundreds or thousands of servers, and getting information out of software quickly becomes a serious problem.
FreeBSD has cut down this problem space with libXo.
LibXo is a library that helps commands provide output not only in text form but also in XML, JSON, and even HTML. Instead of using grep(1) and awk(1) and whatever appalling combination of shell or Perl or Python you’ve brewed up to find desired information, you can have a parser extract data from a tagged format. You can dump command output straight to a web page.
Not all programs support libXo, but support is continually added to more programs. The man page declares whether a program supports libXo, but if you’re too lazy to read it, you can try the command with the --libxo flag. All commands that support libXo use that command line option. You must also specify the output format, either text, XML, JSON, or HTML. Here, I run arp -an and identify JSON as the output format.
$ arp -an --libxo json
{"__version": "1", "arp": {"arp-cache": [{"hostname":"?","ip-address":"
203.0.113.221","mac-address":"08:00:27:31:91:0d","interface":"em0",
"permanent":true,"type":"ethernet"}
--snip--
How do you use this? Many of us won’t. But if you’re running dozens or hundreds of servers, you probably have the expertise in-house to painlessly parse this. Hundreds of tools can select tagged data, and your application developer probably has their preferred software already installed on your hosts. And while the output of arp(8) is fairly consistent, libXo also handles any arbitrary combination of flags to netstat(1), vmstat(8), and more. Learn to grab tagged data from the output once, and you’re done writing those horrible scripts forever.
Universal Configuration Language
Unix systems have a pretty standard configuration file format. Hash marks are comments. There are variables. Maybe the presence of the variable in a config file is enough to activate a feature, or perhaps you have to set the variable to a value. They’re all a little bit different, though. Some programs can pull in configuration snippets from a primary file and the files in a directory, like cron(8) does with /etc/crontab and /etc/cron.d/. Others can’t. Some use braces to set aside chunks of configuration, where others use . . . whatever the programmer thought was a good idea 30 years ago. The result is that nobody looks at syslog.conf and thinks it looks like pkg.conf, even though they share common underlying concepts.
The universal configuration language (UCL) aims to change that. If all of these programs have a similar syntax, why not use a single parsing library for each? And if you have a parsing library, why not let it parse multiple formats? UCL lets you provide configuration files in classic Unix style, JSON, or YAML, ideal for automated management. It can extract configuration settings in shell code, UCL, JSON, or YAML.
At the time I write this, FreeBSD uses UCL for pkg(8). Support for other utilities, such as bhyve(8), is slowly happening. If you’re managing large numbers of servers, check to see the status of UCL in your release.
Diskless FreeBSD
While FreeBSD isn’t difficult to manage, dozens or hundreds of nearly identical systems can become quite a burden. One way to reduce your maintenance overhead is to use diskless systems. Diskless systems aren’t forbidden to have hard drives; rather, they load their kernel and operating system from an NFS server elsewhere on the network.
Why use a diskless system for your server farm? Multiple systems can boot off of a single NFS server, centralizing all patch and package management. This is excellent for collections of terminals, computation clusters, and other environments where you have large numbers of identical systems. Rolling out an operating system update becomes a simple matter of replacing files on the NFS server. Similarly, when you discover that an update has problems, reverting it is as simple as restoring files on the NFS server. In either case, the only thing you have to do at the client side is reboot. As the clients have read-only access to the server, untrusted users can’t make any changes to the operating system. If you have only a couple of systems running, diskless is probably too much work for you, but any more than that and diskless is a clear winner.
Before you can run diskless systems, you must have an NFS server, a DHCP server, a TFTP server, and hardware that supports diskless booting. Let’s go through each and see how to set it up.
Diskless Clients
Machines that run diskless must have enough smarts to find their boot loader and operating system over the network. There are two standard ways of doing this: BOOTP and PXE. BOOTP, the internet Bootstrap Protocol, is an older standard that fell out of favor long ago. PXE, Intel’s Preboot Execution Environment, has been supported on almost every new machine for years now, so we’ll concentrate on that.
Boot your diskless client machine and go into the BIOS setup. Somewhere in the BIOS, you’ll find an option to set the boot device order. If the machine supports PXE, one of those options will be the network. Enable that option and have the machine try it first.
Your diskless client is ready. Now let’s get the server ready.
DHCP Server Setup
While most people think of DHCP as a way to assign IP addresses to clients, it can provide much more than that. You can configure your DHCP server to provide the locations of a TFTP server, an NFS server, and other network resources. Diskless systems make extensive use of DHCP, and you’ll find that we use DHCP options you’ve never tried before.
OpenBSD’s DHCP server won’t support FreeBSD diskless clients; you must use ISC’s DHCP server or some other more full-featured version. Configuring the ISC DHCP server to handle diskless systems is pretty straightforward once you have the MAC address of your diskless workstation.
MAC Address
To assign configuration information to a DHCP client, you need the MAC address of that client’s network card. Some BIOS implementations provide the MAC addresses of integrated network cards, and some server-grade hardware has labels with the MAC address printed on them. Those options, however, are too easy, so we’ll try the hard way.
When a machine tries to boot off the network, it makes a DHCP request for its configuration information. While you don’t have a diskless configuration yet, any DHCP server logs the MAC address of clients. You can get the client information from the leases file, /var/db/dhcpd.leases.
--snip--
➊ lease 198.51.100.10 {
starts 6 2017/09/16 06:57:23;
ends 6 2017/09/16 07:07:23;
--snip--
➋ hardware ethernet 08:00:27:d8:c1:1c;
uid " 01 10 00'330301 34";
set vendor-class-identifier = "PXEClient:Arch:00000:UNDI:002001";
}
--snip--
This client has a MAC address of 08:00:27:d8:c1:1c ➋ and has been offered IP address 198.51.100.10 ➊. Given this information, we can create a DHCP configuration to assign this host a static IP address and provide its boot information.
DHCP Configuration: Specific Diskless Hosts
We configured basic DHCP services in Chapter 20. Here’s a sample dhcpd(8) configuration for a diskless client. This doesn’t go inside a subnet statement but is a top-level statement on its own, even if it’s on a subnet shared with nondiskless DHCP clients.
➊ group diskless {
➋ next-server 198.51.100.1;
➌ filename "pxeboot";
➍ option root-path "198.51.100.1:/diskless/1/";
➎ host compute1.mwl.io {
➏ hardware ethernet 08:00:27:d8:c1:1c ;
➐ fixed-address 198.51.100.101 ;
}
}
We define a group called diskless ➊. This definition will allow us to assign certain parameters to the group and then just add hosts to the group. Every host in the group gets those same parameters.
The next-server setting ➋ tells the DHCP clients the IP address of a TFTP server, and the filename option ➌ tells clients the name of the boot loader file to request from that TFTP server. Remember from Chapter 4 that the boot loader is the software that finds and loads the kernel. Finally, option root-path ➍ tells the boot loader where to find the root directory for this machine. All of these options and settings are given to all clients in the diskless group.
We then assign our diskless client to the diskless group using the host statement and the hostname of this system ➎. Our first client is called compute1. This client is identified by its MAC address ➏ and is assigned a static IP ➐. It also receives the standard configuration for this group.
Create additional host entries just like this for every diskless host on your network.
Restart dhcpd(8) to make this configuration take effect. Now reboot your diskless client. The DHCP log should show that you’ve offered this client its static address. However, the DHCP client can’t boot any further without a boot loader, which means you need a TFTP server.
DHCP Configuration: Diskless Farms
Perhaps you have a large number of identical diskless hosts, such as thin clients in a terminal room. It’s perfectly sensible not to want to make a static DHCP entry for each thin client. Let these hosts get their boot information from the DHCP server, but without specifying a host address. They’ll just take an address out of the DHCP pool. Many clustering solutions include client services that register new hosts with whatever “cluster manager” they’re using, so hardcoded addresses aren’t so important.
You can also specifically identify hosts that are requesting DHCP information from PXE and assign those hosts to a specific group of addresses. A host booting with PXE identifies itself to the DHCP server as a client of type PXEclient. You can write specific rules to match clients of that type and configure them appropriately. Look in the DHCP manual for information on how to match on vendor-class-identifier and dhcp-client-identifier.
tftpd and the Boot Loader
We covered configuring a TFTP server in Chapter 20. The TFTP server must provide the pxeboot file for your diskless clients. FreeBSD provides pxeboot in the /boot directory.
# cp /boot/pxeboot /tftpboot
# chmod +r /tftpboot/pxeboot
Try to download pxeboot via TFTP from your workstation. If that works, reboot your diskless client and watch it try to boot. The console should show a message like this:
Building the boot loader arguments
Relocating the loader and the BTX
Starting the BTX loader
You’ve seen this message before, when a regular FreeBSD boots off its hard drive. Your diskless client will identify the PXE version, print the memory, and declare that it’s running the bootstrap loader. At that point, it’ll circle endlessly trying to load the kernel. It can’t load the kernel because we haven’t yet set up the NFS server.
Diskless Security
Diskless systems run over NFS and have all of NFS’s security issues. Even if you deploy Kerberos to encrypt NFS traffic, the initial network boot and mounting of the root filesystem is always unencrypted. Don’t run diskless nodes on the open internet.
You can somewhat protect your NFS server by assigning a different user for the NFS root account. Running find /diskless/1 -user 0 -exec chown nfsroot {} ; changes the owner of all files owned by root to be owned by the user nfsroot. You can then edit the exports file to map root to the nfsroot user. You’d need to revert that to run freebsd-update(8), however, and then restore it after applying patches. But when you’re first learning, don’t get fancy. Get a basic userland working first.
The NFS Server and the Diskless Client Userland
Many tutorials on diskless operation suggest using the server’s userland and root partition for diskless clients. That might be easy to do, but it’s not even vaguely secure. Your diskless server probably has programs on it that you don’t want the clients to have access to, and it certainly has sensitive security information that you don’t want to hand out to a whole bunch of workstations. Providing a separate userland is a much wiser option.
While you can provide a separate userland in many ways, I find that the simplest is to slightly modify the jail(8) construction process from Chapter 22. First, make a dataset, UFS filesystem, or directory for our diskless clients to use as their root directory, and then install a userland and kernel in that directory. Extract the base.txz and kernel.txz distribution files for the version of FreeBSD in that directory.
# tar -xpf base.txz -C /diskless/1/
# tar -xpf kernel.txz -C /diskless/1/
If you’ve built a FreeBSD you want to run, that works too. Here, we install a locally built userland in /diskless/1:
# cd /usr/src
# make installworld DESTDIR=/diskless/1
# make installkernel DESTDIR=/diskless/1
# make distribution DESTDIR=/diskless/1
Now tell your NFS server about this directory. I intend to install several diskless systems on this network, so I offer this directory via NFS to my entire subnet. The clients don’t need write access to the NFS root, so I export it read-only. The following /etc/exports line does this:
/diskless/1 -ro -maproot=0 -alldirs -network 198.51.100.0 -mask 255.255.255.0
Restart mountd(8) to make this share available, and try to mount it from a workstation. Confirm that the directory contains a basic userland visible from the client and that clients can’t write to the filesystem.
Your diskless host needs a root password. Set it using chroot(8) and passwd(1).
# chroot /diskless/1/ passwd
You’ll need to tell the host that its root filesystem is read-only. Create /diskless/1/etc/rc.conf and set root_rw_mount to NO. While you’re in that directory, also create a resolv.conf for your client.
Now reboot your diskless client and see what happens. It should find the kernel and boot into an unconfigured multiuser mode. Depending on the server, client, and network speed, this might take a while to complete.
At this point, you could configure your userland to specifically match your single diskless client. You could make changes in /etc, such as creating /etc/fstab that reflects your needs, and copy password files into place. That suffices for one diskless client, but FreeBSD has infrastructure designed specifically to support dozens or hundreds of hosts off the same filesystem. Let’s look into how this is done.
Diskless Farm Configuration
One of the benefits of diskless systems is that multiple machines can share the same filesystem. However, even on machines that are mostly identical, you’ll probably find that you must make certain configuration files slightly different. FreeBSD includes a mechanism for offering personalized configuration files on top of a uniform userland by remounting directories on tmpfs(5) temporary filesystems and copying custom files to these partitions.
FreeBSD’s default diskless setup lets you configure diskless workstations across multiple networks and subnets—an invaluable feature on large networks. If you have only a few diskless systems, however, you might find it slightly cumbersome at first. Over time, however, you’ll find that you make more and more use of it. Diskless systems are a convenient solution to many problems.
A booting FreeBSD system uses the vfs.nfs.diskless_valid to see whether it’s running diskless. If the sysctl equals 0, it’s running off a hard drive; otherwise, it’s running diskless. On diskless systems, FreeBSD runs the /etc/rc.initdiskless script to parse and deploy the hierarchical diskless configuration.
Configuration Hierarchy
Configure your diskless farm in the diskless host’s /conf. The /conf directory can have a whole bunch of directories in it. The two critical ones are /conf/base and /conf/default, but you might also have separate directories for subnets and/or individual IP addresses. Diskless systems use the contents of these directories to build tmpfs filesystems on top of the mounted root partition so individual hosts can have unique settings and read-write filesystems. You can make any directory a tmpfs filesystem and populate it from this hierarchy, but every host needs a read-write /etc directory, so we’ll use that as our example.
The /conf/base directory contains base system files that need to be mounted read-write on the diskless client. Create /conf/base/etc and populate it with a set of /etc files, and the diskless host can use them as the base of its tmpfs /etc. (It can also recycle the diskless root’s /etc, as we’ll see later.)
The /conf/default directory contains defaults for your environment. Perhaps every host in your environment needs an /etc/fstab that directs it to mount a shared data store. You’d create /conf/defaults/etc/fstab, and the diskless system would copy that to every host on top of the base system from /conf/base/etc. I’d also distribute your environment’s generic rc.conf in the default directory.
You can also have per-subnet directories. Name that directory after the subnet’s broadcast address, the top address in the network. My diskless farm runs on the subnet 198.51.100.0/24, with a broadcast address of 198.51.100.255. If I created /conf/198.51.100.255/etc/rc.conf, every host in that subnet would get that rc.conf. If I had a special /etc/fstab for diskless hosts on that subnet, I could put it in /conf/198.51.100.255/etc/fstab and it would overwrite the default. I’d also add files in /etc/rc.conf.d/ for special services that run only on that subnet.
Finally, I could have per-host directories. If I created /conf/198.51.100.101/etc/rc.conf.d/apache, the host 198.51.100.101—and only that host—would get that file. If that particular host needed a truly unique /etc/fstab, I could put it in /conf/198.51.100.101/etc/fstab, and it would overwrite both the default and the subnet /etc/fstab.2
This hierarchical configuration gets deployed through a process called diskless remounting.
Diskless Remounting /etc
The diskless system checks the file /conf/base/etc/diskless_remount for a list of directories it should mount as memory filesystems. Without this file, no memory filesystems get created, and your diskless host shares a single read-only userland with all of the other diskless hosts. The diskless_remount file contains a list of filesystems to be remounted.
/etc
This tells FreeBSD to build an MFS /etc and copy the diskless root’s existing /etc onto it, giving us a base to work from.
You don’t necessarily want all of the files in the diskless root’s /etc on your diskless host’s /etc. It’s a memory filesystem, so why waste memory holding stuff you don’t need? You also don’t want to imply to junior sysadmins that the hosts support functions that they don’t. Diskless systems shouldn’t keep logs locally, so they don’t need newsyslog or /etc/newsyslog.conf. You don’t back up diskless clients, so /etc/dumpdates is also unnecessary. Browsing /etc will reveal quite a few files irrelevant to diskless hosts. If you remove too much, however, your system won’t boot, and the list of necessary files isn’t intuitive. For example, if you remove /etc/mtree, the machine will hang in single-user mode because it can’t repopulate the MFS /var partition.
Put the full paths to your unwanted files and directories in the file /conf/base/etc.remove. For example, the following entries remove the /etc/gss and /etc/bluetooth directories as well as the syslog and backup files discussed earlier. You don’t need to copy over /etc/resolv.conf. FreeBSD’s /etc/rc.d/resolv startup script creates one from the original DHCP response that booted the host.
/etc/gss
/etc/bluetooth
/etc/dumpdates
/etc/resolv.conf
/etc/newsyslog.conf
/etc/syslog.conf
Not so hard, is it?
Now let’s put some things back into our configuration.
Finalizing Setup
Now that you have an installed system, let’s do some fine-tuning. Diskless clients need third-party packages and assorted configuration files. The easiest and safest way to finish setting up your client is through using the chroot(8) program, which locks you into a subdirectory of the filesystem. By using chroot(8) on the NFS server, you can get read-write access to the filesystem almost exactly as it will exist on the diskless client.
# chroot /diskless/1
Yes, /etc still has hierarchical overrides, but other parts of the system exist exactly as the diskless client sees them. Any changes you make while chrooted will be coherent to the client.
Installing Packages
Use pkg(8) to install software on a diskless client. Use the -c flag to specify the diskless root directory and have pkg(8) chroot into it.
# pkg -c /diskless/1/ install pkg
You now have the package tools, database, and repository information on your diskless client.
# pkg -c /diskless/1/ install sudo
Install any software you need this way.
SSH Keys
Perhaps the most annoying thing about diskless clients is the host’s SSH keys. In normal operation, every host needs unique SSH keys. If you’re running on a private network, you might decide to have all the diskless clients share the same SSH key. You might decide to have each host autogenerate new SSH keys at boot time. As /etc exists on tmpfs, those keys will vanish at shutdown, but users will quickly grow accustomed to the “host key has changed” messages. That’s not something you want users to grow accustomed to, though.
Establishing persistent, unique host keys for each diskless client, however, isn’t hard. Create a /conf directory for each host.
# mkdir -p /diskless/1/conf/198.51.100.101/etc/ssh
# cd /diskless/1/conf/198.51.100.101/etc/ssh
In this directory, create the SSH keys for each algorithm your version of SSH uses. While ssh-keygen(1) includes the -A flag to autogenerate missing keys, it places those keys in /etc/ssh. That won’t work for your diskless userland or even in a chroot. You’ll need to create those keys the old-fashioned way.
# ssh-keygen -N "" -qt algorithm -f ssh_host_algorithm_key
You’ll need to substitute the name of the cryptographic algorithm twice, in lowercase. For example, here’s how you’d create a DSA SSH key:
# ssh-keygen -N "" -qt dsa -f ssh_host_dsa_key
Today, OpenSSH creates keys for RSA, ECDSA, and ED25519. Create each of those. Key creation is easily scriptable. See /etc/rc.d/sshd for examples.
Diskless clients let you easily run thousands of nearly identical machines. Now let’s look at protecting just one.
Storage Encryption
FreeBSD supports two different disk encryption methods, GBDE and GELI. Both tools work very differently, support different cryptographic algorithms, and are designed for different threat models. People talk about encrypting disks all the time, but you rarely hear discussions of what disk encryption is supposed to protect the disk from.
GBDE, or Geom-Based Disk Encryption, has specific features for high-security environments where protecting the user is just as important as concealing the data. In addition to a cryptographic key provided by the user, GBDE uses keys stored in particular sectors on the hard drive. If either key is unavailable, the partition can’t be decrypted. Why is this important? If a secure data center (say, in an embassy) comes under attack, the operator might have a moment or two to destroy the keys on the hard drive and render the data unrecoverable. If the bad guys have a gun to my head and tell me to “enter the passphrase or else,” I want the disk system to say, The passphrase is correct, but the keys have been destroyed. I don’t want a generic error saying, Cannot decrypt disk. In the first situation, I still have value as a blubbering hostage; in the latter, either I’m dead or the attackers get unpleasantly creative.3
GELI is much more flexible, but it won’t protect me from bodily harm the way GBDE might. If someone might steal my laptop for the confidential documents on it, or if an untrusted system user might snoop my swap space to steal secrets, GELI suffices. GELI doesn’t try to protect my person, just my data. As I won’t take any job that poses a higher than average risk of exposure to firearms (keeping in mind that I live in Detroit), that’s perfectly fine with me. GELI also uses FreeBSD’s cryptographic device driver, which means that if your server has a hardware cryptographic accelerator, GELI takes advantage of it transparently.
I should mention that people lose more data to encryption misconfiguration or lost keys than to laptop theft. When I hear someone say, “I’ve encrypted my whole hard drive!” I have a nearly psychic vision of the future where that same person is saying, “I’ve lost access to everything on my hard drive!” More often than not, I’m correct. Consider carefully whether you really, truly need disk encryption. If you do need it, also back your files up. Those government spooks aren’t going to crack the encryption on your laptop. They’re going to wait for you to decrypt it yourself—and then they’ll break in.
If you want to encrypt your laptop, use the FreeBSD installer to do so. You should still read this section so you understand how the disk encryption works, but if the installer wants to do the work for you, let it. We’ll walk through using GELI to encrypt a disk partition on /dev/da0, storing the cryptographic keys on the USB storage device mounted on /media. You might find it more sensible to use a filesystem in a file (see Chapter 13) as an encrypted partition. Very few people actually need to encrypt their entire hard drive, and in certain circumstances, doing so might raise suspicions. I have enough trouble explaining to airport security why my computer “looks so weird.” In their minds, a boot prompt that says, Insert cryptographic key and enter cryptographic passphrase is only one step away from This man is a dangerous lunatic who requires a very thorough body cavity search. If you really do need to encrypt certain documents, chances are they total only a few megabytes. That’s a perfect application for a filesystem in a file or a flash drive.
Note that you must load the geom_eli.ko kernel module before working with GELI.
Generating and Using a Cryptographic Key
GELI lets you use a key file and/or a passphrase as cryptographic keys for an encrypted device. We’ll use both. To generate your cryptographic key file, use dd(1) to grab a suitable amount of data from /dev/random and write it to a file. Remember, /media is where our USB device is mounted. If you really want to protect your data, create your key directly on the USB device and don’t leave it on your filesystem where a hypothetical intruder could recover it. (Even deleting the file still leaves remnants that a skilled attacker could conceivably extract.)
# dd if=/dev/random of=/media/da0p1.key bs=64 count=1
1+0 records in
1+0 records out
64 bytes transferred in 0.000149 secs (429497 bytes/sec)
The 64 bytes of data constitute a 512-bit key. You can increase the size of the key if you like, at the cost of extra processor overhead when accessing the encrypted filesystem. Don’t forget that your passphrase also increases key complexity.
To assign a passphrase to the key, use geli init. The -s flag tells geli(8) the desired sector size on the encrypted filesystem; 4,096 bytes, or 4KB, is usually a decent sector size for this application. The -K indicates the key file. You must also specify the device to be encrypted.
# geli init -s 4096 -K /media/da0p1.key /dev/da0p1
Enter new passphrase:
Reenter new passphrase:
A passphrase is much like a password except that it can contain spaces and be of any length. If you really want to protect your data, I recommend using a passphrase that is several words long, contains nonalphanumeric characters, and is not a phrase in your native language.
Now that you have a key, attach it to the device to be encrypted.
# geli attach -k /media/da0p1.key /dev/da0p1
Enter passphrase:
GELI now knows that /dev/da0p1 is an encrypted disk and that the file /media/da0p1.key contains the key file. Once you enter the passphrase, you can access the decrypted contents of the encrypted disk at the new device node, /dev/da0p1.eli. Of course, you need a filesystem to put any data on that disk.
Filesystems on Encrypted Devices
Before you build a filesystem on your encrypted device, purge the disk of any lingering data. Programs like newfs(8) and zpool(8) don’t actually overwrite most of the bits in a new partition; they simply add superblocks that indicate the location of inodes. If you’ve used this disk before, an intruder would be able to see chunks of old files on the disk. Worse, he’d see chunks of encrypted data placed there by GELI. Before you put a filesystem on the disk, it’s best to cover the disk with a deceptive film of randomness to make it much more difficult for an intruder to identify which blocks contain data and which do not. Use dd(1) again:
# dd if=/dev/random of=/dev/da0p1.eli bs=1m
FreeBSD has an infinite supply of chaos—or, in technical terms, /dev/random is nonblocking. The amount of time needed to cover the whole disk with high-quality randomness depends on your storage system. It might take a day.
Now that your disk is full of garbage, put a filesystem on it and attach it to your system. I’ll often use UFS on such encrypted devices.
# newfs /dev/da0p1.eli
# mount /dev/da0p1.eli /mnt
Your encrypted disk device is now available on /mnt. Store your confidential files there.
Encrypted disks have many more possibilities. Either read geli(8) or check out my book FreeBSD Mastery: Storage Essentials (Tilted Windmill Press, 2014).
This takes you through some of FreeBSD’s murkier corners. Now let’s see what to do when things go really wrong . . . .