One of the distinguishing characteristics of the BSDs is the ease with which you can keep your operating system source and installed software up-to-date. In fact, each of the BSDs provides multiple alternatives, allowing users to choose the approaches that best match their time and bandwidth requirements.
This chapter provides a plethora of ways to maintain an updated system. While many are written from the FreeBSD perspective, don’t let that stop you from hacking your own customized NetBSD or OpenBSD solutions. In fact, this chapter concludes with one user demonstrating how to enjoy the benefits of the BSD ports and packages collections on Mac OS X!
If you’re responsible for installing multiple systems, hopefully you’ve discovered the art of automating installs.
Most operating systems have some sort of scripting mechanism that allows you to predefine the answers to the questions asked by the install program. Once you’ve started the actual install, you can leave and return to a fully installed system. The alternative is to sit there, answering every prompt when it appears. No, thank you!
Even as a home user, it’s well worth your while to spend a few minutes customizing the install script that comes with FreeBSD. Try this hack once and you’ll never want to sit and watch an install again.
Before installing any system, you need to know the following:
The IP settings and hostname of the host you’re installing
The FreeBSD name of that host’s NIC
Which distributions, or parts of the OS, to install
Your desired partitioning scheme
Which packages (applications) to install
Of course, it’s always a good idea to record this information and include it with the documentation for the system.
FreeBSD’s install mechanism lives in /stand/sysinstall. Not surprisingly,
man
sysinstall
describes all of the scriptable
bits of this program. I’ll go over some useful parameters, but you’ll
definitely want to skim through the manpage to see if there are
additional parameters suited to your particular environment.
FreeBSD also comes with a commented, ready-to-customize install script, located in /usr/src/usr.sbin/sysinstall/install.cfg. Copy this file, then edit the copy in your favorite editor. Start by inserting your own network settings:
# This is the installation configuration file for my test machine, # crate.cdrom.com. # It is included here merely as a sort-of-documented example. # # $FreeBSD: src/usr.sbin/sysinstall/install.cfg,v 1.11 2001/09/06 10:04:27 murray Exp $ # Turn on extra debugging. debug=yes ################################ # My host specific data hostname=crate.cdrom.com domainname=cdrom.com nameserver=204.216.27.3 defaultrouter=204.216.27.228 ipaddr=204.216.27.230 netmask=255.255.255.240 ################################
Replace the example network information with the name and IP settings associated with the specific host you’d like to install. If you’re using DHCP to obtain this information, fill in the hostname line and replace the other lines with:
tryDHCP=YES
Next, replace the name of the NIC and the path to the FTP site.
In this example, the NIC is rl0
and
I’m using the default FTP site:
################################ # Which installation device to use _ftpPath=ftp://ftp.freebsd.org/pub/FreeBSD/ netDev=rl0 mediaSetFTP ################################
Next come the desired distributions. (See man sysinstall
for more details.) Include
them all on the one dists=
line,
separated by a space:
################################ # Select which distributions we want. dists=bin doc games manpages dict compat4x ports src sbase ssys Xbin Xcfg Xdoc Xlib Xman Xset Xfnt Servers/XS3V Xfsrv distSetCustom ################################
Note that distSetCustom
allows you to customize which distributions to install. If you’d
like to install the works, use distSetEverything
and don’t specify any
dists=
.
The partitioning scheme section is very important. If you don’t want to use the default scheme which uses the entire disk, read this section of the manpage carefully.
Also, the default file gives examples for three disks. Make sure you remove the examples and replace them with your own partitioning scheme.
The following example is the equivalent of choosing a
for “all,” followed by a
for “auto defaults”:
############################################################# # Set the parameters for the partition editor # ad = IDE, da = SCSI disk=ad0 partition=exclusive diskPartitionEditor ############################################################# # - All sizes are expressed in 512 byte blocks! # - "Size in MB" = sectors * 512 / 1024 / 1024 # - "Number of blocks" = xsize in mb * 1024 * 1024 / 512 # The non-zero value after the mountpoint means enable soft updates # 256MB UFS ad0s1a ad0s1-1=ufs 524288 / # 240MB SWAP ad0s1b ad0s1-2=swap 491520 none # 256MB UFS ad0s1d ad0s1-3=ufs 524288 /var # 256MB UFS ad0s1e ad0s1-4=ufs 524288 /tmp # Rest of FreeBSD partition ad0s1f ad0s1-5=ufs 0 /usr diskLabelEditor # runs diskLabelCommit diskPartitionWrite installCommit
Finally, list which applications you would like to install. List
each package on its own line, followed by the packageAdd
command:
# Install some packages at the end. package=fetchmail-6.2.0 packageAdd package=pine-4.55 packageAdd package=lynx-2.8.5d14 packageAdd
The FreeBSD package list (ftp://ftp.freebsd.org/pub/FreeBSD/releases/i386/5.1-RELEASE/packages/All) has the exact names of each available package. Replace i386/5.1-RELEASE with your platform and desired operating system version.
Now that you’ve created a customized version of install.cfg, prepare a freshly formatted UFS floppy:
#fdformat -f 1440 /dev/fd0
#bsdlabel -w /dev/fd0 fd1440
#newfs /dev/fd0
Once the floppy is ready, copy install.cfg onto it.
On a test system, start the install process either by booting
from a FreeBSD CD-ROM/DVD or with the two install floppies. When you
receive the sysinstall Main Menu
screen, choose Load Config
. Insert
the floppy containing your customized install.cfg and press OK. Once the
configuration file has been loaded, you’ll receive the message
You may remove the floppy from floppy drive
unit A
.
While this is meant to be an unattended install, you should be present during your first test install. This will give you the opportunity to ensure that your script runs smoothly, without hanging at any portion of the install. If it does hang, check your install.cfg for a typo in that section.
Once the install is complete, you’ll return to the sysinstall Main Menu
. At this point, you can
either configure the system interactively by choosing Configure
or use a prepared
post-configuration script, as found in /usr/doc/en_US.ISO8859-1/articles/pxe/post.
Once you’re happy with your floppy, label it with your operating system version. Store it where you can find it the next time you’re ready to install a version of that operating system.
For those who prefer to wipe their disks clean before they upgrade their systems.
Have you ever upgraded your system with make world
? If you have only one system on
your disks, you may run into a problem: if the installworld
fails partway through, you may
end up with a broken system that might not even boot. It’s also possible
that the installworld
will run
smoothly, but the new kernel will not boot.
What if you’re like me and believe in the “wipe your disks when upgrading systems” paradigm? Reformatting ensures there is no old cruft left lying around. It also means you have to recompile or reinstall all your ports and packages and then redo all your carefully crafted configuration tweaks.
FreeBSD From Scratch solves all these problems. The
strategy is simple: use a running system to install a new system under
an empty directory tree, mounting new partitions in that tree as
appropriate. Many config files can copy straight across, and mergemaster
can take care of those that cannot. You can perform
arbitrary post-configuration of the new system from within the old
system, up to the point where you can chroot
to the new system.
This upgrade has three stages, where each stage either runs a
shell script or invokes make
:
Creates a new bootable system under an empty directory, merges or copies as many files as are necessary, and then boots the new system
Installs your desired ports
Does post-configuration for software installed in the previous stage
From now on, whenever you feel like an update is in order, simply toggle the partitions you want to wipe and reinstall.
While compiling the ports during stage two, the system will not be available for its usual duties. If you run a production server, consider the downtime caused by stage two. If time is an issue, consider using precompiled packages instead of ports.
This hack uses several scripts and configuration files that you
can download from the original document’s site (listed in this hack’s
Section 8.3.4 section).
Also, if you keep your docs up-to-date with cvsup
, the scripts and original document can
be found in /usr/doc/en_US.ISO8859-1/articles/fbsd-from-scratch.
The script for stage one is stage_1.sh. When run with exactly one argument:
# ./stage_1.sh default
it will read its configuration from stage_1.conf.default and write a log to stage_1.log.default.
You’ll need to customize stage_1.conf.default to match your idea of the perfect system. I have tried to comment all of the sections you should adapt. In addition to the customized sections, the configuration script must provide four shell functions:
create_file_systems
create_etc_fstab
copy_files
all_remaining_customization
Before you run stage_1.sh,
make sure you have completed the usual tasks in preparation for
make
installworld/installkernel
:
Configure your kernel config file.
Complete make
buildworld
.
Complete make buildkernel
KERNCONF=
whatever
.
The stage_1.sh script will stop at the first command that fails, so you cannot overlook errors. It will also stop if you use an unset environment variable, which is probably due to a typo.
Answer no
or press Enter when
mergemaster
asks if whether should
delete /var/tmp/temproot.stage1.
This directory contains some files that must be copied to the new
system later.
*** Comparison complete
Do you wish to delete what is left of /var/tmp/temproot.stage1? [no] no
After that, it will list the files it installed:
*** You chose the automatic install option for files that did not exist on your system. The following were installed for you: /newroot/etc/defaults/rc.conf ... /newroot/COPYRIGHT (END)
Type q
to quit the pager.
Then, you’ll have to deal with login.conf:
*** You installed a login.conf file, so make sure that you run '/usr/bin/cap_mkdb /newroot/etc/login.conf' to rebuild your login.conf database Would you like to run it now? y or n [n]
The answer does not matter, since we will run cap_mkdb
in either case.
You can download the author’s stage_1.conf.default, which you’ll need to modify substantially. The comments should give you enough information regarding what to change.
Pay attention to the newfs
commands. While you cannot create new filesystems on mounted
partitions, the script will happily erase any unmounted partitions.
This can be enough to ruin your day, so be sure to modify the device
names to match your scenario.
Running this script installs a system that, when booted, provides inherited users and groups, firewalled Internet connectivity over Ethernet and PPP, correct time zone settings and NTP, and more minor configurations, such as /etc/ttys and /etc/inetd.conf.
Other areas of configuration will not work until stage two completes. For example, we have copied files to configure printing and X11. Printing, however, needs applications not found in the base system. Similarly, X11 will not run before we have compiled the server, libraries, and programs.
It is possible to install precompiled packages at this
stage instead of compiling ports. In this case, stage_2.sh will be nothing more than a
scripted list of pkg_add
commands.
I install my favorite ports via the downloadable stage_2.sh script. You can run it multiple
times safely, as it will skip all ports that are already installed. It
also supports the dry run option (-n
), which will show what would be done. Run
it like stage_1.sh, with exactly
one argument to denote a config file:
# ./stage_2.sh default
This example will read the list of ports from stage_2.conf.default.
The actual list of ports consists of lines with two or more
space-separated words: the category and the port, optionally followed
by an installation command that will compile and install the port. By
default, this is make
install
. Most of the time, it suffices to
name only the category and port. You can fine-tune some ports by
specifying make
variables, as found
in the port’s Makefile:
www mozilla make WITHOUT_MAILNEWS=yes WITHOUT_CHATZILLA=yes install mail procmail make BATCH=yes install
In fact, you can specify arbitrary shell commands, so you are
not restricted to simple make
invocations:
java linux-sun-jdk14 yes | make install news inn-stable CONFIGURE_ARGS="--enable-uucp-rnews --enable-setgid-inews" make install
Note that the line for news/inn-stable
includes an example of a
one-shot shell variable assignment to CONFIGURE_ARGS
. The port’s Makefile will use this as an initial value
and augment some other essential args.
The difference between specifying a make
variable on the command line (as in the
last example) and the following:
news inn-stable make CONFIGURE_ARGS="--enable-uucp-rnews --enable-setgid-inews" install
is that the latter will override instead of augment.
Be careful that your ports do not use an interactive install; they should not try to read from stdin. If they do, they will read the next line or lines from your list of ports and get confused. If stage_2.sh mysteriously skips a port or stops processing, this is likely the reason.
Finally, this script will create a log file named LOGDIR/category+port for each port it installs.
When you download the stage_2.sh script, you may want to modify these variables at the beginning of the script to reflect your environment:
DBDIR="/var/db/pkg" PORTS="/usr/ports" LOGDIR="/home/root/setup/ports.log"; mkdir -p ${LOGDIR}
You installed your beloved ports during stage two, but some ports require a little bit of configuration. This is the job of stage three, the post-configuration stage. I have chosen to implement stage three as a Makefile because this allows easy selection of what you want to configure simply by running:
# make -f stage_3.mk target
As with stage_2.sh, make sure you have stage_3.mk available after booting the new system, either by putting it on a shared partition or by copying it somewhere on the new system.
Automating the installation of a port may prove difficult if it
is interactive and does not support make
BATCH=YES install
. For a few ports, the interaction is
nothing more than typing yes
when
asked to accept some license. If such input is read from the standard
input, we simply pipe the appropriate answers to the installation
command, usually make install
. This
is how I dealt with java/linux-sun-jdk14 in the previous
example.
This strategy, however, does not work for editors/staroffice52, which requires that X11 is running. The installation procedure involves a fair amount of clicking and typing, so it cannot be automated like other ports can. However, the following workaround does the trick for me. First, I created a staroffice package on the old system with:
#cd /usr/ports/editors/staroffice52
#make package
= ==> Building package for staroffice-5.2_1 Creating package /usr/ports/editors/staroffice52/staroffice-5.2_1.tbz Registering depends:. Creating bzip'd tar ball in '/usr/ports/editors/staroffice52/staroffice-5.2_1.tbz'
During stage two, I used pkg_add
to add this package:
# pkg_add /usr/ports/editors/staroffice52/staroffice-5.2_1.tbz
The downloadable stage_3.mk will give you an idea of how to automate all reconfiguration.
“FreeBSD From Scratch” (includes links to the scripts) at http://www.freebsd.org/doc/en_US.ISO8859-1/articles/fbsd-from-scratch/article.html
Use a three-way merge to deal with upgraded configuration files.
Even though you probably run cvsup
on a daily basis, you likely run make
world
only a few times a year, whenever a new version of the
OS is released. The steps required to upgrade your system are well
documented and fairly straightforward. That is, it’s easy until it’s
time to run mergemaster
.
mergemaster
is an important
step, as it integrates changes to /etc. For example, occasionally a core
utility such as Sendmail will require a new user or group in /etc/passwd. Problems can occur if those
changes aren’t integrated.
If you’ve used mergemaster
before, you know it’s not the most user-friendly utility out there.
Misinterpret a diff
, and you might
lose your configuration file changes or, worse, miss a necessary change.
You might even end up blowing away your own users in /etc/passwd—not the most convenient way to
start off a new upgrade.
An alternative is to use etcmerge
(/usr/ports/sysutils/etcmerge). This utility
does most of the work for you. Unlike the two-way diff
used by mergemaster
, this utility can compare the
changes between three sets of edits:
The /etc from your original version of FreeBSD
Any changes you’ve made to /etc since then
The /etc for your new version of FreeBSD
Before any upgrade, you definitely want a fresh, tested backup of all of your data, including /etc.
Once you’ve installed etcmerge
, ensure you have a backup copy of
/etc:
# tar czvf etc.tgz /etc
Here, I’ve saved a copy only to the local hard drive. Be sure to copy it to another location as well, just to be safe: to another system, a removable media, or even your email account.
The next step is to locate a copy of /etc that is original to your current operating system and save it to /var/db/etc. (This is a good step to add to your regime when you install a new system.) Assuming this isn’t a fresh install and you’ve made changes to /etc, you can get the original, unmodified /etc for your operating system version at http://people.freebsd.org/~eivind/etc/.
Here, I’ve downloaded the 5.1-RELEASE version and untarred it to the correct place:
#tar -C /var/db -zxvpf etc-5.1-RELEASE.tar.gz
#ls /var/db/etc/
So, now you have a copy of the original /etc, as well as your own customized /etc. You’ll receive the /etc for a newer version of FreeBSD once you’ve changed your cvs-supfile to reflect the newer tag [Hack #80] .
For example, I’m currently running 5.1-RELEASE, so my custom supfile contains this line:
*default tag=RELENG_5_1_0_RELEASE
When I’m ready to upgrade to 5.2, I’ll change that line to reflect the new tag:
*default tag=RELENG_5_2_0_RELEASE
My next cvsup
will grab the sources for the new operating system
version.
None of the changes to /usr/src will be integrated until you
make buildworld
and make installworld
as per the instructions
in the handbook. Simply downloading the changes does not upgrade
your operating system.
Once cvsup
has finished
downloading all of the changes, take the time to read /usr/src/UPDATING, which lists all of the
known gotchas for this release. For example, there may be mandatory
options for the kernel process of the upgrade, certain stages may
require a reboot before the next stage works, or perhaps directory
structures such as /etc have seen
major changes.
Once you’ve made your necessary preparations, ensure these steps
have succeeded before using etcmerge
:
make buildworld
make buildkernel
make installkernel
make installworld
Now that you have a new world, use etcmerge
to integrate any changes to
/etc. As per its manpage, start
with the initialization step:
# etcmerge init
The script will perk along for a moment or two before producing
a screen full of lines that start with ETCMERGE
. Here’s the beginning of that
output:
ETCMERGE: >>> Finding classes of files ETCMERGE: >>> Working from ETCMERGE: >>> Active: /etc ETCMERGE: >>> Reference: /var/db/etc ETCMERGE: >>> New: /root/etc-work/200401191624/etc-new
Note the name of the directory in the last line. It contains the working files that are ready for your review.
You’ll then receive lines for different classes—see man etcmerge
for a description of each
conflict class. Here’s a sample output from a system I recently
upgraded:
ETCMERGE: >>>> Class 7: 3 conflict(s)
A class 7 conflict means a file existed for all three versions
of /etc. Any differences will
appear with diff
-style markers.
This particular system has three files containing conflicts. Their
names are in the file called 7.conflicts:
# more /root/etc-work/200401191624/7.conflicts
./manpath.config
./pwd.db
./spwd.db
The etc-merged subdirectory contains copies of those files with the differences marked. Look there and examine each file listed as containing conflicts:
#cd /root/etc-work/200401191624/etc-merged
#vi manpath.config
Don’t send pwd.db or
spwd.db to an editor—these are
the database versions of your password files. Instead, use diff
to see if the conflict is because
you’ve added users or because FreeBSD has added a new user:
# diff etc-new/master.passwd /etc/master.passwd
Remove the two .db
lines
from 7.conflicts manually so
etcmerge
is aware that you’ve
resolved the conflicts to your password databases.
As you review your own files, the angle bracket markers indicate which lines have changed. Next to each angle bracket marker is the name of the file containing the conflicting lines. For example, if the name of the file includes the /etc-new directory, the lines in question belong to the new version of the file. Once you’ve decided which version of the lines you wish to keep, remove the angle bracket lines as well as the unwanted version of the lines.
Once you’re finished your edits, this command will integrate them:
# etcmerge install
/etc/mail/aliases: 24 aliases, longest 10 bytes, 246 bytes total
Install done - removing copies of old /etc/ and old reference.
Done.
#
Congratulations! You’ve successfully upgraded your operating system while maintaining your customizations to /etc.
man mergemaster
man etcmerge
man build
The makeworld
section of
the FreeBSD Handbook, which includes directions for using mergemaster
(http://www.freebsd.org/doc/en_US.ISO8859-1/books/handbook/makeworld.html)
FreeBSD provides many tools to make software upgrades as painless as possible. In fact, the entire process is fully scriptable. Simply choose the pieces you want and how up-to-date you want to be.
End users and administrators alike share a desire to keep their operating systems and applications as up-to-date as possible. However, if you’re an operating systems veteran, you’re well aware that this desire doesn’t always translate into foolproof, easy execution. For example, do you have to scour the far corners of the Internet to find the latest updates? Once you find them, is it possible to upgrade safely without overwriting the dependencies required by other applications?
The cvsup
process provides the latest updates to the FreeBSD
operating system, ports collection, and documents collection. You no
longer have to scour the Internet looking for the latest sources.
Simply run cvsup
!
Since our intention is to script the whole process, install the cvsup-without-gui port:
#cd /usr/ports/net/cvsup-without-gui
#make install clean
If you’ve never used cvsup
before, take the time to read its section in the FreeBSD Handbook so
you have an overview of how the process works.
When the install finishes, copy /usr/share/examples/cvsup/cvs-supfile to a location that makes sense to you (e.g., /root or /usr/local/etc). Use the comments in that file and the instructions in the handbook to customize the file so it reflects your closest mirror, operating system (tag), and what you would like to update.
Here’s my cvs-supfile. It uses a Canadian mirror and updates all sources, ports, and documents on a FreeBSD 5.1-RELEASE system:
# more /root/cvs-supfile
#use the Canadian mirror
*default host=cvsup.ca.freebsd.org
#keep these lines as-is!
*default base=/usr/local/etc/cvsup
*default prefix=/usr
#this is a 5.1-RELEASE system
*default tag=RELENG_5_1_0_RELEASE
#keep this line as-is!
*default release=cvs delete use-rel-suffix compress
#update all src, ports, and docs
src-all
ports-all tag=.
doc-all tag=.
If you want to specify which source, ports, and docs to install, see the handbook for directions on creating a refuse file.
If your cvs-supfile
includes the ports-all tag=
. line,
install portupgrade
. This
port will not only keep track of which ports need upgrading, it will
also track dependencies and automate the entire application upgrade
process:
#cd /usr/ports/sysutils/portupgrade
#make install clean
We can also take advantage of the fastest_cvsup
port. As the name implies, it
looks for the fastest cvsup
mirror:
#cd /usr/ports/sysutils/fastest_cvsup
#make install clean
With the necessary pieces in place, let’s run them from the
command line to see how they work. First, use cvsup
to download any changes to the
operating system, software, or documents tree:
# cvsup -L2 /root/cvs-supfile
Parsing supfile "/root/cvs-supfile"
Connecting to cvsup.ca.freebsd.org
Connected to cvsup.ca.freebsd.org
Server software version: SNAP_16_1f
Negotiating file attribute support
Establishing collection information
Establishing multiplexed-mode data connection
Running
Updating collection src-all/cvs
Updating collection ports-all/cvs
<snip downloaded sources>
Updating collection doc-all/cvs
<snip downloaded sources>
Shutting down connection to server
Finished successfully
The -L2
switch turns on
verbosity. Substitute /root/cvs-supfile with the location of your
customized cvs-supfile.
It’s rare for src to change. When it does, it is usually due to a security patch. If you notice changes to src, go to http://www.freebsd.org/security/#adv to see if the security incident affects you and how to apply the patch if it does.
Once cvsup
is complete,
integrate the changes to the ports and the documents trees. This will
take care of the document changes:
#cd /usr/doc
#make install
You need the docproj-nojadetex port [Hack #89] for this command to succeed.
For the ports, first update your ports index:
#cd /usr/ports
#make index
Generating INDEX-5 - please wait.. Done.
An alternative is to instead run portsdb -Uu
. Note that if you’ve created a
refuse file, either command will
produce a screen or two of error messages. You can safely ignore
these.
Once your ports tree is up-to-date, see if any of your installed applications need upgrading:
# portversion -l "<"
[Updating the pkgdb <format:bdb1_btree> in /var/db/pkg ...
256 packages found (-0 +1) . done]
ghostscript-gnu <
gimp-print <
linux-sun-jdk <
p5-MIME-Base64 <
subversion <
xmlcatmgr <
The -l "<
" flag tells
portversion
to list only the ports
matching that pattern (which represents ports that need upgrading).
This particular system has 256 installed ports. I’ve added one
(+1
) new port since my last
cvsup
, and six packages need
upgrading.
To perform the actual upgrade:
# portupgrade -arR
-a
means to upgrade all ports
requiring an upgrade. -rR
is
very important—it will
ensure that the upgrade takes care of all dependencies
properly.
I’ve only scratched the surface of all of these utilities. Spend some time reviewing the resources at the end of this hack to ensure you’re getting the most out of your upgrade process.
Once you have a few dry runs under your belt and are happy with
your results, create a shell script to automate the process. You can
start out with something as simple as a Bourne script that strings
together the desired commands and switches. Here, the only new command
I’ve introduced is fastest-cvsup
.
I’ve also added an else
statement
to terminate the script if there is a problem with cvsup
—for example, if the network connection
fails.
# more /root/bin/mycustomupgrade.sh
#!/bin/sh
# script to automate cvsup of latest src, ports, and doc
# then rebuilds doc and ports trees
# then checks for and upgrades out-of-date software
# when finished, prints date and time
# use fastest_cvsup to find fastest Canadian or US mirror
# store the results in $SERVER to be passed to cvsup command
# substitute /root/cvs-supfile with path to custom cvs-supfile
# terminate the script if a connection is not available to
# the cvsup server
if SERVER=`fastest_cvsup -q -c ca,us`
then
echo "Running cvsup"
cvsup -L2 -h $SERVER /root/cvs-supfile
else
echo "There's a problem!" 1>&2
exit 1
fi
echo "Updating docs"
cd /usr/doc
make install
echo "Updating ports index"
cd /usr/ports
make index
echo "The following ports need upgrading"
portversion -l "<"
echo "Upgrading ports"
portupgrade -arR
echo "Finished at `/bin/date`."
exit
Don’t forget to make your script executable with chmod +x
and to test it to ensure all of the
steps execute as desired. On some of my systems, I’m really picky
about which software updates to apply, so I don’t include the portupgrade -arR
command in my script. This
allows me to review which ports need upgrading so I can manually
upgrade the ones I deem necessary.
man portversion
man portupgrade
man fastest-cvsup
The cvsup
section of the
FreeBSD Handbook (http://www.freebsd.org/doc/en_US.ISO8859-1/books/handbook/cvsup.html)
The CVS tags section of the FreeBSD Handbook (http://www.freebsd.org/doc/en_US.ISO8859-1/books/handbook/cvs-tags.html)
“portupgrade,” from the FreeBSD Basics column (http://www.onlamp.com/pub/a/bsd/2003/08/28/FreeBSD_Basics.html)
Combine the advantages of compiling from source and installing packages.
We saw in [Hack #69] that compiling applications
from source, i.e., by making their ports, has several advantages. You
can tune /etc/make.conf to take
advantage of your architecture. You can also customize the installation
by passing various arguments to make
.
However, if you’re responsible for maintaining software on multiple machines, do you always want to install from source? If your systems run similar hardware, why not create your own customized packages on one machine and make them available to your other systems via a package repository?
Creating your own custom packages allows you to retain all the
benefits of make
. Even better, the
resulting package installs the desired software very quickly. This can
be a real time-saver when you maintain multiple systems.
Pick a machine in your network to contain the package repository, and install the ports collection on that system. The rest of your systems won’t need the ports collection, which saves their disk space for other purposes.
On the system containing the ports collection, create a directory to store the packages:
# mkdir /usr/ports/packages
Then, decide which packages you’d like to create. I’ll start
with Exim. Before creating the package, I’ll search through the port’s
Makefile to see if there are any
make
options:
# grep WITH /usr/ports/mail/exim/Makefile
#WITH_TCP_WRAPPERS= yes
#WITH_MYSQL= yes
#WITH_SASLAUTHD= yes
#WITHOUT_TLS= yes
#WITHOUT_PERL= yes
#WITHOUT_PAM= yes
<snip>
This particular port has dozens of tweakables. After a more
careful read-through of the Makefile, I’ve chosen to use WITHOUT_IPV6
and WITH_SASLAUTHD
.
Next, I need to determine if there are any dependencies:
# grep DEP /usr/ports/mail/exim/Makefile
LIB_DEPENDS= iconv.3:${PORTSDIR}/converters/libiconv
RUN_DEPENDS= ${LOCALBASE}/sbin/eximon:${PORTSDIR}/mail/exim-monitor
LIB_DEPENDS+= db4.0:${PORTSDIR}/databases/db4
LIB_DEPENDS+= db41.1:${PORTSDIR}/databases/db41
LIB_DEPENDS+= db-4.2.2:${PORTSDIR}/databases/db42
RUN_DEPENDS+= ${LOCALBASE}/sbin/saslauthd:${PORTSDIR}/security/
cyrus-sasl2-saslauthd
RUN_DEPENDS+= ${LOCALBASE}/sbin/pwcheck:${PORTSDIR}/security/cyrus-sasl
LIB_DEPENDS+= pq.3:${PORTSDIR}/${POSTGRESQL_PORT}
Yup. Lots of those as well. This means I’ll pass an extra
argument to make
to ensure the
package also creates packages for each dependency. Once I know the
desired make
arguments, I create
the package:
#cd /usr/ports/mail/exim
#make package -DWITHOUT_IPV6 -DWITH_SASLAUTHD DEPENDS_TARGET=package
Notice that I used make
package
rather than the usual make
install
. I then included my two make
options. I ended the command with the
DEPENDS_TARGET=package
option. (I
found this argument on a mailing list as the result of a Google
search.) If you’re building any package that has
dependencies, remember to include that option.
make package
does two things.
First, it creates and stores the package in a subdirectory of
/usr/ports/packages. In this
example, that subdirectory will be mail. Second, it installs the port on the
local machine, if necessary. If you don’t want to keep the application
installed on the machine acting as the package repository, simply type
make deinstall
after creating the
package.
Once you’ve populated /usr/ports/packages with the packages
required by your network, set up an NFS mount to share the package
repository. The easiest way to do this is with stand/sysinstall
. On the machine holding the
packages:
# /stand/sysinstall
Choose Configure, then Networking, and then NFS server. You should see the following message:
Operating as an NFS server means that you must first configure an /etc/exports file to indicate which hosts are allowed certain kinds of access to your local file systems. Press [ENTER] now to invoke an editor on /etc/exports
Unless you’ve changed your default editor, /etc/exports will open in vi
. The default file contains some example
syntax; see man exports
for
additional tips.
I added this line to reflect my network settings:
/usr/ports/packages -network 192.168.2.0 -mask 255.255.255.0
Once you’ve saved your changes, initialize and start the NFS server:
#/etc/rc.d/nfsd rcvar
#/etc/rc.d/nfsd start
Then, ensure the NFS server is listening for requests:
# sockstat | grep nfs
root nfsd 3973 tcp4*:2049 *:*
Next, you’ll need to create an NFS client on each machine that
will use the package repository. This time, in /stand/sysinstall
, choose NFS client instead
of NFS server. There are no prompts, so just select the box. Once
you’ve exited the utility, type:
# nfsiod -n 4
This will optimize the performance of the NFS client.
Then, check to see if you can access your package repository. In my example, the machine containing the packages has an IP address of 192.168.2.12 and the local machine has a mount point called /packages:
#mkdir /packages
#mount 192.168.2.12:/usr/ports/packages /packages
#ls /packages
All Latest ipv6 mail security sysutils
These various subdirectories contain the Exim package and its
dependencies. To get an idea of which packages are available, use
ls /packages/All
.
It’s also a good idea to try a test installation of a package:
# pkg_add /packages/mail/exim-4.30.tbz
Don’t forget to unmount the NFS share when you’re finished:
# umount /packages
man exports
man nfsiod
While the ports tree is one of the most useful FreeBSD directory structures, you may have systems where it’s not appropriate to maintain the entire ports structure.
On some of your systems, disk space may be an issue. The ports tree tarball itself is a 21 MB download. Once untarred, it will occupy around 500 MB of disk space. That space will continue to grow as you install ports since, by default, source files download into /usr/ports/distfiles.
Does this mean that installing packages is your only alternative?
Packages are convenient, but since they are precompiled, you don’t have
the option of providing your own make
arguments to optimize the install for your environment.
One alternative is the anonymous CVS system. Even a minimal install of FreeBSD
includes the cvs
command. This allows
you to check out only the particular port skeleton you need. You’ll
still have the convenience of the ports collection without actually
having to install it.
The first time you use cvs
,
create an empty CVS password file, as CVS will complain if this file
is missing:
# touch ~root/.cvspass
Then, ensure your present working directory is /usr:
# cd /usr
When using cvs
to maintain
your ports, be sure you are in /usr. cvs
downloads the requested files to your
current working directory and will overwrite any files of the same
name.
Then, use the cvs login
command to connect to a CVS server. There are five FreeBSD anonymous
CVS servers; see the Handbook reference at the end of this hack for
their names and passwords. Use the setenv
command to specify the server to log
into:
#setenv CVSROOT :pserver:[email protected]/home/ncvs
#cvs login
Logging in to :pserver:[email protected]:2401/home/ncvs CVS password:anoncvs
#
Once you’ve successfully logged in, you’ll receive your normal prompt back. You’ll remain connected to the CVS server until you explicitly log off. In the meantime, you now have the ability to issue commands either on the CVS server or on your own system.
Let’s assume you have a minimum install and don’t have an existing /usr/ports directory structure. To install a port, you need the Mk and Templates directories as well as the port’s Makefile.
Use the cvs checkout
command
to retrieve the necessary files from the CVS server:
#cvs checkout -A -P ports/Mk
cvs server: Updating ports/Mk U ports/Mk/bsd.emacs.mk U ports/Mk/bsd.gnome.mk U ports/Mk/bsd.gnustep.mk U ports/Mk/bsd.java.mk U ports/Mk/bsd.kde.mk U ports/Mk/bsd.openssl.mk U ports/Mk/bsd.port.mk U ports/Mk/bsd.port.post.mk U ports/Mk/bsd.port.pre.mk U ports/Mk/bsd.port.subdir.mk U ports/Mk/bsd.python.mk U ports/Mk/bsd.ruby.mk U ports/Mk/bsd.sites.mk #cvs checkout -A -P ports/Templates
cvs server: Updating ports/Templates U ports/Templates/README.category U ports/Templates/README.port U ports/Templates/README.top U ports/Templates/config.guess U ports/Templates/config.sub #
Since you’re in the /usr
directory, cvs
will create
/usr/ports for you and will
populate the Mk and Templates subdirectories with their sets of
files. It’s interesting to note how little disk space this
bare-minimum ports tree requires:
# du -h /usr/ports | tail -n1
418K ports
That’s a pretty big difference from 500 MB!
Next, decide which port you’d like to install. The only
disadvantage to not having the entire ports structure is that you need
an alternate method of discovering the name of the port you’d like to
install. For example, in order to install lynx
, I need to know that it is in the
www subdirectory and that there
are three different versions of lynx
to choose from. The easiest way to
discover this information is to use the search utility at http://www.freshports.org.
Once you find the port you’re looking for, it will indicate the name of its directory. In my example, lynx-2.8.5d17 lives in www/lynx-current.
Now it’s a simple matter of checking out that port’s skeleton:
# cvs checkout -A -P ports/www/lynx-current
cvs server: Updating ports/www/lynx-current
U ports/www/lynx-current/Makefile
U ports/www/lynx-current/distinfo
U ports/www/lynx-current/pkg-descr
U ports/www/lynx-current/pkg-plist
Next, check the port’s Makefile to see if there are any dependencies:
# grep DEPENDS /usr/ports/www/lynx-current/Makefile
LIB_DEPENDS= intl.5:${PORTSDIR}/devel/gettext
As it stands right now, this port will not install, as I don’t have the ports skeleton for the dependency devel/gettext. So, I’ll download that port skeleton and double-check that that port doesn’t have any dependencies:
#cvs checkout -A -P ports/devel/gettext
<snip output> #grep DEPENDS /usr/ports/devel/gettext/Makefile
#
Okay, it looks like all dependencies are there. I’m ready to build the port:
#cd /usr/ports/www/lynx-current
#make install clean
If disk space is an issue, instead use make install distclean
, which will delete
the source from /usr/ports/distfiles once the build
successfully completes.
That’s it. As long as you remember to look for dependencies
before you issue your make install
command, your minimal ports structure should work as flawlessly as the
full ports collection.
Don’t forget to use cvs
logout
when you’re finished retrieving the files you need
from the CVS server.
man cvs
The AnonCVS section of the FreeBSD Handbook, which includes the names of the BSD CVS servers (http://www.freebsd.org/doc/en_US.ISO8859-1/books/handbook/anoncvs.html)
Keep your ports up-to-date without using cvsup.
If you have a slow Internet connection, it can take a while
to download the ports tree; the current tarball is over 21 MB in size.
Once you have the ports collection, keeping up-to-date with cvsup
might not be such an attractive option
if it involves tying up your phone line.
Perhaps bandwidth isn’t the problem. Perhaps you’re just looking
for an alternative way to stay current, without having to install and
configure cvsup
. After all, why
install additional software if you can achieve the same results using
commands that come with the base system?
Regardless of which category you fall into, CTM may be what you’re looking for.
CTM was originally CVS Through Email, meaning you could receive
the changes you usually receive through cvsup
via email. (In the case of numerous
changes, you’d receive several, smaller mails instead of one monolithic
message.) This can be a cheaper alternative to cvsup
if you’re charged for the amount of time
you are connected to the Internet.
However, it’s even easier to retrieve these changes with ftp
. FreeBSD
maintains several CTM servers that contain the changes, or deltas, to
the FreeBSD source and the ports collection. This hack will concentrate
on keeping your ports up-to-date using ftp
and the CTM servers.
Let’s start with a system that doesn’t have the ports collection
installed. First, I’ll create an empty ports directory for ctm
to work with:
#mkdir /usr/ports/
#cd /usr/ports
Then, instead of downloading and untar
ring the ports tree tarball, I’ll ftp
into a CTM server and download the
latest ports tree delta. The Handbook’s section on CTM includes the
addresses of the CTM mirrors.
#ftp ftp.freebsd.org
<snip banner and login> ftp>cd pub/FreeBSD/development/CTM/ports-cur
ftp>ls
<snip most of long listing> -rw-r--r-- 1 110 root 22332066 Jan 23 08:46 ports-cur.5100xEmpty.gz -rw-r--r-- 1 110 root 67953 Jan 24 00:43 ports-cur.5101.gz -rw-r--r-- 1 110 root 14256 Jan 24 16:51 ports-cur.5102.gz
Look toward the end of the listing for the large file closest to
the present date. It will have the word xEmpty
in its name. That file is your
starting delta. Download that and any subsequent deltas.
ftp>get ports-cur.5100xEmpty.gz
ftp>get ports-cur.5101.gz
ftp>get ports-cur.5102.gz
ftp>quit
Your first ftp
transfer
will be the largest and longest, as you are downloading the elements
necessary to build the ports tree structure. Subsequent sessions
will be very quick.
Note the .gz extension; leave the files compressed. CTM will still work, and you’ll save disk space.
Save your deltas to /usr/ports, and remain in this directory
when you use the ctm
command.
Now that you have your starting deltas, apply them with ctm
:
# ctm ports-cur.5100xEmpty.gz
ctm: warning: .ctm_status not found
<snip long output>
The first time you use ctm
,
it will complain about a missing .ctm_status file. Don’t worry; it will
create it for you. After a few seconds, it will send a lot of output
to stdout. Once the command has finished, you’ll have a fully
installed version of the ports tree.
That .ctm_status file will tell you the delta number of that ports tree:
# more .ctm_status
ports-cur 5100
Then, simply apply any subsequent deltas in ascending order. This will correctly incorporate all of the changes to the ports tree.
#ctm ports-cur.5101.gz
#ctm ports-cur.5102.gz
#more .ctm_status
ports-cur 5102
That’s it. Whenever you want to update your ports tree, ftp
into your CTM mirror, download the
deltas containing a higher number than your current version, and apply
them in order.
It’s up to you whether to keep the compressed versions of the files you download. Once you’ve successfully applied a delta—as indicated by .ctm_status—you no longer need to store that delta file. However, if download speed or time is an issue, consider keeping a copy of that large starting delta, just in case you ever want to recreate your ports tree from scratch.
If you’re too lazy or forgetful to ftp
for changes periodically, consider
receiving them automatically via email. Changes occur once or twice a
day. Subscribe to the ctm-ports-cur mailing list to receive them (http://lists.freebsd.org/mailman/listinfo/ctm-ports-cur/).
Complete the online subscription form, and reply to the email that asks you to confirm your subscription.
However, do not subscribe to that mailing
list until you’ve configured your system to handle those emails.
Basically, you want the system to intercept those CTM updates instead
of sending them directly to your mailbox. There are two ways to do
this: either create a sendmail
alias or create a procmail
recipe.
See man ctm_rmail
for detailed
instructions.
It’s also a good idea to verify the PGP signatures before
applying those updates. You can find detailed instructions for this,
as well as for using ctm_rmail
to
handle incoming deltas, in this message from the ctm-users mailing
list: http://lists.freebsd.org/pipermail/ctm-users/2003-October/000039.html.
man ctm_rmail
The CTM section of the FreeBSD Handbook (http://www.freebsd.org/doc/en_US.ISO8859-1/books/handbook/ctm.html)
Use built-in commands to keep abreast of the FreeBSD ports collection.
What first attracted me to FreeBSD—and what has definitely kept
my attention since—is the ports collection. Over 10,000 applications are
a mere make
install clean
away. For a software junkie like
myself, it is indeed Nerdvana to no longer scour the Internet for
software or fight my way through dependency hell just to convince an
application to install.
Admittedly, it’s easy to get lost in a sea of ports. How do you choose which application best suits your needs? How do you keep track of which ports have been installed on your system? How do you make sure you don’t inadvertently delete a dependency? Read on to see how to get the most out of the built-in utilities for managing ports.
You know you want to install some software to add functionality to your system. Wouldn’t it be great if you could generate a list of all the ports that are available for your specific need? Well, you can, and it’s almost too easy with the built-in port search facility. In this example, I’ll look for ports dealing with VPN software:
%cd /usr/ports
%make search key=vpn | more
Port: poptop-1.1.4.b4_2 Path: /usr/ports/net/poptop Info: Windows 9x compatible PPTP (VPN) server Maint: [email protected] Index: net B-deps: expat-1.95.6_1 gettext-0.12.1 gmake-3.80_1 libiconv-1.9.1_3 R-deps: <snip>
I snipped the results for brevity as this command gives the details of each port associated with VPNs. The format of the output is quite useful, as it gives the name of the port itself, its location in the ports tree, a brief description, the address of the maintainer, as well as the build and run dependencies.
If you’re only interested in seeing how many ports are
available, pipe the results to grep
instead of more
:
% make search key=vpn | grep Port
Port: poptop-1.1.4.b4_2
Port: pptpclient-1.3.1
Port: ike-scan-1.2
Port: openvpn-1.5.0
Port: tinc-1.0.2
Port: vpnd-1.1.0
Perhaps you’d prefer to know their locations:
% make search key=vpn | grep Path
Path: /usr/ports/net/poptop
Path: /usr/ports/net/pptpclient
Path: /usr/ports/security/ike-scan
Path: /usr/ports/security/openvpn
Path: /usr/ports/security/tinc
Path: /usr/ports/security/vpnd
What if you already know the name of the port you want to
install but aren’t sure what versions are available? Use search name=
instead. For example, this
command will search for all ports with netscape
in their names:
% make search name=netscape | grep Port
Port: pt_BR-netscape7-7.02
Port: netscape-remote-1.0_1
Port: netscape-wrapper-2000.07.07
Port: netscape-communicator-4.78
Port: netscape-navigator-4.78
Port: linux-netscape-communicator-4.8
Port: linux-netscape-navigator-4.8
Port: netscape7-7.1
If you find the search facility useful, it is a good idea to update your ports index periodically. Become the superuser and issue the following command (it may take a while, so don’t execute it if you’re in a hurry):
#cd /usr/ports
#make index
Finally, if you really want to fine-tune your search results, spend a few moments reading the examples in /usr/ports/Tools/scripts/README.portsearch.
You’ve spent a few months installing software and trying out
new applications. How do you keep track of all of that software and
all of those dependencies? pkg_info
is your friend.
My favorite pkg_info
switch
is definitely -x
. (There’s not
really a mnemonic for this switch; I tend to think of it as “give me
version x.”) If I stack it with any other switch, I don’t need to know
the full name (including the complete version number) of a port. For
example:
% pkg_info -xc lynx
will show the one-line comment (-c
) of every application that starts with
lynx
, regardless of the version
number. Besides saving memory cells for other purposes, it’s an
excellent way to find out if you have more than one version of
lynx
installed.
After installing a port, it’s useful to see if there were any messages, as these often contain configuration instructions:
% pkg_info -xD xmms
Information for xmms-esound-1.2.8_2:
Install notice:
Xmms supports Gzipped and uncompressed skins. If you would like to use
Zip format skins you will need to ensure archivers/unzip is installed.
How many times have you installed a port and had no clue
regarding the name of the executable, much less the names and
locations of any configuration files or documentation? Thank goodness
for -L
, the file-listing
flag:
% pkg_info -xL lynx | more
Information for lynx-2.8.4.1d:
Files:
/usr/local/man/man1/lynx.1.gz
/usr/local/bin/lynx
/usr/local/etc/lynx.cfg.default
/usr/local/share/doc/lynx/CHANGES
<snip>
Depending upon the application, the listing may be quite long. A
judicious pipe to grep bin
,
grep man
, or grep doc
may better suit your
purposes.
Before uninstalling an application, it is always a good idea to see if any
other packages require that application as a dependency. For example,
you’ve typed pkg_info | more
and
see the application ORBit-0.5.17
.
You think to yourself, “I don’t remember installing, or even ever
using, this application. Where did it come from? Maybe I should just
get rid of it.” This command will clear up your mini-mystery:
% pkg_info -xR ORBit
Information for ORBit-0.5.17_1:
Required by:
bonobo-1.0.22
flashplugin-mozilla-0.4.10_4
<snip>
Since the snipped output took up most of a page, it looks like
this application is useful after all. Don’t worry; if you did try to
uninstall that application, pkg_delete
would refuse since it is required
by those other applications. However, it is always nice to be aware of
these things ahead of time.
What happens if you go a little install-crazy and end up with
more applications than disk space? Use the -s
(size) switch to determine how much space
an application uses. Send the output either to a pager:
% pkg_info -as | more
or to a file that you can read at your leisure:
% pkg_info -as > sizes
You’ll then have an idea of which applications are using the most space so that you can decide which ones are worth uninstalling. Remember, you also have the comment and dependencies switches to help you decide.
Yet another way to find out what software you have installed is
to use pkg_version
:
% pkg_version | more
This will list each installed application, in alphabetical order. You’ll note that each application is followed by one of the three symbols in Table 8-1.
Symbol | Meaning |
| The application is up-to-date. |
| There is a newer version of the application available. |
| Your index may be out-of-date. |
So, to determine which applications require upgrading:
% pkg_version -l "<"
Note that you need to place quotes around the less-than sign or your shell will complain about a missing name for your redirect. If you don’t receive any output, congratulations! All of your applications are up-to-date. If you do receive some output, you know which applications require an upgrade.
Alternately, this command will show all applications that are out-of-date:
% pkg_version -L "="
If you prefer a more verbose output than =
, >
,
or <
, try this command:
% pkg_version -v | more
If for some reason you’re not using cvsup
to keep your ports tree up-to-date,
you can still check your installed ports against the latest ports
tree:
%pkg_version -v ftp://ftp.freebsd.org/pub/FreeBSD/branches/-current
/ports/INDEX | more
/usr/ports/README
man pkg_info
man pkg_delete
man pkg_version
man ports
The Installing Packages and Ports section of the FreeBSD Handbook (http://www.freebsd.org/doc/en_US.ISO8859-1/books/handbook/ports.html)
It doesn’t happen often, but occasionally portupgrade will upgrade a port to a newer version that doesn’t sit well with your system.
It can be very frustrating when an application that was working just fine an hour ago suddenly stops working after an upgrade. Now what?
At first glance, the solution isn’t obvious. Because ports don’t
contain revision labels, you can’t just cvsup
back to an earlier version. However, the
commits or changes to each port are tracked in the CVS repository. You
could learn the syntax of the cvs
command and use it to connect to the CVS
repository, manually review the port’s commit history, find an earlier
version that worked on your system, check out that version, and rebuild
the port. Whew! There must be an easier way.
That’s what Heiner Eichmann thought when he created portdowngrade
. His script does all of the work for you; you only need
to choose which version of the port to use.
Installing portdowngrade
is
easy enough:
#cd /usr/ports/sysutils/portdowngrade
#make install clean
A few moments later, you’ll have the script and an informative manpage. To run the script, simply specify which port you’d like to downgrade. Here, I’ll demonstrate an arbitrary port:
# portdowngrade apinger
portdowngrade 0.1 by Heiner Eichmann
Please note, that nothing is changed in the ports tree
unless it is explicitly permitted in step 6!
Seeking port apinger ... found: net/apinger
Step 1: Checking out port from CVS repository
CVS root directory: :pserver:anoncvs:[email protected]/home/ncvs
Step 2: Reading the port history from the CVS repository
Step 3: Analyzing the port history from the CVS repository
Step 4: Load port version numbers and present results
Keys: <space> : next page d : details
p : previous page
<enter> : leave presentation and downgrade if wanted
number date portversion comment
1 2003/11/05 15:39:39 Fix whitespace.
2 2003/06/07 11:43:13 Fix breakage.
3 2003/06/04 09:49:31 Add startup script for apinger.
4 2003/05/07 11:37:52 Change maintainer email to my @FreeBSD.
5 2003/03/28 03:41:45 Update to 0.6.1
6 2003/02/21 13:14:34 De-pkg-comment.
7 2003/01/02 17:54:17 Update to 0.6
8 2002/10/14 14:02:52 upgrade to 0.5
9 2002/10/05 19:06:00 Upgrade to 0.4.1.
10 2002/07/19 23:02:53 Update to 0.3
11 2002/07/18 12:55:14 Alarm Pinger (apinger) is a little tool
Here are the first four of six steps run by portdowngrade
. It has logged into the CVS
server, found the desired port, and presented you with its commit
history. This particular port has had 11 revisions and number 1 is the
latest.
At this point, the script pauses for user input. I’m going to go back a few revisions to Version 4:
Total lines: 11. Command:press enter
Enter version number to change port to (0: exit):4
Step 5: Checking out chosen date of the port from the CVS repository Step 6: Modifying the port Port: net/apinger at : 2003/05/07 11:37:52 Type 'yes' to bring the port to the state of the date above or 'no' to exit without changing anything. Note, that this only changes the port, not the installed software! yes or no:yes
The port has been set to the selected version. Install it if you wish. If you have portupgrade installed, you should run portsdb -Uu now, to see the changes in the ports database. In any case portupgrade -f apinger will install the changed port. Note: if you run cvsup, the port is changed back to the chosen label! #
When I typed yes
, I chose to
change the port version in the ports tree. The downgrade won’t
actually take place until I run portupgrade
-f apinger
. Note the use of the -f
flag to force the reinstallation of an
installed port. Since this port has changed in my tree, the
reinstallation will overwrite my previously installed version.
# portupgrade -f apinger
[Updating the pkgdb <format:bdb1_btree> in /var/db/pkg ... - 288
packages found (-0 +2) .. done]
---> Downgrading 'apinger-0.6.1_1' to 'apinger-0.6.1' (net/apinger)
<snip build output>
= ==> Registering installation for apinger-0.6.1
= ==> Cleaning for apinger-0.6.1
---> Cleaning out obsolete shared libraries
[Updating the pkgdb <format:bdb1_btree> in /var/db/pkg ... - 288
packages found (-0 +1) . done]
You’ll notice that the next time you run your cvsup process [Hack #80] , your downgraded port will appear as needing upgrading. If you’ve totally automated the process, it may re-upgrade to that new, buggy version.
It’s easy to prevent that from happening. In fact, you can
prevent automated upgrading of any port by using the HOLD_PKGS
array in pkgtools.conf. Start by copying the sample
configuration file to the real configuration file:
# cp /usr/local/etc/pkgtools.conf.sample /usr/local/etc/pkgtools.conf
Then, open /usr/local/etc/pkgtools.conf in your favorite editor and search for this section:
# HOLD_PKGS: array # This is a list of ports you don't want portupgrade(1) to upgrade, # portversion(1) to suggest upgrading, or pkgdb(1) to fix. # You can use wildcards ("ports glob" and "pkgname glob"). # -f/--force with each command will override the held status. # e.g.: # HOLD_PKGS = [ # 'bsdpan-*', # 'x11*/XFree86*', # ] HOLD_PKGS = [ 'bsdpan-*', ]
Simply follow the syntax to add the packages you want to keep as is:
HOLD_PKGS = [ 'bsdpan-*', 'apinger-*', ]
man cvs
man portdowngrade
The portdowngrade
home
page (http://portdowngrade.sourceforge.net)
Ensure your favorite installed applications start at boot time.
Some ports are nice enough to create their own startup scripts in /usr/local/etc/rc.d when you install them. Unfortunately, not all ports do. You may wonder why you’re not receiving any email, only to discover a week later that your mail server didn’t start at your last bootup!
In those cases, you’ll have to write your own startup script. Fortunately, that’s easy.
Every port comes with a packing list of installed executables,
files, and manpages. To see if a particular port will install a
startup script, search its pkg-plist for the word rc
. Here, I’ll check the packing lists for
the stunnel
and messagewall
ports:
%grep -w rc /usr/ports/security/stunnel/pkg-plist
etc/rc.d/stunnel.sh.sample %grep -w rc /usr/ports/mail/messagewall/pkg-plist
%
Use the -w
switch so grep
searches for the full word rc
, not just words containing those two
characters. If there isn’t a startup script, as is the case for
messagewall
, you’ll just get your
prompt back.
If the startup script ends with .sample, you’ll need to copy it to a new file without that extension. This is often the case with applications that expect you to change the sample configuration file to suit your system’s requirements.
Also, note the relative path. The packing list knows that, by
default, the files installed by a port will start with the prefix
/usr/local. That is, in the
previous example, you’ll find stunnel
’s startup script in /usr/local/etc/rc.d, not in /etc/rc.d.
Suppose you’d like to have messagewall
start automatically at boot
time. That means you’ll need to write a script. Fortunately, you don’t
have to reinvent the wheel, as all startup scripts follow the same
pattern. If you’ve installed some applications, you most likely
already have startup scripts populating /usr/local/etc/rc.d. If you don’t, use the
template startup script from the Handbook:
#!/bin/sh echo -n ' FooBar' case "$1" in start) /usr/local/bin/foobar ;; stop) kill -9 `cat /var/run/foobar.pid` ;; *) echo "Usage: `basename $0` {start|stop}" >&2 exit 64 ;; esac exit 0
This script starts a generic application named foobar
. When you copy the template, copy it
to /usr/local/etc/rc.d with the
name of the application followed by a .sh extension. In my case, I’ll call the
file /usr/local/etc/rc.d/messagewall.sh.
Next, replace the word foobar
with the name of the application. Change these three lines to reflect
the application’s name:
echo -n ' Messagewall' /usr/local/bin/messagewall kill -9 `cat /var/run/messagewall.pid`
Remember to double-check the location of that executable, as some ports instead install to /usr/local/sbin or /usr/X11R6/bin:
% which messagewall
/usr/local/bin/messagewall
Occasionally, a port will install its main binary with an odd
executable name. For example, the executable for netcat is not
netcat
. In that case, searching the
packing list will reveal all:
% grep bin /usr/ports/net/netcat/pkg-plist
bin/nc
Just remember that there’s a /usr/local in front of that bin/nc.
Once you’ve saved your changes, make the script executable with
chmod +x
. Then, see if it
works:
#/usr/local/etc/rc.d/messagewall.sh
MessagewallUsage: messagewall.sh {start|stop} #/usr/local/etc/rc.d/messagewall.sh start
<snip startup messages>
Pay attention if you receive any error messages. Often they indicate a typo in the application’s configuration file. Address those and ensure you can successfully start the application.
Once the application successfully starts, make sure you can stop it:
# /usr/local/etc/rc.d/messagewall.sh stop
<snip error message regarding PID>
Some applications, like this one, don’t record their PID in /var/run, so your script will produce an error instead of stopping the application. Most of these applications take over your prompt when you start them, so you can simply return to the terminal (or background process if you started it as such) and press Ctrl-c to end the process. This isn’t the cleanest of procedures, but it is effective nonetheless.
If you’re using FreeBSD 5.1 or higher, you might want to experiment with writing your own scripts using the new rc.d structure inherited from NetBSD. As of this writing, /etc/rc.d, or the collection of system scripts, uses this structure. In the future, /usr/local/etc/rc.d will likely migrate to this scripting style.
The new structure adds other commands, such as status
and reload
, so your scripts can do more than
start
and stop
.
When writing your own scripts, add these lines to your template:
. /etc/rc.subr
name="foo"
command="/usr/local/bin/${name}"
pidfile="/var/run/${name}.pid"
your stuff here
load_rc_config $name
run_rc_command "$1"
The first line is mandatory, as it calls the needed subroutines. Your script will also require the last two lines. Next come three variables that every script should include. There are dozens of other useful variables, so read through the scripts in /etc/rc.d/ for ideas.
I also find NetBSD’s packages list useful (see ftp://ftp.netbsd.org/pub/NetBSD/packages/pkgsrc/README-all.html). If you select a port and click on its history then files, you can look for existing scripts. These scripts are written in the NetBSD rc.d style, so you’ll have lots of examples to browse.
man rc.subr
The startup scripts section of the FreeBSD Handbook (http://www.freebsd.org/doc/en_US.ISO8859-1/books/handbook/configtuning-starting-services.html)
Use a sandbox to build applications that play nicely within your network.
Many NetBSD users are responsible for multiple systems running on different architectures. Instead of rebuilding the same package on machine after machine, it’s often desirable to build packages for all of these machines from the most powerful one, delivering the appropriate binary packages across the network. However, problems can arise when not all machines run the same version of NetBSD or when you want different optimizations or build settings on each box.
The solution to this dilemma is simple: create a sandbox with the version of NetBSD used in the target machine and build the necessary binary packages inside it. This sounds easy, but it can be a very tedious and error-prone task. It is even more complex if you want to automate periodic package rebuilding. Fortunately, that’s our final goal in this hack.
To simplify things, I assume that you have a relatively fast desktop machine running NetBSD-current, where you will build binary packages, and a server machine running the stable version of NetBSD (1.6.2 at the time of this writing).
pkg_comp
(also known as
Package Compiler) can simplify the creation of these
sandboxes: it handles any version of NetBSD inside a chroot
jail and automates the build process
of binary packages inside it. Its only restriction is that both the
builder and the destination machine share the same
architecture.
Let’s begin by installing pkg_comp
on the builder machine (make sure
you have Version 1.15 or greater):
#cd /usr/pkgsrc/pkgtools/pkg_comp
#make install && make clean
After installation, spend some time reading man 8 pkg_comp
and getting familiar with its
structure because you will be using it as a reference guide during the
configuration. Also ensure that your kernel configuration file
contains file-system NULLFS
. (See
man 4 options
for more
information.)
Now you are ready to set up pkg_comp
. The configuration file tells
pkg_comp
how to create the sandbox.
Type the following commands to create and edit a sample configuration
file:
#pkg_comp maketemplate
#vi /root/pkg_comp/default.conf
You will notice lots of variable definitions. All you need to do
is set some values; pkg_comp
handles everything else. For our purposes, you need to know only some
of these variables (see Table
8-2) and change them to suit your system.
Variable | Usage |
Gives the location of the sandbox. This needs lots of disk space, as it will store a complete NetBSD system. In this example, use /var/chroot/pkg_comp/default. | |
The location of NetBSD installation sets, whether
downloaded from the FTP site or built using | |
Specifies the version of NetBSD to unpack in the
sandbox. This version must be compatible with | |
Provides the location of | |
Identifies the destination of binary packages. In this example, use /home/NetBSD/packages/1.6.2. | |
Locates the | |
Gives the location of the NetBSD source tree in your system. In this example, use /usr/src-1.6. Because we are building for 1.6.2 and the builder is running current, this will not be /usr/src. | |
Lists the NetBSD sets to be extracted inside the sandbox. Do not change the default value. | |
Lists the X11R6 sets to be extracted inside the
sandbox. Set this to | |
The location of the pkg-vulnerabilities file in your
system. In this example, use
/usr/pkg/share. If you are not using
|
Now is the time to enable compile-time optimizations for the
packages you are going to build. As you modify the CFLAGS
and CXXFLAGS
variables, keep in mind that the
configuration file is a shell script. Remember to quote your values
properly.
After setting your values and creating all of the referenced directories, it’s time to initialize the sandbox. It is as easy as typing:
# pkg_comp makeroot
When this command finishes, the sandbox is ready to build packages for your server. In this example, the packages will linked against 1.6.2 libraries using any specified optimizations.
Suppose you want binary packages for Apache and screen
. Compile them with the following call
to pkg_comp
:
# pkg_comp build www/apache misc/screen
This will place apache-1.3.29.tgz and screen-4.0.2.tgz—as well as their
dependencies—under /home/NetBSD/packages/1.6.2/All. They’re
now suitable for transferring to the destination machine. Install them
with pkg_add
.
If you do not need to build more packages using pkg_comp
, you can safely free the space used
by the sandbox with the command shown next. Note that this removes
only the sandbox, not binary packages:
# pkg_comp removeroot
We can go one step further and configure pkg_comp
to create the sandbox, build a
predefined set of packages for your server, and remove the sandbox
when finished, all automatically. This takes only a single command
with pkg_comp
’s automatic
mode.
To enable automatic mode, re-edit the configuration file,
/root/pkg_comp/default.conf, and
define the AUTO_PACKAGES
variable.
This variable takes the list of packages you want to build for your
server. In this example:
AUTO_PACKAGES="misc/screen www/apache"
That’s it for the configuration side. To check if this works,
make sure the sandbox does not exist, and execute pkg_comp
’s automatic mode:
#pkg_comp removeroot
#pkg_comp auto
After a while, you will find binary packages for screen
and Apache in your package
repository, just as in the earlier example.
If the list of packages is extensive, the build will take a long while, which may not be desirable in some environments (for example, in cases when you need to shut down the builder during the night). This is not a problem: if you stop the automatic process with Ctrl-c at any point, you can resume it later by issuing:
# pkg_comp auto resume
To finish the automation, configure a cron
job to rebuild your package set
automatically once a week. Edit root
’s crontab
to add the line:
# crontab -e
0 3 * * * /usr/pkg/sbin/pkg_comp auto
I’ve shown the most basic usage of pkg_comp
in this hack. If you found it
useful, there are many more things to learn, and the manpage is a good
starting point.
Here are some other ideas to try:
man pkg_comp
man pkg_add
Many Mac users often seem a little surprised when I tell them I run XChat and other Unix applications on Mac OS X alongside native Aqua applications (such as Safari, Finder, and iPhoto). What they don’t realize is that it’s simple to install such applications thanks to the Fink and DarwinPorts projects. This hack is dedicated to installing and using DarwinPorts.
This hack assumes you have a basic understanding of Terminal.app and the underlying Unix bits of Mac OS X. You also need to have the Developer Tools installed.
Before you can use DarwinPorts, you must install the build system and the actual ports tree. The easiest way to accomplish this is by using CVS. Before checking the project out of CVS, you’ll need to decide where you’d like it to exist on your hard drive. I usually use ~/work.
Open Terminal.app (or an
xterm
if you have X11 installed),
and change to the directory where you’ll install DarwinPorts. Then
type the following commands at the prompt (when the server asks for a
password, just press Return):
%alias dcvs cvs -d
:pserver:[email protected]:/Volumes/src/cvs/od
%dcvs login
%dcvs co -P darwinports
You should now see a bunch of output scrolling past in the terminal window. If you do, good; the project is checking out of CVS and onto your hard disk. If you don’t, double-check the three commands just shown to make sure you typed everything correctly. Once you’ve fetched the project, it’s time to install it.
Run ls
in the terminal
window; you should see a darwinports directory. cd
to it and rerun ls
:
%cd darwinports
%ls
CVS Makefile README README.fr base doc dports www
At this point, it’s a very good idea to read the README file.
The next step is to build and install the applications that will allow you to install various ports. From the darwinports directory:
%sudo -s
<enter your password> #make && make install && make clean
By default, DarwinPorts uses /opt/local as its prefix. To change that to something else, edit /etc/ports/ports.conf.
Next, open /etc/ports/sources.conf and change the
file://
line to point to the proper
location on your system. For example:
file:///Users/jim/work/darwports/dports
Now that everything is configured, add the directory containing
DarwinPorts binaries to your shell’s path. If you’re using tcsh
(the default shell on Mac OS X 10.2 and
earlier), add the following to your ~/.cshrc file:
set path = ($path /opt/local/bin)
If you’re using bash
, as Mac
OS X 10.3 does, add the following line to your ~/.bashrc file:
export PATH=$PATH:/opt/local/bin
In order for your shell to recognize the new path, either start
a new shell or source
your
configuration file:
%source ~/.cshrc
$source ~/.bashrc
Before you can install a port, you’ll need to make sure it
exists in the ports tree. This can be done in one of two ways. The
first is using port search
, which
is very simple to use. For example, to look for xchat
:
% port search xchat
irc/xchat 1.8.11 IRC client with gtk and text interfaces
irc/xchat2 2.0.1 IRC client for gtk2
The alternative is to use the web-based interface found on the
DarwinPorts web site. You can view by category and search from this
interface, but because the PortIndex file it uses isn’t always
up-to-date, you may have better luck with the port
command.
Now that we’ve found something to install, it’s time to learn how to install it. If you’ve ever worked with the FreeBSD ports collection, this section should look very familiar to you.
Sticking with XChat as our example, we have two options. We can
install the xchat
port, which uses
GTK+ version 1, or the xchat2
port,
which uses GTK+ Version 2. For the sake of example, we’ll choose
xchat2
.
There are also two ways to install the port. The first way is to
change to the port’s directory and run port
install
:
%cd /path/to/darwinports/dports/irc/xchat2
%sudo -s
<enter your password> #port install && port clean
The second method can be run from anywhere on the filesystem:
%sudo -s
<enter your password> #port install xchat2 && port clean xchat2
As long as you have your path set properly and the port you’re trying to install is in the PortIndex, installation should proceed normally.
Since the ports developers frequently add new ports and update existing ports, you’ll want to keep your ports tree up-to-date. Doing so is fairly simple:
%cd /path/to/darwinports
%cvs -q up -Pd
If you notice changes to the base directory, you’ll want to rebuild the DarwinPorts base system as well. This is done using the same commands used to install it initially:
%cd /path/to/darwinports
%sudo -s
<enter your password> #make && make install && make clean
As you’d expect, the port
command has other options, such as uninstall
, fetch
, extract
, and build
, to name a few. Check the port
manpage for a full explanation of each
option and more information.
At the time of writing, there are over 750 ports in the DarwinPorts tree and that number is growing daily. If your favorite application isn’t already available in the ports tree, you can either create a port of it or join the DarwinPorts mailing list and request that someone else create a port of it.
man port
http://www.bsdnews.org/01/darwinports.php (the original article on BSDnews)
The DarwinPorts web site (http://darwinports.opendarwin.org/)
The DarwinPorts web interface to the ports collection (http://darwinports.opendarwin.org/ports/)
The DarwinPorts mailing list (http://www.opendarwin.org/mailman/listinfo/darwinports/)
The Fink web site (http://fink.sourceforge.net/)