The Ports Collection

The Internet is full of free software that is normally distributed in source form. That can be a problem in it self: the way from the source archive that you get free from the Internet to the finished, installed, running program on your machine—normally called porting, — can be a long and frustrating one. See my book Porting UNIX Software, for more details of the porting process.

To get a software package up and running on your system, you need to go through most of these steps:

1. Get the source files on your machine. They are usually contained in an archive, a file containing a number of other files. Archives used for the ports collection are generally gzipped tar, files, packaged with tar, and compressed with gzip, but other formats are also possible. What ever the format, you'll typically use ftp, to get them to your machine.
2. Unpack the archive source tree, in this case using gunzip, and tar,
3. Configure the package. Most packages include shell scripts to do this. Configuration performs a threefold adaptation of the package:
   1. It adapts it to the system hardware.
   2. It adapts it to the software environment you're running (in this case, FreeBSD).
   3. It adapts it to your personal preferences.
4. Build the package. For most packages, this involves compiling the source files and creating executable. The main tool for this purpose is make, which uses a set of rules, traditionally stored in a file called Makefile, to decide how to build the package. There is nearly always a Makefile, in the sources, but the Ports Collection includes a second one that controls the build at a higher level.
5. Install the package. This involves mainly copying the executable, configuration files and documentation created by a build to the correct place in the directory hierarchy.
6. Configure the installed software. This is similar in concept to package configuration, except that it occurs in the run-time environment. The package configuration may perform all the necessary configuration for you.

These are a lot of steps, and you'll often find they're laid through a minefield: one false move and everything blows up. To make porting and installing software easier, the FreeBSD team created a framework called the Ports Collection, which makes it trivial to perform these steps. It also provides a method of packaging and installing the resultant ported software, called packages, The CD-ROM edition of FreeBSD includes a large number of pre-built packages that can be installed directly.

In this chapter, we'll consider the following points as they relate to the FreeBSD ports collection:

• How to install a pre-compiled package. We'll look at this in the next section.
• What the ports tree is, and how to compile and install (''build'') a package. We'll look at this on page 169.
• How to create and submit a new port, on page 174.

How to install a package

In FreeBSD parlance, a package is simply a special archive that contains those files (usually executable binary files) that are installed when you build and install a port. Effectively it's a snapshot of the port build process that we saw above, taken after step 4 has completed. Compared to the full-blown port, packages are much faster to install—it's usually a matter of seconds. On the other hand, they don't give you the choice of configuration that the complete port does. The distribution CD-ROMs contain a directory packages, with a large number of pre-compiled software packages. Alternatively, you can find FreeBSD packages on many servers on the Internet—check the online handbook for some places to look.

To help maintain an overview, both ports and packages are divided into categories. They are stored in directories named after the category. See the file /usr/ports/INDEX, for a list. For example, emacs, under editors, is currently in the file packages/editors/emacs-21.2.tgz, though this name will change with updated versions of emacs. For, the latest version of the packages only, you'll find another copy without the extension in packages/Latest/emacs.tgz. To install it, you enter:

pkg_add /cdrom/packages/Latest/emacs.tgz

Alternatively, you can install packages from the sysinstall, final configuration menu shown in Figure 6-1.

Building a port

The more general way to install third-party software is with a port, The FreeBSD project uses the term port to describe the additional files needed to adapt a package to build under FreeBSD. It does not include the source code itself, though the CD-ROM distribution includes many code archives in the directory /ports/distfiles, spread over several of the CD-ROMs.

Before you get started with the ports, you need to install the port information on your system. Normally this will be in /usr/ports. This directory tree is frequently called the Ports Tree. There are a number of ways to install them.

Installing ports during system installation

The simplest way to install the Ports Collection is when you install the system. When you choose the components to install, sysinstall, offers to install the Ports Collection for you as well.

Installing ports from the first CD-ROM

The file ports/ports.tgz, on the first CD-ROM is a tar, archive containing all the ports. If you didn't install it during system installation, use the following method to install the complete collection (about 200 MB). Make sure your CD-ROM is mounted (in this example on /cdrom), and enter:

#  cd /usr
#  tar xzvf /cdrom/ports/ports.tgz

If you only want to extract a single package, say inn, which is in the categorynews, enter :

#  cd /usr
#  tar xzvf /cdrom/ports/ports.tgz ports/news/inn

It takes a surprisingly long time to install the ports; although there isn't much data in the archive, there are about 250,000 files in it, and creating that many files takes a lot of disk I/O.

Installing ports from the live file system CD-ROM

Alternatively, the files are also on the live file system CD-ROM. This is not much of an advantage for installation, but you may find it convenient to browse through the source trees in the directory ports, on the CD-ROM. Let's assume you have found a directory /cdram/ports/graphics/hp scan, on the CD-ROM, and it is your current working directory. You can move the data across with the following:

#  cd /cdram/ports/graphics
#  mkdir -p /usr/ports/graphics
#  tar cf - . | (cd /usr/ports/graphics; tar xvf -)

Getting new ports

What happens when a new version of a port comes out? For example, you've been using Emacs Version 20 forever, and now Version 21.2 becomes available? It's brand new, so it's obviously not on your CD-ROM.

One way to get the port is via ftp. This used to be quite convenient: you could download a tar ball directly and extract it locally. That is unfortunately no longer possible: currently you must download files a directory at a time. If you're following the Ports Collection at all closely, you should consider using cvsup, which can keep your sources up to date automatically. See "Chapter 31" , page 585, for more details.

All ports are kept in subdirectories of the URL ftp://ftp.FreeBSD.org/pub/FreeBSD/ports/. This directory has the following contents:

drwxr-xr-x    6  1006  1006     512  Jun   8  13:18  alpha
drwxr-xr-x  209  1006  1006  401408  May  28  14:08  dist files
drwxr-xr-x    6  1006  1006    1536  May  28  17:53  i386
drwxr-xr-x    3  1006  1006     512  Apr   6  13:45  ia64
drwxr-xr-x   83  1006  1006    3072  May  20  15:35  local-dist files
lrwxrwxrwx    1  root  wheel     13  Jun   1   2001  packages -> i386/packages
lrwxrwxrwx    1  root  wheel     24  Jun   1   2001  ports -../FreeBSD-current/ports
lrwxrwxrwx    1  root  wheel      5  Jun   1   2001  ports-current -> ports
lrwxrwxrwx    1  root  wheel      5  Jun   1   2001  ports-stable -> ports
drwxr-xr-x    4  1006  1006     512  Apr   9  10:37  sparc64

The directories alpha, i386, ia64, and sparc64, contain packages (not ports) for the corresponding architecture. dist files contains a large number of the original sources for the third-party packages; it's intended as a ''last resort'' location if you can't find them at other locations.

The directory local-dist files, is used by people working on the Ports Collection; you don't normally need anything from these directories. The important directories for you are ports, ports-current, and ports-stable. Currently these are really all the same directory, but things may not remain like that.

Getting back to your emacs, port: you would find it in the directory /pub/Free-BSD/ports/ports/editors/ . Note the final / in that directory name: if you leave it out, ftp prints an error message and exits. Here's what might happen:

$ ftp ftp://ftp.FreeBSD.org/pub/FreeBSD/ports/ports/editors/
Connected to ftp.beastie.tdk.net.
220 ftp.beastie.tdk.net FTP server (Version 6.00LS) ready.
331 Guest login ok, send your email address as password.
230- The FreeBSD mirror at Tele Danmark Internet.
...much blurb omitted
250 CWD command successful.
250 CWD command successful.
ftp> ls
229 Entering Extended Passive Mode (|||55649|)
150 Opening ASCII mode data connection for '/bin/ls'.
total 704

drwxr-xr-x  3  1006  1006  512  May  20  10:07  emacs
drwxr-xr-x  4  1006  1006  512  May  20  10:08  emacs20
drwxr-xr-x  4  1006  1006  512  May  20  10:08  emacs20-dl
drwxr-xr-x  4  1006  1006  512  May  20  10:08  emacs20-mule-devel
drwxr-xr-x  3  1006  1006  512  May  20  10:08  emacs21
drwxr-xr-x  2  1006  1006  512  May  20  10:08  eshell-emacs20

This shows that your files will be in the directory emacs21. You can get them with the ftp mget, command:

ftp> mget emacs21
mget emacs21/files [anpqy?]? a           answer a for all files
Prompting off for duration of mget.
ftp: local: emacs21/files: No such file or directory
ftp: local: emacs21/Makefile: No such file or directory
(etc)

This happens because you need to create the destination directory manually. Try again:

ftp> !mkdir emacs21                      create the local directory
ftp> mget emacs21
mget emacs21/files [anpqy?]? a
Prompting off for duration of mget.
229 Entering Extended Passive Mode (|||57074|)
550 emacs21/files: not a plain file.
229 Entering Extended Passive Mode (|||57085|)
150 Opening BINARY mode data connection for 'emacs21/Makefile'   (2185 bytes).
100%  |*************************************|   2185   2.34 MB/s   00:00 ETA
226 Transfer complete.
(etc)

You get one of these for each file transferred. But note the error message: not a plain file. emacs21/files, is a directory, so we need to get it separately:

ftp> !mkdir emacs21/files
ftp> mget emacs21/files
mget emacs21/files/patch-lib-src:Makefile.in [anpqy?]? a
Prompting off for duration of mget.
229 Entering Extended Passive Mode (|||57258|)
150 Opening BINARY mode data connection for 'emacs21/files/patch-lib-src: Make file. in '(908 bytes).
100%  |*************************************|      908  1.64 MB/s       00:00 ETA
226 Transfer complete.
(etc)

Note that the ftp command specifies the URL of the directory. It must have a trailing /, otherwise ftp will complain. This form is supported by FreeBSD ftp but many other ftp clients will require you to do it in two steps:

# ftp ftp.EreeBSD.org
Connected to ftp.beastie.tdk.net.
(etc)
ftp> cd /pub/FreeBSD/ports/ports/editors
250 CWD command successful.