How to make a Solaris package

Packages are the method Sun use to manage software on Solaris systems. Ordinary users can make them too, so I'm going to show you how. This is a very simplified version of Sun's Application Packaging Developer's Guide, with a few additions based on my experiences. Sun's guide is available from http://docs.sun.com/db/doc/805-6338?a=load (Solaris 8 version) or http://docs.sun.com/db/doc/806-7008?a=load (Solaris 9 version). I haven't yet read the Solaris 9 version, so I don't know if there are any major differences. The documentation available from Sun covers everything, but they don't really provide a step-by-step guide which takes you through making a simple package.

Q: Why would I want to make a Solaris package?

A: There are a few reasons, but basically it'll make your life easier.

  1. Self-containedness: A package is a self-contained piece of software, perhaps with some external dependencies (e.g. libraries). You know that once you have the file containing the package, you have everything you need (modulo dependencies).

  2. Easy addition/removal: pkgadd(1M) and pkgrm(1M) add/remove all the files contained in the package. You don't have to hunt around /etc or /var 1 looking for remnants, try to remember whether
    /usr/local/bin/bar is part of program foo, or put everything in /usr/local/depot/foo-3.5.7 and play simlink games with things like opt_depot(1) or stow(1)

  3. Upgrade/downgrade/reinstall made simple: Want to upgrade something?

    pkgrm foo; pkgadd -d foo-new.pkg

    Want to downgrade, because this version is buggy and produces garbage output?

    pkgrm foo; pkgadd -d foo-old.pkg

    Want to move to a new machine, and need to have all the same programs installed on the new one?

    pkginfo | grep -v SUNW # Skip Sun packages.

    How to install all the packages in the current directory:2

    yes "" | for package in *.pkg; do
            pkgadd -d "$package";
    done
    

  4. Dependencies: For me this is the "killer app" of packaging: you can specify dependencies when you make a package. E.g.: OpenSSH depends on OpenSSL and zlib. When you decide to upgrade OpenSSL, you can check which packages depend on it, and thus test which programs might break when you install the new version.

  5. Filename collisions: When you install a new package, pkgadd(1M) checks to see if any of the pathnames exist already, and asks if you want to overwrite them. It also checks for setuid/setgid files, so you don't accidentally install something nasty.

  6. Package recreation: Today you package foo-3.5.7, for Solaris 8; it takes you most of the day, because it depends on four external libraries, requires a couple of extra compiler options because you're using gcc(1) instead of Sun's Forte, and has a complicated build process that gets confused if you interrupt it. You shudder when you think about upgrading it next year - how will you remember what you had to do? Simple: record it in the package. If you follow my instructions below, you'll include a shell script in the package which will recompile the program and make it ready for packaging. It's not guaranteed to be foolproof when you try to compile a new version3, but at least it's a start.

Enough arguments in favour of packaging: lets get to the instructions.

How to do it

There are 5 steps in packaging:

  1. Extract the source code and create package documentation.

  2. Configure the program.

  3. Compile the program.

  4. Install it to a temporary location.

  5. Package it.

Steps 1, 2, 3 and 5 are easy; step 4 is the one you might have trouble with. None of these steps need to be done as root.

All the programs listed here should be in your PATH already on Computer Science or netsoc machines. If not they're in ~tobinjt/bin or can be downloaded from wherever you got this document4. All the examples will use the mythical foo program, version 3.5.7. These instructions are not cast in stone; you'll need to interpret them for yourself, as they won't work for every program. Likewise the instructions on extra files to include don't necessarily have to be followed, but experience has shown that including them helps.

Extract the source code and create package documentation
You'll
need the source tarball for this, but I can't tell you where to get that.

tar zxf foo-3.5.7.tar.gz
make_doc foo-3.5.7

The first command (tar zxf foo-3.5.7.tar.gz) extracts the source, and should put it in a directory named foo-3.5.7. The second command (make_doc foo-3.5.7) creates another directory for documentation, foo-3.5.7-doc, and places 4 files within that directory:

README
You should put a short description of the program in here, possibly what it is used for and where to get the source, documentation, help, mailing lists etc.

README.depend
This should specify this packages dependencies, and will be used when creating the package. To specify that this package (foo) is dependant on another package (bar) the format is as follows:
P bar description of bar
This means that bar is a prerequisite (P) to installing foo, and the description of bar is the description used when the bar was created (see the README.pkginfo section).

E.g. OpenSSH depends on OpenSSL, so you would put this (all on one line) in README.depend
P OpenSSL OpenSSL is an open source SSL and TLS
library, used by OpenSSH, web browsers and other
clients.

P makes this a prerequesite.
OpenSSL is the package name.
OpenSSL is an open source SSL and TLS library, used by
OpenSSH, web browsers and other clients.
is the package description.

README.install
A shell/Perl/foo script to automate steps 2, 3 and 4 of packaging. This is the file which will make your life easier when you upgrade this package to foo version 3.5.8. Its initial contents are (these lines may be wrapped):

#!/usr/bin/env bash

set -e

./configure --mandir=/usr/local/share/man \
        --infodir=/usr/local/share/info
make
make DESTDIR=/tmp/foo/foo install

README.pkginfo
This file contains information used when creating the package. Its initial contests are (beware of linewrap again):

PKG=
NAME=
VERSION=
CATEGORY=
EMAIL=John.Tobin@cs.tcd.ie
PSTAMP=John Tobin 2003/
BASEDIR=/usr/local
CLASSES=none

You might want to customise the script so that it uses your name instead of mine. The meaning of the various variables are as follows:

PKG:
The name of the package, e.g. foo, OpenSSL, vim. Should be shorted than 9 characters, but in practice that doesn't seem to be a real restriction. Must consist only of alphanumeric characters, i.e. number and letters.

NAME:
The descriptive name of the package. Can be up to 255 characters long, so try to make it as useful as possible. Good example: "OpenSSL is an open source SSL and TLS library, used by OpenSSH, web browsers and other clients." Bad example: "openssl." This name will be used in messages and dependencies, so make it as clear as possible.

VERSION:
The version of the program.

CATEGORY:
An alphanumeric description of what category this program should be in, e.g. Libraries, Programming, Utilities. Just make up something that fits the program.

EMAIL:
Your email address.

PSTAMP:
Your name and the date you made the package.

BASEDIR:
This directory will be prepended at install time to all pathnames. Usually it'll be /usr/local, but it should be / if any files are outside /usr/local.

CLASSES:
The files included in a package can be broken up into different classes, and the classes treated differently. In general you won't bother doing this, so you'll just set this value to none.

Configure the program
Read the documentation and pick which options you're compiling with, where the files are going to go, etc. You should choose the locations that the files will eventually end up in, rather than the temporary location they'll be packaged in. This is usually done by means of a configure script, but a few programs don't bother with that. Useful files from the source to look at are README, INSTALL, BUGS, TODO, doc/* and the output of
./configure -help | less

For foo, we use (lines wrapped once more):

./configure --mandir=/usr/local/share/man \
        --infodir=/usr/local/share/info

The binaries will go in /usr/local/bin, the configuration files in /usr/local/etc, manpages in /usr/local/share/man.

Compile the program
Usually just a single command:

make

If this fails you may need to go back and change the options you passed to configure, install dependencies, or possibly fix broken Makefiles/source code. Sometimes there are tests for the program, which are usually run by make test, make check or make checks. You should create/compile as much documentation about the program as you can; many programs have HTML documentation, additional manpages, info pages, etc., and your users will thank you for providing as much information as you can.

Install it to a temporary location
This is the tricky part. The program has been configured and compiled to expect its constituent parts to be in /usr/local/..., and we need to put them all into somewhere else, e.g. /tmp/foo. There are several possible ways of doing this; I'll list them in order from easiest to hardest.

  1. With programs whose Makefiles were generated with recent versions of AutoMake, this should be as simple as:

    make DESTDIR=/tmp/foo/foo install

    DESTDIR is prepended to every path created. However if the author has written their own make(1) rules, then this might not work properly. You can check for this with the following command:

    grep DESTDIR Makefile */Makefile

    If you see any output, there's a good chance that it'll work.

  2. For other programs using AutoConf, you'll probably be able to redefine ${prefix} on the command line (line wrapping, sigh):

    make prefix=/tmp/foo/foo \
            mandir='${prefix}/share/man' \
            infodir='${prefix}/share/info' \
            sysconfigdir='${prefix}/etc' \
            install
    

    Here the values of mandir, infodir and sysconfigdir are based on the value of prefix, thus the need for quoting.

  3. If either of the above commands appear to partially work, but not in subdirs, make(1) may not be passing the parameters on properly. You can use the -e switch so that make(1) will let environment variables take precedence over definitions in the Makefile:

    DESTDIR=/tmp/foo/foo make -e install

    This will work for sh(1), bash(1), ksh(1) and zsh(1) (tcsh(1) and csh(1) are left as exercises for the interested reader). Making it work for case 2 is similar.

    If the installation is failing because the ownership of the files cannot be changed, try creating a dummy chown(1) program. It can be as simple as a symlink to /bin/true, or it might record the files and their correct ownership in a temporary file.

  4. Now we're into nasty territory. Possibilities include

    • reconfiguring with a different installation path and hoping that will recreate the Makefile but not the binaries

    • using sed(1) or perl(1) to munge the Makefile

    • replacing install(1) or ./install-sh with a program to munge the paths

    • writing your own Makefile, or a shell script to perform the installation.

    If you've gotten to here, you'll really want to record how you got it to work in the package via the README.install file.

Package it
The all important part.

cd /tmp/foo
mv foo/usr/local/* foo && rmdir -p foo/usr/local
add_doc .../foo-3.5.7-doc foo/share/doc
make_package foo tmp foo-3.5.7-solX-local.pkg \
     "s/$USER $USER/root/bin/; s/0600/0644/; s/0700/0755/"

Several simple steps here:

  1. cd /tmp/foo

    Change to the directory you've installed the program into.

  2. mv foo/usr/local/* foo && rmdir -p foo/usr/local

    Remove the /usr/local/ prefix from the files. This will only be necessary if you've used DESTDIR; redefining prefix will strip /usr/local/ regardless. Don't do this if there are any files installed outside /usr/local/, e.g. in /etc or /var. Then empty directories foo/usr/local and foo/usr are unnecessary and removed.

  3. add_doc .../foo-3.5.7-doc5foo/share/doc

    add_doc creates foo/share/doc/foo-3.5.76, and copies everything from .../foo-3.5.7-doc to foo/share/doc/foo-3.5.7. It then hardlinks:
    foo/share/doc/foo-3.5.7/README.pkginfo to ./pkginfo,
    foo/share/doc/foo-3.5.7/README.depend to ./depend,
    ready for the next command. Generally add_doc will Do What I Mean with the arguments you give it.

  4. make_package foo tmp foo-3.5.7-solX-local.pkg
    "s/$USER $USER/root bin/; s/0600/0644/; s/0700/0755/"

    make_package creates a package from a directory full of files. The arguments are:

    1. The directory containing the files to be packaged (foo).

    2. The temporary directory to use (tmp).

    3. The name of the file the package ends up in; usually
      package_name-version-os_version-basedir.pkg
      (foo-3.5.7-solX-local.pkg).

    4. If this argument is given, it is passed to sed(1) and the generated prototype file filtered accordingly. This one changes the permissions and ownership of files and directories.
      ("s/$USER $USER/root bin/; s/0600/0644/; s/0700/0755/")

    make_package will generate a prototype file, determine the name of the package from the pkginfo file, run $EDITOR7 on that file so that you can make changes, create the package and then offer to remove the temporary files created during the process. You should ensure that ownership and permissions on every file are correct. You should also change the ownership and permissions on preexisting directories8 to ? ? ? so that they won't clash with the existing permissions. You may want to remove some files, e.g. static libraries if you prefer dynamic linking. When you quit the editor, the package will be created. If you've made any mistakes such as putting non-alphanumeric characters in certain pkginfo fields the packaging process will halt with an error message. If make_package offers to delete the temporary directory, you know you've been successful, and there should be a file containing the new package in the current directory.

    You can now install this with:
    pkgadd -d foo-3.5.7-sol9-local.pkg
    Woohoo!, you've made a package :)

    Because it's a single file, you can compress it, move it elsewhere, whatever you want. If you're not sure the package is correct, you can install it to an alternate root directory like so:
    pkgadd -d foo-3.5.7-sol9-local.pkg -R /some/directory

Now you're done with the first package. Play around with packaging - some programs will be trivially easy to package, others will be a nightmare (nethack comes to mind here). After you've done three or four you'll be flying. Good luck, and I hope this has helped somewhat.

This document was written as part of my job as a Systems Administrator in the Computer Science Department of Trinity College, Dublin. It is thanks to their generosity that this document has been made available to others.

Written by John Tobin $<$John.Tobin@cs.tcd.ie$>$, 2003.
Copyright Computer Science Department, Trinity College, Dublin.

$\$$Id: packaging.tex,v 1.4 2003/05/15 13:35:07 tobinjt Exp $\$$



Footnotes

.../var 1
Log files may be left behind, usually in /var. They're generally called something like /var/log/foo/error_log, and easily identifiable. If you like, you can even add them to the package, but that's a bit beyond the scope of this document.
... directory:2
This won't work if there are setuid files in the package, but you should have to be more careful with those anyway.
... version3
They may have fixed the problems, or changed the build process and created new problems, added dependencies etc.
... document4
http://www.cs.tcd.ie/John.Tobin/packaging_guide/
http://www.netsoc.tcd.ie/~tobinjt/
http://tech.netsoc.tcd.ie/
... .../foo-3.5.7-doc5
.../foo-3.5.7-doc should be replaced with the real path
...foo/share/doc/foo-3.5.76
It's smart enough to strip the -doc, and add foo-3.5.7 to the second path, if you haven't already included it.
...$EDITOR7
If you haven't set $EDITOR, make_package will use vim(1).
... directories8
Examples include /var, /etc, /usr/local/bin, and so on.


John Tobin 2003-05-17