Before you download any pkgsrc files, you should decide whether you want the current branch or the stable branch. The latter is forked on a quarterly basis from the current branch and only gets modified for security updates. The names of the stable branches are built from the year and the quarter, for example 2019Q2.

The second step is to decide how you want to download pkgsrc. You can get it as a tar file or via CVS. Both ways are described here.

Note that tar archive contains CVS working copy. Thus you can switch to using CVS at any later time.

$ ftp ftp://ftp.NetBSD.org/pub/pkgsrc/pkgsrc-2019Q2/pkgsrc.tar.gz

$ tar -xzf pkgsrc.tar.gz -C /usr

$ ftp ftp://ftp.NetBSD.org/pub/pkgsrc/current/pkgsrc.tar.gz

$ cd /usr && cvs -q -z2 -d anoncvs@anoncvs.NetBSD.org:/cvsroot checkout -r pkgsrc-2019Q2 -P pkgsrc

$ cd /usr && cvs -q -z2 -d anoncvs@anoncvs.NetBSD.org:/cvsroot checkout -P pkgsrc

$ cd /usr && env CVS_RSH=ssh cvs -q -z2 -d anoncvs@anoncvs.NetBSD.org:/cvsroot checkout -P pkgsrc

# set CVS remote shell command
CVS_RSH=ssh
export CVS_RSH

# recommended CVS configuration file from the pkgsrc guide
cvs -q
checkout -P
update -dP
diff -upN
rdiff -u
release -d

The preferred way to keep pkgsrc up-to-date is via CVS (which also works if you have first installed it via a tar file). It saves bandwidth and hard disk activity, compared to downloading the tar file again.

$ cd /usr/pkgsrc && cvs update -dP

$ cd /usr/pkgsrc && env CVS_RSH=ssh cvs up -dP

See Section 5.1, “Using binary packages”.

pkgsrc can be bootstrapped for use in two different modes: privileged and unprivileged one. In unprivileged mode in contrast to privileged one all programs are installed under one particular user and cannot utilise privileged operations (packages don't create special users and all special file permissions like setuid are ignored).

Installing the bootstrap kit from source should be as simple as:

# env CVS_RSH=ssh cvs -d anoncvs@anoncvs.NetBSD.org:/cvsroot checkout -P pkgsrc
# cd pkgsrc/bootstrap
# ./bootstrap
    

To bootstrap in unprivileged mode pass “--unprivileged” flag to bootstrap

By default, in privileged mode pkgsrc uses /usr/pkg for prefix where programs will be installed in, and /usr/pkg/pkgdb for the package database directory where pkgsrc will do its internal bookkeeping, /var is used as varbase, where packages install their persistent data. In unprivileged mode pkgsrc uses ~/pkg for prefix, ~/pkg/pkgdb for the package database, and ~/pkg/var for varbase.

You can change default layout using command-line arguments. Run “./bootstrap --help” to get details.

On the cdn.NetBSD.org site and mirrors, there are collections of binary packages, ready to be installed. These binary packages have been built using the default settings for the directories, that is:

If you cannot use these directories for whatever reasons (maybe because you're not root), you cannot use these binary packages, but have to build the packages yourself, which is explained in Section 4.2, “Bootstrapping pkgsrc”.

# PATH="/usr/pkg/sbin:$PATH"
# PKG_PATH="http://cdn.NetBSD.org/pub/pkgsrc/packages/OPSYS/ARCH/VERSIONS/All/"
# export PATH PKG_PATH

# pkg_add libreoffice
# pkg_add ap24-php71-*

# pkg_delete -r jpeg
    

Package samba-2.0.9 has a local-root-shell vulnerability, see
    http://www.samba.org/samba/whatsnew/macroexploit.html

# Download vulnerabilities file
0 3 * * * /usr/pkg/sbin/pkg_admin fetch-pkg-vulnerabilities >/dev/null 2>&1
# Audit the installed packages and email results to root
9 3 * * * /usr/pkg/sbin/pkg_admin audit |mail -s "Installed package audit result" \
	    root >/dev/null 2>&1
      

fetch_pkg_vulnerabilities=YES
      

check_pkg_vulnerabilities=YES
      

% lintpkgsrc -i
...
Version mismatch: 'tcsh' 6.09.00 vs 6.10.00
    

After obtaining pkgsrc, the pkgsrc directory now contains a set of packages, organized into categories. You can browse the online index of packages, or run make readme from the pkgsrc directory to build local README.html files for all packages, viewable with any web browser such as www/lynx or www/firefox.

The default prefix for installed packages is /usr/pkg. If you wish to change this, you should do so by setting LOCALBASE in mk.conf. You should not try to use multiple different LOCALBASE definitions on the same system (inside a chroot is an exception).

The rest of this chapter assumes that the package is already in pkgsrc. If it is not, see Part II, “The pkgsrc developer's guide” for instructions how to create your own packages.

DISTDIR=/cdrom/pkgsrc/distfiles

% make fetch-list | sh

% cd misc/figlet
% make
    

% make install
    

% make clean
    

% make clean-depends
    

The following variables apply to all pkgsrc packages. A complete list of the variables that can be configured by the user is available in mk/defaults/mk.conf, together with some comments that describe each variable's intent.

Packages have to support installation into a subdirectory of WRKDIR. This allows a package to be built, before the actual filesystem is touched. DESTDIR support exists in two variations:

With basic DESTDIR support, make clean needs to be run as root.

Considering the foo/bar package, DESTDIR full support can be tested using the following commands

$ id
uid=1000(myusername) gid=100(users) groups=100(users),0(wheel)
$ mkdir $HOME/packages
$ cd $PKGSRCDIR/foo/bar

Verify DESTDIR full support, no root privileges should be needed

$ make stage-install

Create a package without root privileges

$ make PACKAGES=$HOME/packages package

For the following command, you must be able to gain root privileges using su(1)

$ make PACKAGES=$HOME/packages install

Then, as a simple user

$ make clean

Some packages have build time options, usually to select between different dependencies, enable optional support for big dependencies or enable experimental features.

To see which options, if any, a package supports, and which options are mutually exclusive, run make show-options, for example:

    The following options are supported by this package:
        ssl      Enable SSL support.
    Exactly one of the following gecko options is required:
        firefox  Use firefox as gecko rendering engine.
        mozilla  Use mozilla as gecko rendering engine.
    At most one of the following database options may be selected:
        mysql    Enable support for MySQL database.
        pgsql    Enable support for PostgreSQL database.

    These options are enabled by default: firefox
    These options are currently enabled: mozilla ssl

The following variables can be defined in mk.conf to select which options to enable for a package: PKG_DEFAULT_OPTIONS, which can be used to select or disable options for all packages that support them, and PKG_OPTIONS.pkgbase, which can be used to select or disable options specifically for package pkgbase. Options listed in these variables are selected, options preceded by “-” are disabled. A few examples:

$ grep "PKG.*OPTION" mk.conf
PKG_DEFAULT_OPTIONS=    -arts -dvdread -esound
PKG_OPTIONS.kdebase=    debug -sasl
PKG_OPTIONS.apache=     suexec 

It is important to note that options that were specifically suggested by the package maintainer must be explicitly removed if you do not wish to include the option. If you are unsure you can view the current state with make show-options.

The following settings are consulted in the order given, and the last setting that selects or disables an option is used:

For groups of mutually exclusive options, the last option selected is used, all others are automatically disabled. If an option of the group is explicitly disabled, the previously selected option, if any, is used. It is an error if no option from a required group of options is selected, and building the package will fail.

Before the options framework was introduced, build options were selected by setting a variable (often named USE_FOO) in mk.conf for each option. To ease transition to the options framework for the user, these legacy variables are converted to the appropriate options setting (PKG_OPTIONS.pkgbase) automatically. A warning is issued to prompt the user to update mk.conf to use the options framework directly. Support for the legacy variables will be removed eventually.

Once you have built and installed a package, you can create a binary package which can be installed on another system with pkg_add(1). This saves having to build the same package on a group of hosts and wasting CPU time. It also provides a simple means for others to install your package, should you distribute it.

To create a binary package, change into the appropriate directory in pkgsrc, and run make package:

$ cd misc/figlet
$ make package

This will build and install your package (if not already done), and then build a binary package from what was installed. You can then use the pkg_* tools to manipulate it. Binary packages are created by default in /usr/pkgsrc/packages, in the form of a gzipped tar file. See Section B.2, “Packaging figlet” for a continuation of the above misc/figlet example.

See Chapter 23, Submitting and Committing for information on how to submit such a binary package.

See Section 19.17, “Other helpful targets”.

First of all, you have to decide whether you build all packages or a limited set of them. Full bulk builds usually consume a lot more resources, both space and time, than builds for some practical sets of packages. There exists a number of particularly heavy packages that are not actually interesting to a wide audience. For a limited bulk builds you need to make a list of packages you want to build. Note that all their dependencies will be built, so you don't need to track them manually.

During bulk builds various packages are installed and deinstalled in /usr/pkg (or whatever LOCALBASE is), so make sure that you don't need any package during the builds. Essentially, you should provide a fresh system, either a chroot environment or something even more restrictive, depending on what the operating system provides, or dedicate the whole physical machine. As a useful side effect this makes sure that bulk builds cannot break anything in your system. There have been numerous cases where certain packages tried to install files outside the LOCALBASE or wanted to edit some files in /etc.

Running a pbulk-style bulk build works roughly as follows:

# (cd /usr && ftp -o - https://cdn.NetBSD.org/pub/pkgsrc/current/pkgsrc.tar.gz | tar -zxf-)
# (cd /usr && fetch -o - https://cdn.NetBSD.org/pub/pkgsrc/current/pkgsrc.tar.gz | tar -zxf-)
# (cd /usr && cvs -Q -z3 -d anoncvs@anoncvs.NetBSD.org:/cvsroot get -P pkgsrc)

# sh pbulk.sh -n # use native make, no bootstrap kit needed (for use on NetBSD)
# sh pbulk.sh -n -c mk.conf.frag # native, apply settings from given mk.conf fragment
# sh pbulk.sh -nlc mk.conf.frag # native, apply settings, configure for limited build

PKG_DEVELOPER=          yes     # perform more checks
X11_TYPE=               modular # use pkgsrc X11
SKIP_LICENSE_CHECK=     yes     # accept all licences (useful
                                # when building all packages)

www/firefox
mail/thunderbird
misc/libreoffice4

# /usr/pbulk/bin/bulkbuild

A complete bulk build requires lots of disk space. Some of the disk space can be read-only, some other must be writable. Some can be on remote filesystems (such as NFS) and some should be local. Some can be temporary filesystems, others must survive a sudden reboot.

After your pkgsrc bulk-build has completed, you may wish to create a CD-ROM set of the resulting binary packages to assist in installing packages on other machines. The pkgtools/cdpack package provides a simple tool for creating the ISO 9660 images. cdpack arranges the packages on the CD-ROMs in a way that keeps all the dependencies for a given package on the same CD as that package.

# mkdir /u2/images
# pkg_add /usr/pkgsrc/packages/All/cdpack
# cdpack /usr/pkgsrc/packages/All /u2/images
      

# mkdir /tmp/common
# echo "This is a README" > /tmp/common/README
# echo "Another file" > /tmp/common/COPYING
# mkdir /tmp/common/bin
# echo "#!/bin/sh" > /tmp/common/bin/myscript
# echo "echo Hello world" >> /tmp/common/bin/myscript
# chmod 755 /tmp/common/bin/myscript
      

# cdpack -x /tmp/common /usr/pkgsrc/packages/All /u2/images

The following directories exist in a typical pkgsrc installation in ${LOCALBASE}.

The following mailing lists may be of interest to pkgsrc users:

To subscribe, do:

% echo subscribe listname | mail majordomo@NetBSD.org

Archives for all these mailing lists are available from http://mail-index.NetBSD.org/.

The directory pkgsrc/pkgtools contains a number of useful utilities for both users and developers of pkgsrc. This section attempts only to make the reader aware of some of the utilities and when they might be useful, and not to duplicate the documentation that comes with each package.

Utilities used by pkgsrc (automatically installed when needed):

OS tool augmentation (automatically installed when needed):

Utilities used by pkgsrc (not automatically installed):

Utilities for keeping track of installed packages, being up to date, etc:

Utilities for people maintaining or creating individual packages:

Utilities for people maintaining pkgsrc (or: more obscure pkg utilities)

To install packages from source as a non-root user, download pkgsrc as described in Chapter 3, Where to get pkgsrc and how to keep it up-to-date, cd into that directory and run the command ./bootstrap/bootstrap --unprivileged.

This will install the binary part of pkgsrc to ~/pkg and put the pkgsrc configuration mk.conf into ~/pkg/etc.

For more details, see mk/unprivileged.mk.

By default, resuming transfers in pkgsrc is disabled, but you can enable this feature by adding the option PKG_RESUME_TRANSFERS=YES into mk.conf. If, during a fetch step, an incomplete distfile is found, pkgsrc will try to resume it.

You can also use a different program than the platform default program by changing the FETCH_USING variable. You can specify the program by using of ftp, fetch, wget or curl. Alternatively, fetching can be disabled by using the value manual. A value of custom disables the system defaults and dependency tracking for the fetch program. In that case you have to provide FETCH_CMD, FETCH_BEFORE_ARGS, FETCH_RESUME_ARGS, FETCH_OUTPUT_ARGS, FETCH_AFTER_ARGS.

For example, if you want to use wget to download, you'll have to use something like:

FETCH_USING=    wget

If you want to use modular X.org from pkgsrc instead of your system's own X11 (/usr/X11R6, /usr/openwin, ...) you will have to add the following line into mk.conf:

X11_TYPE=modular

If you are sitting behind a firewall which does not allow direct connections to Internet hosts (i.e. non-NAT), you may specify the relevant proxy hosts. This is done using an environment variable in the form of a URL, e.g. in Amdahl, the machine “orpheus.amdahl.com” is one of the firewalls, and it uses port 80 as the proxy port number. So the proxy environment variables are:

ftp_proxy=ftp://orpheus.amdahl.com:80/
http_proxy=http://orpheus.amdahl.com:80/

Some fetch tools are not prepared to support HTTPS by default (for example, the one in NetBSD 6.0), or the one installed by the pkgsrc bootstrap (to avoid an openssl dependency that low in the dependency graph).

Usually you won't notice, because distribution files are mirrored weekly to “ftp.NetBSD.org”, but that might not be often enough if you are following pkgsrc-current. In that case, set FETCH_USING in your mk.conf file to “curl” or “wget”, which are both compiled with HTTPS support by default. Of course, these tools need to be installed before you can use them this way.

This depends on which utility is used to retrieve distfiles. From bsd.pkg.mk, FETCH_CMD is assigned the first available command from the following list:

On a default NetBSD installation, this will be /usr/bin/ftp, which automatically tries passive connections first, and falls back to active connections if the server refuses to do passive. For the other tools, add the following to your mk.conf file: PASSIVE_FETCH=1.

Having that option present will prevent /usr/bin/ftp from falling back to active transfers.

You would like to download all the distfiles in a single batch from work or university, where you can't run a make fetch. There is an archive of distfiles on ftp.NetBSD.org, but downloading the entire directory may not be appropriate.

The answer here is to do a make fetch-list in /usr/pkgsrc or one of its subdirectories, carry the resulting list to your machine at work/school and use it there. If you don't have a NetBSD-compatible ftp(1) (like tnftp) at work, don't forget to set FETCH_CMD to something that fetches a URL:

At home:

% cd /usr/pkgsrc
% make fetch-list FETCH_CMD=wget DISTDIR=/tmp/distfiles >/tmp/fetch.sh
% scp /tmp/fetch.sh work:/tmp

At work:

% sh /tmp/fetch.sh

then tar up /tmp/distfiles and take it home.

If you have a machine running NetBSD, and you want to get all distfiles (even ones that aren't for your machine architecture), you can do so by using the above-mentioned make fetch-list approach, or fetch the distfiles directly by running:

% make mirror-distfiles

If you even decide to ignore NO_{SRC,BIN}_ON_{FTP,CDROM}, then you can get everything by running:

% make fetch NO_SKIP=yes

When compiling the pkgtools/pkg_install package, you get the error from make that it doesn't know how to make /usr/share/tmac/tmac.andoc? This indicates that you don't have installed the “text” set (nroff, ...) from the NetBSD base distribution on your machine. It is recommended to do that to format man pages.

In the case of the pkgtools/pkg_install package, you can get away with setting NOMAN=YES either in the environment or in mk.conf.

You didn't install the compiler set, comp.tgz, when you installed your NetBSD machine. Please get and install it, by extracting it in /:

# cd /
# tar --unlink -zxvpf .../comp.tgz

comp.tgz is part of every NetBSD release. Get the one that corresponds to your release (determine via uname -r).

When installing packages as non-root user and using the just-in-time su(1) feature of pkgsrc, it can become annoying to type in the root password for each required package installed. To avoid this, the sudo package can be used, which does password caching over a limited time. To use it, install sudo (either as binary package or from security/sudo) and then put the following into your mk.conf, somewhere after the definition of the LOCALBASE variable:

.if exists(${LOCALBASE}/bin/sudo)
SU_CMD=        ${LOCALBASE}/bin/sudo /bin/sh -c
.endif

As the system administrator, you can choose where configuration files are installed. The default settings make all these files go into ${PREFIX}/etc or some of its subdirectories; this may be suboptimal depending on your expectations (e.g., a read-only, NFS-exported PREFIX with a need of per-machine configuration of the provided packages).

In order to change the defaults, you can modify the PKG_SYSCONFBASE variable (in mk.conf) to point to your preferred configuration directory; some common examples include /etc or /etc/pkg.

Furthermore, you can change this value on a per-package basis by setting the PKG_SYSCONFDIR.${PKG_SYSCONFVAR} variable. PKG_SYSCONFVAR's value usually matches the name of the package you would like to modify, that is, the contents of PKGBASE.

Note that after changing these settings, you must rebuild and reinstall any affected packages.

Please be aware that there can often be bugs in third-party software, and some of these bugs can leave a machine vulnerable to exploitation by attackers. In an effort to lessen the exposure, the NetBSD packages team maintains a database of known-exploits to packages which have at one time been included in pkgsrc. The database can be downloaded automatically, and a security audit of all packages installed on a system can take place. To do this, refer to the following two tools (installed as part of the pkgtools/pkg_install package):

Use of these tools is strongly recommended! See Section 5.1.5, “Checking for security vulnerabilities in installed packages” for instructions on how to automate checking and reporting.

If this database is installed, pkgsrc builds will use it to perform a security check before building any package.

When you add your own preferences to the CFLAGS variable in your mk.conf, these flags are passed in environment variables to the ./configure scripts and to make(1). Some package authors ignore the CFLAGS from the environment variable by overriding them in the Makefiles of their package.

Currently there is no solution to this problem. If you really need the package to use your CFLAGS you should run make patch in the package directory and then inspect any Makefile and Makefile.in for whether they define CFLAGS explicitly. Usually you can remove these lines. But be aware that some “smart” programmers write so bad code that it only works for the specific combination of CFLAGS they have chosen.

You have modified a file from pkgsrc, and someone else has modified that same file afterwards in the CVS repository. Both changes are in the same region of the file, so when you updated pkgsrc, the cvs command marked the conflicting changes in the file. Because of these markers, the file is no longer a valid Makefile.

Have a look at that file, and if you don't need your local changes anymore, you can remove that file and run cvs -q update -dP in that directory to download the current version.

Building, installation and creation of a binary package are all controlled by the package's Makefile. The Makefile describes various things about a package, for example from where to get it, how to configure, build, and install it.

A package Makefile contains several sections that describe the package.

In the first section there are the following variables, which should appear exactly in the order given here. The order and grouping of the variables is mostly historical and has no further meaning.

The second section contains information about separately downloaded patches, if any.

The third section contains the following variables.

Other variables that affect the build:

Please pay attention to the following gotchas:

The distinfo file contains the message digest, or checksum, of each distfile needed for the package. This ensures that the distfiles retrieved from the Internet have not been corrupted during transfer or altered by a malign force to introduce a security hole. To provide maximum security, all distfiles are protected using three different message digest algorithms (SHA1, RMD160, SHA512), as well as the file size.

The distinfo file also contains the checksums for all the patches found in the patches directory (see Section 13.3, “patches/*”). These checksums ensure that patches are only applied intentionally and that they don't accidentally change, e.g. when merging different changes together. They also make sure that new patches are actually added to CVS and old ones are removed. Too see whether the patches and the distinfo file match, run pkglint after changing the patches.

To regenerate the distinfo file, use the make distinfo command.

Some packages have different sets of distfiles depending on the platform, for example lang/openjdk7. These are kept in the same distinfo file and care should be taken when upgrading such a package to ensure distfile information is not lost.

Some packages don't work out-of-the box on the various platforms that are supported by pkgsrc. These packages need to be patched to make them work. The patch files can be found in the patches/ directory.

In the patch phase, these patches are applied to the files in WRKSRC directory after extracting them, in alphabetic order.

PATCHDIR= ${.CURDIR}/../xemacs/patches

case ${target_os} in
netbsd*) have_kvm=yes ;;
*)       have_kvm=no  ;;
esac

AC_CHECK_LIB(kvm, kvm_open, have_kvm=yes, have_kvm=no)

#if defined(__NetBSD__)
#  include <sys/event.h>
#endif

#if defined(HAVE_SYS_EVENT_H)
#  include <sys/event.h>
#endif

int
monitor_file(...)
{
#if defined(__NetBSD__)
        int fd = kqueue();
        ...
#else
        ...
#endif
}

int
monitor_file(...)
{
#if defined(HAVE_KQUEUE)
        int fd = kqueue();
        ...
#else
        ...
#endif
}

When you type make, the distribution files are unpacked into the directory denoted by WRKDIR. It can be removed by running make clean. Besides the sources, this directory is also used to keep various timestamp files. The directory gets removed completely on clean. The default is ${.CURDIR}/work or ${.CURDIR}/work.${MACHINE_ARCH} if OBJMACHINE is set.

If you have any files that you wish to be placed in the package prior to configuration or building, you could place these files here and use a ${CP} command in the “pre-configure” target to achieve this. Alternatively, you could simply diff the file against /dev/null and use the patch mechanism to manage the creation of this file.

If you want to share files in this way with other packages, set the FILESDIR variable to point to the other package's files directory, e.g.:

FILESDIR=${.CURDIR}/../xemacs/files

Makefile variables contain strings that can be processed using the five operators =, +=, ?=, := and !=, which are described in the make(1) man page.

When a variable's value is parsed from a Makefile, the hash character # and the backslash character \ are handled specially. If a backslash is the last character in a line, that backslash is removed from the line and the line continues with the next line of the file.

The # character starts a comment that reaches until the end of the line. To get an actual # character, such as in a URL, write \# instead.

The evaluation of variables either happens immediately or lazy. It happens immediately when the variable occurs on the right-hand side of the := or the != operator, in a .if condition or a .for loop. In the other cases, it is evaluated lazily.

Some of the modifiers split the string into words and then operate on the words, others operate on the string as a whole. When a string is split into words, double quotes and single quotes are interpreted as delimiters, just like in sh(1).

Be sure to add a RCS ID line as the first thing in any PLIST file you write:

@comment $NetBSD $

An artificial space has been added between NetBSD and $, this is a workaround here to prevent CVS expanding to the filename of the guide. When adding the RCS ID the space should be omitted.

You can use the make print-PLIST command to output a PLIST that matches any new files since the package was extracted. See Section 19.17, “Other helpful targets” for more information on this target.

The PRINT_PLIST_AWK variable takes a set of AWK patterns and actions that are used to filter the output of print-PLIST. You can append any chunk of AWK scripting you like to it, but be careful with quoting.

For example, to get all files inside the libdata/foo directory removed from the resulting PLIST:

PRINT_PLIST_AWK+=       /^libdata\/foo/ { next; }

A number of variables are substituted automatically in PLISTs when a package is installed on a system. This includes the following variables:

For a list of values which are replaced by default, the output of make help topic=PLIST_SUBST as well as searching the pkgsrc/mk directory with grep for PLIST_SUBST should help.

If you want to change other variables not listed above, you can add variables and their expansions to this variable in the following way, similar to MESSAGE_SUBST (see Section 13.5, “Optional files”):

PLIST_SUBST+=   SOMEVAR="somevalue"

This replaces all occurrences of “${SOMEVAR}” in the PLIST with “somevalue”.

The PLIST_VARS variable can be used to simplify the common case of conditionally including some PLIST entries. It can be done by adding PLIST_VARS+=foo and setting the corresponding PLIST.foo variable to yes if the entry should be included. This will substitute “${PLIST.foo}” in the PLIST with either “""” or “"@comment "”. For example, in Makefile:

PLIST_VARS+=    foo
.if condition
PLIST.foo=      yes
.else

And then in PLIST:

@comment $NetBSD $
bin/bar
man/man1/bar.1
${PLIST.foo}bin/foo
${PLIST.foo}man/man1/foo.1
${PLIST.foo}share/bar/foo.data

An artificial space has been added between NetBSD and $, this is a workaround here to prevent CVS expanding to the filename of the guide. When adding the RCS ID the space should be ommited.

Man pages should be installed in compressed form if MANZ is set (in bsd.own.mk), and uncompressed otherwise. To handle this in the PLIST file, the suffix “.gz” is appended/removed automatically for man pages according to MANZ and MANCOMPRESSED being set or not, see above for details. This modification of the PLIST file is done on a copy of it, not PLIST itself.

To use one or more files as source for the PLIST used in generating the binary package, set the variable PLIST_SRC to the names of that file(s). The files are later concatenated using cat(1), and the order of things is important. The default for PLIST_SRC is ${PKGDIR}/PLIST.

Some packages decide to install a different set of files based on the operating system being used. These differences can be automatically handled by using the following files:

Some packages decide to generate hard-to-guess file names during installation that are hard to wire down.

In such cases, you can set the GENERATE_PLIST variable to shell code terminated (with a semicolon) that will output PLIST entries which will be appended to the PLIST

You can find one example in editors/xemacs:

GENERATE_PLIST+=        ${ECHO} bin/${DISTNAME}-`${WRKSRC}/src/xemacs -sd`.dmp ;

which will append something like bin/xemacs-21.4.23-54e8ea71.dmp to the PLIST.

A “shared directory” is a directory where multiple (and unrelated) packages install files. These directories were problematic because you had to add special tricks in the PLIST to conditionally remove them, or have some centralized package handle them.

In pkgsrc, it is now easy: Each package should create directories and install files as needed; pkg_delete will remove any directories left empty after uninstalling a package.

If a package needs an empty directory to work, create the directory during installation as usual, and also add an entry to the PLIST:

@pkgdir path/to/empty/directory

or take a look at MAKE_DIRS and OWN_DIRS.

The process of converting packages to use the buildlink3 framework (“bl3ifying”) is fairly straightforward. The things to keep in mind are:

If a dependency on a particular package is required for its libraries and headers, then we replace:

DEPENDS+=       foo>=1.1.0:../../category/foo

with

.include "../../category/foo/buildlink3.mk"

The buildlink3.mk files usually define the required dependencies. If you need a newer version of the dependency when using buildlink3.mk files, then you can define it in your Makefile; for example:

BUILDLINK_API_DEPENDS.foo+=   foo>=1.1.0
.include "../../category/foo/buildlink3.mk"

There are several buildlink3.mk files in pkgsrc/mk that handle special package issues:

The comments in those buildlink3.mk files provide a more complete description of how to use them properly.

A package's buildlink3.mk file is included by Makefiles to indicate the need to compile and link against header files and libraries provided by the package. A buildlink3.mk file should always provide enough information to add the correct type of dependency relationship and include any other buildlink3.mk files that it needs to find headers and libraries that it needs in turn.

To generate an initial buildlink3.mk file for further editing, Rene Hexel's pkgtools/createbuildlink package is highly recommended. For most packages, the following command will generate a good starting point for buildlink3.mk files:

% cd pkgsrc/category/pkgdir
% createbuildlink >buildlink3.mk
    

# $NetBSD: buildlink3.mk,v 1.16 2009/03/20 19:24:45 joerg Exp $

BUILDLINK_TREE+=        tiff

.if !defined(TIFF_BUILDLINK3_MK)
TIFF_BUILDLINK3_MK:=

BUILDLINK_API_DEPENDS.tiff+=    tiff>=3.6.1
BUILDLINK_ABI_DEPENDS.tiff+=    tiff>=3.7.2nb1
BUILDLINK_PKGSRCDIR.tiff?=      ../../graphics/tiff

.include "../../devel/zlib/buildlink3.mk"
.include "../../graphics/jpeg/buildlink3.mk"
.endif # TIFF_BUILDLINK3_MK

BUILDLINK_TREE+=        -tiff

Some packages in pkgsrc install headers and libraries that coincide with headers and libraries present in the base system. Aside from a buildlink3.mk file, these packages should also include a builtin.mk file that includes the necessary checks to decide whether using the built-in software or the pkgsrc software is appropriate.

The only requirements of a builtin.mk file for pkg are:

.if !defined(IS_BUILTIN.foo)
#
# IS_BUILTIN.foo is set to "yes" or "no" depending on whether "foo"
# genuinely exists in the system or not.
#
IS_BUILTIN.foo?=        no

# BUILTIN_PKG.foo should be set here if "foo" is built-in and its package
# version can be determined.
#
.  if !empty(IS_BUILTIN.foo:M[yY][eE][sS])
BUILTIN_PKG.foo?=       foo-1.0
.  endif
.endif  # IS_BUILTIN.foo

.if !defined(USE_BUILTIN.foo)
USE_BUILTIN.foo?=       ${IS_BUILTIN.foo}
.  if defined(BUILTIN_PKG.foo)
.    for _depend_ in ${BUILDLINK_API_DEPENDS.foo}
.      if !empty(USE_BUILTIN.foo:M[yY][eE][sS])
USE_BUILTIN.foo!=                                                       \
        ${PKG_ADMIN} pmatch '${_depend_}' ${BUILTIN_PKG.foo}            \
        && ${ECHO} "yes" || ${ECHO} "no"
.      endif
.    endfor
.  endif
.endif  # USE_BUILTIN.foo

CHECK_BUILTIN.foo?=     no
.if !empty(CHECK_BUILTIN.foo:M[nN][oO])
#
# Here we place code that depends on whether USE_BUILTIN.foo is set to
# "yes" or "no".
#
.endif  # CHECK_BUILTIN.foo

PREFER_PKGSRC=  yes
PREFER_NATIVE=  getopt skey tcp_wrappers

# ./bootstrap --prefer-pkgsrc yes

As you already know, the PLIST file holds a list of files and directories that belong to a package. The names used in it are relative to the installation prefix (${PREFIX}), which means that it cannot register files outside this directory (absolute path names are not allowed). Despite this restriction, some packages need to install files outside this location; e.g., under ${VARBASE} or ${PKG_SYSCONFDIR}. The only way to achieve this is to create such files during installation time by using installation scripts.

The generic installation scripts are shell scripts that can contain arbitrary code. The list of scripts to execute is taken from the INSTALL_FILE variable, which defaults to INSTALL. A similar variable exists for package removal (DEINSTALL_FILE, whose default is DEINSTALL). These scripts can run arbitrary commands, so they have the potential to create and manage files anywhere in the file system.

Using these general installation files is not recommended, but may be needed in some special cases. One reason for avoiding them is that the user has to trust the packager that there is no unwanted or simply erroneous code included in the installation script. Also, previously there were many similar scripts for the same functionality, and fixing a common error involved finding and changing all of them.

The pkginstall framework offers another, standardized way. It provides generic scripts to abstract the manipulation of such files and directories based on variables set in the package's Makefile. The rest of this section describes these variables.

Configuration files are special in the sense that they are installed in their own specific directory, PKG_SYSCONFDIR, and need special treatment during installation (most of which is automated by pkginstall). The main concept you must bear in mind is that files marked as configuration files are automatically copied to the right place (somewhere inside PKG_SYSCONFDIR) during installation if and only if they didn't exist before. Similarly, they will not be removed if they have local modifications. This ensures that administrators never lose any custom changes they may have made.

CONFIGURE_ARGS+= --sysconfdir=${PKG_SYSCONFDIR}

EGDIR=        ${PREFIX}/share/doc/mutt/samples
CONF_FILES=   ${EGDIR}/Muttrc ${PKG_SYSCONFDIR}/Muttrc

System startup scripts are special files because they must be installed in a place known by the underlying OS, usually outside the installation prefix. Therefore, the same rules described in Section 17.1, “Files and directories outside the installation prefix” apply, and the same solutions can be used. However, pkginstall provides a special mechanism to handle these files.

In order to provide system startup scripts, the package has to:

Once this is done, pkginstall will do the following steps for each script in an automated fashion:

If a package needs to create special users and/or groups during installation, it can do so by using the pkginstall framework.

Users can be created by adding entries to the PKG_USERS variable. Each entry has the following syntax:

user:group

Further specification of user details may be done by setting per-user variables. PKG_UID.user is the numeric UID for the user. PKG_GECOS.user is the user's description or comment. PKG_HOME.user is the user's home directory, and defaults to /nonexistent if not specified. PKG_SHELL.user is the user's shell, and defaults to /sbin/nologin if not specified.

Similarly, groups can be created by adding entries to the PKG_GROUPS variable, whose syntax is:

group

The numeric GID of the group may be set by defining PKG_GID.group.

If a package needs to create the users and groups at an earlier stage, then it can set USERGROUP_PHASE to either configure,build, or pre-install to indicate the phase before which the users and groups are created. In this case, the numeric UIDs and GIDs of the created users and groups are automatically hardcoded into the final installation scripts.

Packages that install system shells should register them in the shell database, /etc/shells, to make things easier to the administrator. This must be done from the installation scripts to keep binary packages working on any system. pkginstall provides an easy way to accomplish this task.

When a package provides a shell interpreter, it has to set the PKG_SHELL variable to its absolute file name. This will add some hooks to the installation scripts to handle it. Consider the following example, taken from shells/zsh:

PKG_SHELL=      ${PREFIX}/bin/zsh

Packages that install X11 fonts should update the database files that index the fonts within each fonts directory. This can easily be accomplished within the pkginstall framework.

When a package installs X11 fonts, it must list the directories in which fonts are installed in the FONTS_DIRS.type variables, where type can be one of “ttf”, “type1” or “x11”. This will add hooks to the installation scripts to run the appropriate commands to update the fonts database files within each of those directories. For convenience, if the directory path is relative, it is taken to be relative to the package's installation prefix. Consider the following example, taken from fonts/dbz-ttf:

FONTS_DIRS.ttf= ${PREFIX}/share/fonts/X11/TTF

Global default options are listed in PKG_DEFAULT_OPTIONS, which is a list of the options that should be built into every package if that option is supported. This variable should be set in mk.conf.

The following example shows how bsd.options.mk should be used by the hypothetical ``wibble'' package, either in the package Makefile, or in a file, e.g. options.mk, that is included by the main package Makefile.

PKG_OPTIONS_VAR=                PKG_OPTIONS.wibble
PKG_SUPPORTED_OPTIONS=          wibble-foo ldap
PKG_OPTIONS_OPTIONAL_GROUPS=    database
PKG_OPTIONS_GROUP.database=     mysql pgsql
PKG_SUGGESTED_OPTIONS=          wibble-foo
PKG_OPTIONS_LEGACY_VARS+=       WIBBLE_USE_OPENLDAP:ldap
PKG_OPTIONS_LEGACY_OPTS+=       foo:wibble-foo

.include "../../mk/bsd.prefs.mk"

# this package was previously named wibble2
.if defined(PKG_OPTIONS.wibble2)
PKG_LEGACY_OPTIONS+=            ${PKG_OPTIONS.wibble2}
PKG_OPTIONS_DEPRECATED_WARNINGS+= \
        "Deprecated variable PKG_OPTIONS.wibble2 used, use ${PKG_OPTIONS_VAR} instead."
.endif

.include "../../mk/bsd.options.mk"

# Package-specific option-handling

###
### FOO support
###
.if !empty(PKG_OPTIONS:Mwibble-foo)
CONFIGURE_ARGS+=    --enable-foo
.endif

###
### LDAP support
###
.if !empty(PKG_OPTIONS:Mldap)
.  include "../../databases/openldap-client/buildlink3.mk"
CONFIGURE_ARGS+=    --enable-ldap=${BUILDLINK_PREFIX.openldap-client}
.endif

###
### database support
###
.if !empty(PKG_OPTIONS:Mmysql)
.  include "../../mk/mysql.buildlink3.mk"
.endif
.if !empty(PKG_OPTIONS:Mpgsql)
.  include "../../mk/pgsql.buildlink3.mk"
.endif

The first section contains the information about which build options are supported by the package, and any default options settings if needed.

A package should never modify PKG_DEFAULT_OPTIONS or the variable named in PKG_OPTIONS_VAR. These are strictly user-settable. To suggest a default set of options, use PKG_SUGGESTED_OPTIONS.

PKG_OPTIONS_VAR must be defined before including bsd.options.mk. If none of PKG_SUPPORTED_OPTIONS, PKG_OPTIONS_OPTIONAL_GROUPS, and PKG_OPTIONS_REQUIRED_GROUPS are defined (as can happen with platform-specific options if none of them is supported on the current platform), PKG_OPTIONS is set to the empty list and the package is otherwise treated as not using the options framework.

After the inclusion of bsd.options.mk, the variable PKG_OPTIONS contains the list of selected build options, properly filtered to remove unsupported and duplicate options.

The remaining sections contain the logic that is specific to each option. The correct way to check for an option is to check whether it is listed in PKG_OPTIONS:

.if !empty(PKG_OPTIONS:Moption)

Options that enable similar features in different packages (like optional support for a library) should use a common name in all packages that support it (like the name of the library). If another package already has an option with the same meaning, use the same name.

Options that enable features specific to one package, where it's unlikely that another (unrelated) package has the same (or a similar) optional feature, should use a name prefixed with pkgname-.

If a group of related packages share an optional feature specific to that group, prefix it with the name of the “main” package (e. g. djbware-errno-hack).

For new options, add a line to mk/defaults/options.description. Lines have two fields, separated by tab. The first field is the option name, the second its description. The description should be a whole sentence (starting with an uppercase letter and ending with a period) that describes what enabling the option does. E. g. “Enable ispell support.” The file is sorted by option names.

When writing buildlink3.mk files, it is often necessary to list different dependencies based on the options with which the package was built. For querying these options, the file pkgsrc/mk/pkg-build-options.mk should be used. A typical example looks like this:

pkgbase := libpurple
.include "../../mk/pkg-build-options.mk"

.if !empty(PKG_BUILD_OPTIONS.libpurple:Mdbus)
...
.endif

Including pkg-build-options.mk here will set the variable PKG_BUILD_OPTIONS.libpurple to the build options of the libpurple package, which can then be queried like PKG_OPTIONS in the options.mk file. See the file pkg-build-options.mk for more details.

This chapter gives a detailed description on how a package is built. Building a package is separated into different phases (for example fetch, build, install), all of which are described in the following sections. Each phase is split into so-called stages, which take the name of the containing phase, prefixed by one of pre-, do- or post-. (Examples are pre-configure, post-build.) Most of the actual work is done in the do-* stages.

Never override the regular targets (like fetch), if you have to, override the do-* ones instead.

The basic steps for building a program are always the same. First the program's source (distfile) must be brought to the local system and then extracted. After any pkgsrc-specific patches to compile properly are applied, the software can be configured, then built (usually by compiling), and finally the generated binaries, etc. can be put into place on the system.

To get more details about what is happening at each step, you can set the PKG_VERBOSE variable, or the PATCH_DEBUG variable if you are just interested in more details about the patch step.

Before outlining the process performed by the NetBSD package system in the next section, here's a brief discussion on where programs are installed, and which variables influence this.

The automatic variable PREFIX indicates where all files of the final program shall be installed. It is usually set to LOCALBASE (/usr/pkg), or CROSSBASE for pkgs in the cross category. The value of PREFIX needs to be put into the various places in the program's source where paths to these files are encoded. See Section 13.3, “patches/*” and Section 21.3.1, “Shared libraries - libtool” for more details.

When choosing which of these variables to use, follow the following rules:

When building a package, various directories are used to store source files, temporary files, pkgsrc-internal files, and so on. These directories are explained here.

Some of the directory variables contain relative pathnames. There are two common base directories for these relative directories: PKGSRCDIR/PKGPATH is used for directories that are pkgsrc-specific. WRKSRC is used for directories inside the package itself.

The CREATE_WRKDIR_SYMLINK definition takes either the value yes or no and defaults to no. It indicates whether a symbolic link to the WRKDIR is to be created in the pkgsrc entry's directory. If users would like to have their pkgsrc trees behave in a read-only manner, then the value of CREATE_WRKDIR_SYMLINK should be set to no.

You can run a particular phase by typing make phase, where phase is the name of the phase. This will automatically run all phases that are required for this phase. The default phase is build, that is, when you run make without parameters in a package directory, the package will be built, but not installed.

The first step in building a package is to fetch the distribution files (distfiles) from the sites that are providing them. This is the task of the fetch phase.

DISTFILES=      ${DEFAULT_DISTFILES} additional-files.tar.gz

DISTFILES=      ${DISTNAME}${EXTRACT_SUFX}
DISTFILES+=     foo-file.tar.gz
SITES.foo-file.tar.gz= \
http://www.somewhere.com/somehow/ \
http://www.somewhereelse.com/mirror/somehow/

MASTER_SITES=   http://www.example.com/download.cgi?file=

DISTNAME=       foo-1.0.0
MASTER_SITES=   -http://www.example.com/archive/v1.0.0.tar.gz

MASTER_SITES=   ${MASTER_SITE_GNU:=subdirectory/name/}
MASTER_SITES=   ${MASTER_SITE_SOURCEFORGE:=project_name/}

${FETCH_CMD} ${FETCH_BEFORE_ARGS} ${site}${file} ${FETCH_AFTER_ARGS}

After the distfile(s) are fetched, their checksum is generated and compared with the checksums stored in the distinfo file. If the checksums don't match, the build is aborted. This is to ensure the same distfile is used for building, and that the distfile wasn't changed, e.g. by some malign force, deliberately changed distfiles on the master distribution site or network lossage.

When the distfiles are present on the local system, they need to be extracted, as they usually come in the form of some compressed archive format.

By default, all DISTFILES are extracted. If you only need some of them, you can set the EXTRACT_ONLY variable to the list of those files.

Extracting the files is usually done by a little program, mk/extract/extract, which already knows how to extract various archive formats, so most likely you will not need to change anything here. But if you need, the following variables may help you:

If the extract program doesn't serve your needs, you can also override the EXTRACT_CMD variable, which holds the command used for extracting the files. This command is executed in the ${WRKSRC} directory. During execution of this command, the shell variable extract_file holds the absolute pathname of the file that is going to be extracted.

And if that still does not suffice, you can override the do-extract target in the package Makefile.

After extraction, all the patches named by the PATCHFILES, those present in the patches subdirectory of the package as well as in $LOCALPATCHES/$PKGPATH (e.g. /usr/local/patches/graphics/png) are applied. Patchfiles ending in .Z or .gz are uncompressed before they are applied, files ending in .orig or .rej are ignored. Any special options to patch(1) can be handed in PATCH_DIST_ARGS. See Section 13.3, “patches/*” for more details.

By default patch(1) is given special args to make it fail if the patches apply with some lines of fuzz. Please fix (regen) the patches so that they apply cleanly. The rationale behind this is that patches that don't apply cleanly may end up being applied in the wrong place, and cause severe harm there.

This is covered in Chapter 20, Tools needed for building or running.

This phase creates wrapper programs for the compilers and linkers. The following variables can be used to tweak the wrappers.

Most pieces of software need information on the header files, system calls, and library routines which are available on the platform they run on. The process of determining this information is known as configuration, and is usually automated. In most cases, a script is supplied with the distfiles, and its invocation results in generation of header files, Makefiles, etc.

If the package contains a configure script, this can be invoked by setting HAS_CONFIGURE to “yes”. If the configure script is a GNU autoconf script, you should set GNU_CONFIGURE to “yes” instead. What happens in the configure phase is roughly:

.for d in ${CONFIGURE_DIRS}
        cd ${WRKSRC} \
        && cd ${d} \
        && env ${CONFIGURE_ENV} ${CONFIGURE_SCRIPT} ${CONFIGURE_ARGS}
.endfor

CONFIGURE_DIRS (default: “.”) is a list of pathnames relative to WRKSRC. In each of these directories, the configure script is run with the environment CONFIGURE_ENV and arguments CONFIGURE_ARGS. The variables CONFIGURE_ENV, CONFIGURE_SCRIPT (default: “./configure”) and CONFIGURE_ARGS may all be changed by the package.

If the program uses the Perl way of configuration (mainly Perl modules, but not only), i.e. a file called Makefile.PL, it should include ../../lang/perl5/module.mk. To set any parameter for Makefile.PL use the MAKE_PARAMS variable (e.g., MAKE_PARAMS+=foo=bar

If the program uses an Imakefile for configuration, the appropriate steps can be invoked by setting USE_IMAKE to “yes”. If you only need xmkmf, add it to USE_TOOLS. You can add variables to xmkmf's environment by adding them to the SCRIPTS_ENV variable.

If the program uses cmake for configuration, the appropriate steps can be invoked by setting USE_CMAKE to “yes”. You can add variables to cmake's environment by adding them to the CONFIGURE_ENV variable and arguments to cmake by adding them to the CMAKE_ARGS variable. The top directory argument is given by the CMAKE_ARG_PATH variable, that defaults to “.” (relative to CONFIGURE_DIRS)

If there is no configure step at all, set NO_CONFIGURE to “yes”.

For building a package, a rough equivalent of the following code is executed.

.for d in ${BUILD_DIRS}
        cd ${WRKSRC} \
        && cd ${d} \
        && env ${MAKE_ENV} \
            ${MAKE_PROGRAM} ${BUILD_MAKE_FLAGS} \
                -f ${MAKE_FILE} \
                ${BUILD_TARGET}
.endfor

BUILD_DIRS (default: “.”) is a list of pathnames relative to WRKSRC. In each of these directories, MAKE_PROGRAM is run with the environment MAKE_ENV and arguments BUILD_MAKE_FLAGS. The variables MAKE_ENV, BUILD_MAKE_FLAGS, MAKE_FILE and BUILD_TARGET may all be changed by the package.

The default value of MAKE_PROGRAM is “gmake” if USE_TOOLS contains “gmake”, “make” otherwise. The default value of MAKE_FILE is “Makefile”, and BUILD_TARGET defaults to “all”.

If there is no build step at all, set NO_BUILD to “yes”.

[TODO]

Once the build stage has completed, the final step is to install the software in public directories, so users can access the programs and files.

In the install phase, a rough equivalent of the following code is executed. Additionally, before and after this code, much magic is performed to do consistency checks, registering the package, and so on.

.for d in ${INSTALL_DIRS}
        cd ${WRKSRC} \
        && cd ${d} \
        && env ${MAKE_ENV} \
            ${MAKE_PROGRAM} ${INSTALL_MAKE_FLAGS} \
                -f ${MAKE_FILE} \
                ${INSTALL_TARGET}
.endfor

The variable's meanings are analogous to the ones in the build phase. INSTALL_DIRS defaults to BUILD_DIRS. INSTALL_TARGET is “install” by default, plus “install.man” if USE_IMAKE is defined and NO_INSTALL_MANPAGES is not defined.

In the install phase, the following variables are useful. They are all variations of the install(1) command that have the owner, group and permissions preset. INSTALL is the plain install command. The specialized variants, together with their intended use, are:

Some other variables are:

In the rare cases that a package shouldn't install anything, set NO_INSTALL to “yes”. This is mostly relevant for packages in the regress category.

Once the install stage has completed, a binary package of the installed files can be built. These binary packages can be used for quick installation without previous compilation, e.g. by the make bin-install or by using pkg_add.

By default, the binary packages are created in ${PACKAGES}/All and symlinks are created in ${PACKAGES}/category, one for each category in the CATEGORIES variable. PACKAGES defaults to pkgsrc/packages.

Once you're finished with a package, you can clean the work directory by running make clean. If you want to clean the work directories of all dependencies too, use make clean-depends.