diff options
author | grant <grant@pkgsrc.org> | 2003-06-23 07:41:44 +0000 |
---|---|---|
committer | grant <grant@pkgsrc.org> | 2003-06-23 07:41:44 +0000 |
commit | 168a82be23e7ea66ea4b700ba61019054dd1405e (patch) | |
tree | cec7c927cdab5855bc9b35af018f5c8cedec5552 /pkgsrc.txt | |
parent | 7152c2f5d8be5f7802e1a3fa4581a7ad31b85349 (diff) | |
download | pkgsrc-168a82be23e7ea66ea4b700ba61019054dd1405e.tar.gz |
deprecate Packages.txt.
now point users at documentation on the web and provide a copy of the
single file HTML and plain-text output.
Diffstat (limited to 'pkgsrc.txt')
-rw-r--r-- | pkgsrc.txt | 3234 |
1 files changed, 3234 insertions, 0 deletions
diff --git a/pkgsrc.txt b/pkgsrc.txt new file mode 100644 index 00000000000..b53a2830ce5 --- /dev/null +++ b/pkgsrc.txt @@ -0,0 +1,3234 @@ + +The NetBSD Packages Collection (pkgsrc) + +Alistair Crooks + + <agc@NetBSD.org> + +Hubert Feyrer + + <hubertf@NetBSD.org> + + $NetBSD: pkgsrc.txt,v 1.1 2003/06/23 07:41:44 grant Exp $ + + Copyright © 1994-2003 The NetBSD Foundation, Inc + + Abstract + + Information about using the NetBSD package system and building + packages. + _________________________________________________________________ + + Table of Contents + + 1. Introduction + + 1.1. Introduction + 1.2. Overview + 1.3. Terminology + 1.4. Typography + + I. pkgsrc user's guide + + 2. Using pkgsrc on systems other than NetBSD + + 2.1. Bootstrapping pkgsrc + 2.2. Platform specific notes + + 3. Using The NetBSD package system + + 3.1. Working with binary packages + 3.2. Building packages from source + + 4. Creating binary packages + + 4.1. Building a single binary package + 4.2. Settings for creation of binary packages + 4.3. Doing a bulk build of all packages + 4.4. Creating a multiple CD-ROM packages collection + + II. pkgsrc developer's guide + + 5. Package components - files, directories and contents + + 5.1. Makefile + 5.2. distinfo + 5.3. patches/* + 5.4. Other mandatory files + 5.5. Optional files + 5.6. work* + 5.7. files/* + 5.8. Portability of packages + + 6. PLIST issues + + 6.1. Miscellaneous + 6.2. PLIST_SRC + 6.3. PLIST_SUBST + 6.4. Perl5 modules + 6.5. User Interaction + + 7. Notes on fixes for packages + + 7.1. CPP defines + 7.2. Shared libraries - libtool + 7.3. Using libtool on GNU packages that already support + libtool + + 7.4. GNU Autoconf/Automake + 7.5. Package configuration files + 7.6. Feedback to the author + + 8. The build process + + 8.1. Program location + 8.2. Main targets + 8.3. Other helpful targets + + 9. buildlink2 methodology + + 9.1. Converting packages to use buildlink2 + 9.2. Writing buildlink2.mk files + + 10. Debugging + 11. FAQs & features of the package system + + 11.1. Packages using GNU autoconf + 11.2. Other distrib methods than .tar.gz + 11.3. Packages not creating their own subdirectory + 11.4. Custom configuration process + 11.5. Packages not building in their DISTNAME directory + 11.6. How to fetch all distfiles at once + 11.7. How to fetch files from behind a firewall + 11.8. If your patch contains an RCS ID + 11.9. How to pull in variables from /etc/mk.conf + 11.10. Is there a mailing list for pkg-related discussion? + 11.11. How do I tell make fetch to do passive FTP? + 11.12. Dependencies on other packages + 11.13. Conflicts with other packages + 11.14. Software which has a WWW Home Page + 11.15. How to handle modified distfiles with the 'old' name + + 11.16. What does "Don't know how to make + /usr/share/tmac/tmac.andoc" mean? + + 11.17. How to handle incrementing versions when fixing an + existing package + + 11.18. Could not find bsd.own.mk - what's wrong? + 11.19. Restricted packages + 11.20. Packages using (n)curses + 11.21. Automated security check + 11.22. What's the proper way to create an account from a + package? + + 11.23. How to handle compiler bugs + 11.24. Packages providing info files + 11.25. Packages whose distfiles aren't available for plain + downloading + + 11.26. Configuration files handling and placement + 11.27. Packages providing login shells + 11.28. Packages providing locale catalogues + 11.29. Using 'sudo' with pkgsrc + 11.30. Packages containing perl scripts + 11.31. Packages that cannot or should not be built + + 12. Submitting and Committing + + 12.1. Submitting your packages + 12.2. Committing: Importing a package into CVS + 12.3. Updating a Package to a Newer Version + 12.4. Moving a Package in pkgsrc + + 13. A simple example of a package: bison + + 13.1. files + 13.2. Steps for building, installing, packaging + + A. Build logs + + A.1. Building top + A.2. Packaging top + + B. Layout of the FTP server's package archive + +Chapter 1. Introduction + + Table of Contents + + 1.1. Introduction + 1.2. Overview + 1.3. Terminology + 1.4. Typography + +1.1. Introduction + + The NetBSD package system, pkgsrc, is a framework for building + third-party software on NetBSD and other UNIX-like systems. It is used + to enable such freely available software to be configured and built + easily on supported platforms. + + Once the software has been built, it is manipulated with the pkg_* + tools so that installation and de-installation, printing of an + inventory of all installed packages and retrieval of one-line comments + or more verbose descriptions are all simple. + + pkgsrc currently contains over 3500 packages, including: + * www/apache - The Apache web server + * www/mozilla - The Mozilla web browser + * x11/gnome - The GNOME Desktop Environment + * x11/kde3 - The K Desktop Environment + + ...just to name a few. + + pkgsrc has built-in support for handling varying dependencies, such as + pthreads and X11, and extended features such as IPv6 support on a + range of platforms. + + pkgsrc was originally developed on NetBSD, and now supports the + following platforms: + * Darwin (MacOS X) + * FreeBSD + * IRIX + * Linux + * NetBSD (of course) + * OpenBSD + * Solaris + +1.2. Overview + + This document is divided into two parts. The first, pkgsrc user's + guide, describes how one can use one of the packages in the Package + Collection, either by installing a precompiled binary package, or by + building one's own copy using the NetBSD package system. The second + part, pkgsrc developer's guide, explains how to prepare a package so + it can be easily built by other NetBSD users without knowing about the + package's building details. + +1.3. Terminology + + There has been a lot of talk about "ports", "packages", etc. so far. + Here is a description of all the terminology used within this + document. + * Package + A set of files and building instructions that describe what's + necessary to build a certain piece of software using the NetBSD + package system. Packages are traditionally stored under + /usr/pkgsrc. + * The NetBSD package system + This is the part of the NetBSD operating system handling building + (compiling), installing, and removing of packages. + * Distfile + This term describes the file or files that are provided by the + author of the piece of freely available software to distribute his + work. All the changes necessary to build on NetBSD are reflected + in the corresponding package. Usually the distfile is in the form + of a compressed tar-archive, but other types are possible, too. + Distfiles are stored below /usr/pkgsrc/distfiles. + * Port + This is the term used by FreeBSD people for what we call a + package. In NetBSD terminology, "port" refers to a different + architecture. + * Precompiled (binary) package + A set of binaries built by the NetBSD package system from a + distfile using the NetBSD package system and stuffed together in a + single .tgz file so it can be installed on machines of the same + machine architecture without the need to recompile. Packages are + generated in /usr/pkgsrc/packages by the NetBSD package system; + there is also an archive on ftp.NetBSD.org. + Sometimes, this is referred to by the term "package" too, + especially in the context of precompiled packages. + * Program + The piece of software to be installed which will be constructed + from all the files in the Distfile by the actions defined in the + corresponding package. + +1.4. Typography + + When giving examples for commands, shell prompts are used to show if + the command should/can be issued as root, or if "normal" user + privileges are sufficient. We use a "#" for root's shell prompt, and a + "%" for users' shell prompt, assuming they use the C-shell or tcsh. + +pkgsrc user's guide + + Table of Contents + + 2. Using pkgsrc on systems other than NetBSD + + 2.1. Bootstrapping pkgsrc + 2.2. Platform specific notes + + 3. Using The NetBSD package system + + 3.1. Working with binary packages + 3.2. Building packages from source + + 4. Creating binary packages + + 4.1. Building a single binary package + 4.2. Settings for creation of binary packages + 4.3. Doing a bulk build of all packages + 4.4. Creating a multiple CD-ROM packages collection + +Chapter 2. Using pkgsrc on systems other than NetBSD + + Table of Contents + + 2.1. Bootstrapping pkgsrc + 2.2. Platform specific notes + +2.1. Bootstrapping pkgsrc + + For Operating Systems other than NetBSD, we provide a bootstrap kit to + build the required tools to use pkgsrc on your platform. As well as + native NetBSD support, pkgsrc and the bootstrap kit have support for + the following operating systems: + * Darwin (MacOS X) + * FreeBSD + * IRIX + * Linux + * OpenBSD + * Solaris + + Support for other platforms is under development. + + Installing the bootstrap kit should be as simple as: +# cvs checkout othersrc/bootstrap-pkgsrc +# cd othersrc/bootstrap-pkgsrc +# ./bootstrap + + This will use the defaults of /usr/pkg for the prefix and /var/db/pkg + for the package database directory. However, these can also be set + using command-line parameters. + + Binary packages for the pkgsrc tools and an initial set of packages is + available for supported platforms. An up-to-date list of these can be + found on www.pkgsrc.org. + +2.2. Platform specific notes + + Here are some platform-specific notes you should be aware of. + +2.2.1. Darwin (Mac OS X) + + Darwin 5.x and 6.x are supported. There are two methods of using + pkgsrc on Mac OS X, by using a disk image, or a UFS partition. + + If you already have a UFS partition, or have a spare partition that + you can format as UFS, it is recommended to use that instead of the + disk image. It'll be somewhat faster and will mount automatically at + boot time, where you must manually mount a disk image. + +Note + + You cannot use a HFS+ file system for pkgsrc, because pkgsrc currently + requires the filesystem to be case-sensitive, and HFS+ is not. + +2.2.1.1. Using a disk image + + Create the disk image: +# cd bootstrap-pkgsrc +# ./ufsdiskimage create ~/Documents/NetBSD 512 # megabytes - season to taste +# ./ufsdiskimage mount ~/Documents/NetBSD +# sudo chown `id -u`:`id -g` /Volumes/NetBSD + + That's it! + +2.2.1.2. Using a UFS partition + + By default, /usr will be on your root file system, normally HFS+. It + is possible to use the default prefix of /usr/pkg by symlinking + /usr/pkg to a directory on a UFS file system. Obviously, another + symlink is required if you want to place the package database + directory outside the prefix. e.g. + # ./bootstrap --pkgdbdir=/usr/pkg/pkgdb --pkgsrc=/Volumes/ufs/pkgsrc + + If you created your partitions at the time of installing Mac OS X and + formatted the target partition as UFS, it should automatically mount + on /Volumes/<volume name> when the machine boots. If you are + (re)formatting a partition as UFS, you need to ensure that the + partition map correctly reflects "Apple_UFS" and not "Apple_HFS". + + The problem is that none of the disk tools will let you touch a disk + that is booted from. You can unmount the partition, but even if you + newfs it, the partition type will be incorrect and the automounter + won't mount it. It can be mounted manually, but it won't appear in + Finder. + + You'll need to boot off of the OS X Installation (User) CD. When the + Installation program starts, go up to the menu and select Disk + Utility. Now, you will be able to select the partition you want to be + UFS, and Format it Apple UFS. Quit the Disk Utility, quit the + installer which will reboot your machine. The new UFS file system will + appear in Finder. + + Be aware that the permissions on the new file system will be writable + by root only. + + This note is as of 10.2 (Jaguar) and applies to earlier versions. + Hopefully Apple will fix Disk Utility in 10.3 (Panther). + +2.2.2. FreeBSD + + FreeBSD 4.7 and 5.0 have been tested and are supported, other versions + may work. + + Care should be taken so that the tools that this kit installs do not + conflict with the FreeBSD userland tools. There are several steps: + 1. FreeBSD stores its ports pkg database in /var/db/pkg. It is + therefore recommended that you choose a different location (e.g. + /usr/pkgdb) by using the --pkgdbdir option to the bootstrap + script. + 2. If you do not intend to use the FreeBSD ports tools, it's probably + a good idea to move them out of the way to avoid confusion, e.g. + +# cd /usr/sbin +# mv pkg_add pkg_add.orig +# mv pkg_create pkg_create.orig +# mv pkg_delete pkg_delete.orig +# mv pkg_info pkg_info.orig + + 3. An example /etc/mk.conf file will be placed in + /etc/mk.conf.example file when you use the bootstrap script. + +2.2.3. IRIX + + IRIX 6.5 is tested and supported, other versions may work. + + You will need a working C compiler, either gcc or SGI's MIPS and + MIPSpro compiler (cc/c89). Please set the CC environment variable + according to your preference. + + Please make sure that you have no conflicting CFLAGS in your + environment or the /etc/mk.conf. Particularly, make sure that you do + not try to link n32 object files with lib64 or vice versa. Check your + /etc/compiler.defaults! + +2.2.4. OpenBSD + + OpenBSD 3.0 and 3.2 are tested and supported. + + Care should be taken so that the tools that this kit installs do not + conflict with the OpenBSD userland tools. There are several steps: + 1. OpenBSD stores its ports pkg database in /var/db/pkg. It is + therefore recommended that you choose a different location (e.g. + /usr/pkgdb) by using the --pkgdbdir option to the bootstrap + script. + 2. If you do not intend to use the OpenBSD ports tools, it's probably + a good idea to move them out of the way to avoid confusion, e.g. + +# cd /usr/sbin +# mv pkg_add pkg_add.orig +# mv pkg_create pkg_create.orig +# mv pkg_delete pkg_delete.orig +# mv pkg_info pkg_info.orig + + 3. An example /etc/mk.conf file will be placed in + /etc/mk.conf.example file when you use the bootstrap script. + OpenBSD's make program uses /etc/mk.conf as well. You can work + around this by enclosing all the pkgsrc specific parts of the file + with: + +.ifdef BSD_PKG_MK +# pkgsrc stuff, e.g. insert bsd.pkg.defaults.mk or similar here +.else +# OpenBSD stuff +.endif + +2.2.5. Solaris + + Solaris 2.6 through 9 are supported. You will need a working C + compiler. Both gcc 2.95.3 and Sun WorkShop 5 have been tested. + + The following packages are required on Solaris 8 for the bootstrap + process and to build packages. + * SUNWsprot + * SUNWarc + * SUNWbtool + * SUNWtoo + * SUNWlibm + + Please note the use of GNU binutils on Solaris is not supported. + +2.2.5.1. If you are using gcc + + It makes life much simpler if you only use the same gcc consistently + for building all packages. + + It is recommended that an external gcc be used only for bootstrapping, + then either build gcc from lang/gcc or install a binary gcc package, + then remove gcc used during bootstrapping. + +2.2.5.2. If you are using Sun WorkShop + + You will need at least the following packages installed (from WorkShop + 5.0) + * SPROcc - Sun WorkShop Compiler C 5.0 + * SPROcpl - Sun WorkShop Compiler C++ 5.0 + * SPROild - Sun WorkShop Incremental Linker + * SPROlang - Sun WorkShop Compilers common components + + You should set CC, CXX and optionally, CPP in /etc/mk.conf, eg. +CC= cc +CXX= CC +CPP= /usr/ccs/lib/cpp + + You may also want to build 64-bit binaries, eg. + CFLAGS= -xtarget=ultra -xarch=v9 + + Whichever compiler you use, please ensure the compiler tools and your + $prefix are in your PATH. This includes /usr/ccs/{bin,lib} and eg. + /usr/pkg/{bin,sbin}. + +Chapter 3. Using The NetBSD package system + + Table of Contents + + 3.1. Working with binary packages + 3.2. Building packages from source + +3.1. Working with binary packages + + This section describes how to find, retrieve and install a precompiled + binary package that someone else already prepared for your type of + machine. + +3.1.1. How to get binary packages + + Precompiled packages are stored on ftp.NetBSD.org and its mirrors in + the directory /pub/NetBSD/packages for anonymous FTP access. Please + pick the right subdirectory there as indicated by uname -p. In that + directory, there is a subdirectory for each category plus a + subdirectory All which includes the actual binaries in .tgz files. The + category subdirectories use symbolic links to those files (this is the + same directory layout as in /usr/pkgsrc/packages). + + This same directory layout applies for CDROM distributions, only that + the directory may be rooted somewhere else, probably somewhere below + /cdrom. Please consult your CDROMs documentation for the exact + location. + +3.1.2. Installing binary packages + + If you have the files on a CDROM or downloaded them to your hard disk, + you can install them with the following command (be sure to su to root + first): + # pkg_add /path/to/package.tgz + + If you have FTP access and you don't want to download the packages via + FTP prior to installation, you can do this automatically by giving + pkg_add an FTP URL: +# pkg_add ftp://ftp.NetBSD.org/pub/NetBSD/packages/<OS Ver>/<arch>/All/package. +tgz + + If there is any doubt, the uname utility can be used to determine the + <OS Ver>, and <arch> by running uname -rp. + + Also note that any prerequisite packages needed to run the package in + question will be installed, too, assuming they are present where you + install from. + + After you've installed packages, be sure to have /usr/pkg/bin in your + PATH so you can actually start the just installed program. + +3.1.3. A word of warning + + Please pay very careful attention to the warnings expressed in that + manual page about the inherent dangers of installing binary packages + which you did not create yourself, and the security holes that can be + introduced onto your system by indiscriminate adding of such files. + +3.2. Building packages from source + + This assumes that the package is already part of the NetBSD package + system. If it is not, see Part II. + +3.2.1. Requirements + + To build packages from source on a NetBSD system the "comp" and the + "text" distribution sets must be installed. If you want to build X11 + related packages the "xbase" and "xcomp" distribution sets are + required, too. + +3.2.1.1. Where to get pkgsrc + + There are three ways to get pkgsrc. Either as a tar file, via SUP, or + via CVS. All three ways are described here. + + To get the package source going, you need to get the pkgsrc.tar.gz + file from ftp.NetBSD.org and unpack it into /usr/pkgsrc. + + As an alternative, you can get pkgsrc via the Software Update + Protocol, SUP. To do so, make sure your supfile has a line + release=pkgsrc + + in it, see the examples in /usr/share/examples/supfiles, and that the + /usr/pkgsrc directory exists. Then, simply run sup -v + /path/to/your/supfile. + + To get pkgsrc via CVS, make sure you have cvs installed. If not + present on your system, it can be found as precompiled binary on + ftp.NetBSD.org. To do an initial (full) checkout of pkgsrc, do the + following steps: +% setenv CVSROOT anoncvs@anoncvs.NetBSD.org:/cvsroot +% setenv CVS_RSH ssh +% cd /usr +% cvs checkout -P pkgsrc + + This will create the pkgsrc directory in your /usr, and all the + package source will be stored under /usr/pkgsrc. To update pkgsrc + after the initial checkout, make sure you have CVS_RSH set as above, + then do: +% cd /usr/pkgsrc +% cvs -q update -dP + + Please also note that it is possible to have multiple copies of the + pkgsrc hierarchy in use at any one time - all work is done relatively + within the pkgsrc tree. + +3.2.1.2. Fetching distfiles + + There is one gotcha: The distribution file (i.e. the unmodified + source) must exist on your system for the packages system to be able + to build it. If it does not, then ftp is used to fetch the + distribution files automatically. + + You can overwrite some of the major distribution sites to fit to sites + that are close to your own. Have a look at + pkgsrc/mk/bsd.pkg.defaults.mk to find some examples - in particular, + look for the MASTER_SORT, MASTER_SORT_REGEX and INET_COUNTRY + definitions. This may save some of your bandwidth and time. + + You can change these settings either in your shell's environment, or, + if you want to keep the settings, by editing the /etc/mk.conf file, + and adding the definitions there. + + If you don't have a permanent Internet connection and you want to know + which files to download, make fetch-list will tell you what you'll + need. Put these distfiles into /usr/pkgsrc/distfiles. + +3.2.1.3. How to build and install + + Once the distfile(s) have been fetched, building a package is as + simple as changing into the package directory and running make: +% cd editors/vim +% make + + Installing the package on your system requires you to be root. + However, pkgsrc has a just-in-time su feature, which allows you to + only become root for the actual installation step. e.g. +% make install +===> Installing for top-3.5beta5 +===> Becoming root@mofo to install top-3.5beta5. +/usr/bin/su Password: <password> +[...installation continues...] + + Taking the top system utility as an example, we can install it on our + system by building as shown in Appendix A, Build logs. + + The program is installed under the default root of the packages tree - + /usr/pkg. Should this not conform to your tastes, simply set the + LOCALBASE variable in your environment, and it will use that value as + the root of your packages tree. So, to use /usr/local, set + LOCALBASE=/usr/local in your environment. Please note that you should + use a root which is dedicated to packages and not shared with other + programs (ie, do not try and use LOCALBASE=/usr). Also, you should not + try to add any of your own files or directories (such as src/, obj/, + or pkgsrc/) below the LOCALBASE tree. This is to prevent possible + conflicts between programs and other files installed by the package + system and whatever else may have been installed there. + + There is, of course, one exception to this - X11 packages are + traditionally installed in the X11 tree. The definition used to + identify the root of the X11 tree is the X11BASE definition. + + It is possible to install X11 packages in the LOCALBASE tree, for + which you must install the pkgtools/xpkgwedge package - see + Section 8.1, "Program location" for further details. + + Some packages look in /etc/mk.conf to alter some configuration options + at build time. Have a look at pkgsrc/mk/bsd.pkg.defaults.mk to get an + overview of what will be set there by default. Environment variables + such as LOCALBASE, and X11BASE can be set in /etc/mk.conf to save + having to remember to set them each time you want to use pkgsrc. + + Occasionally, people want to "look under the covers" to see what is + going on when a package is building or being installed. This may be + for debugging purposes, or out of simple curiosity. A number of + utility values have been added to help with this. + 1. If you invoke the make(1) command with PKG_DEBUG_LEVEL=2, then a + huge amount of information will be displayed. For example, + +% make patch PKG_DEBUG_LEVEL=2 + + will show all the commands that are invoked, up to and including + the "patch" stage. + 2. If you want to know the value of a certain make(1) definition, + then the VARNAME definition should be used, in conjunction with + the show-var target. e.g. + +% make show-var VARNAME=DISTFILES + + will show the expansion of the make(1) variable DISTFILES. + + If you want to de-install and re-install a binary package that you've + created (see next section), that you put into pkgsrc/packages manually + or that's located on a remote FTP server, you can use the the + "bin-install" target. This target will install a binary package - if + available - via pkg_add, else do a make package. The list of remote + FTP sites searched is kept in the variable BINPKG_SITE, which defaults + to ftp.NetBSD.org. Any flags that should be added to pkg_add(8) can be + put into BIN_INSTALL_FLAGS. See pkgsrc/mk/bsd.pkg.defaults.mk for more + details. + + A final word of warning: If you setup a system that has a non-standard + setting for LOCALBASE (or X11BASE, for that matter), be sure to set + that before any packages are installed, as you can not use several + directories for the same purpose. Doing so will result in pkgsrc not + being able to properly detect your installed packages, and fail + miserably. Note also that precompiled binary packages are usually + built with the default LOCALBASE of /usr/pkg, and that you should not + install any if you use a non-standard LOCALBASE. + +Chapter 4. Creating binary packages + + Table of Contents + + 4.1. Building a single binary package + 4.2. Settings for creation of binary packages + 4.3. Doing a bulk build of all packages + 4.4. Creating a multiple CD-ROM packages collection + +4.1. Building a single binary package + + Once you have built and installed a package, you can create a binary + package which can be installed on another system with pkg_add(1). This + saves having to build the same package on a group of hosts and wasting + CPU time. It also provides a simple means for others to install your + package, should you distribute it. + + Create a binary package: +# cd sysutils/top +# make package + + This will build and install your package (if not already done), and + then build a binary package from what was installed. You can then use + the pkg_* tools to manipulate it. Binary packages are created by + default in /usr/pkgsrc/packages, in the form of a gzip or bzip2 tar + file. See Section A.2, "Packaging top" for a continuation of the above + sysutils/top example. + + See Chapter 12, Submitting and Committing for information on how to + submit such a binary package. + +4.2. Settings for creation of binary packages + + See Section 8.3, "Other helpful targets". + +4.3. Doing a bulk build of all packages + + If you want to get a full set of precompiled binary packages, this + section describes how to get them. Beware that the bulk build will + remove all currently installed packages from your your system! Having + a FTP server configured either on the machine doing the bulk builds or + on a nearby NFS server can help to make the packages available to + everyone. See ftpd(8) for more information. If you use a remote NFS + server's storage, be sure to not actually compile on NFS storage, as + this slows things down a lot. + +4.3.1. Configuration + +4.3.1.1. /etc/mk.conf + + You may want to set things in /etc/mk.conf. Look at + pkgsrc/mk/bsd.pkg.defaults.mk for details of the default settings. You + will want to ensure that ACCEPTABLE_LICENSES meet your local policy. + As used in this example, _ACCEPTABLE=yes accepts all licenses. +PACKAGES?= ${_PKGSRCDIR}/packages/${MACHINE_ARCH} +WRKOBJDIR?= /usr/tmp/pkgsrc # build here instead of in pkgsrc +BSDSRCDIR= /usr/src +BSDXSRCDIR= /usr/xsrc # for x11/xservers +OBJHOSTNAME?= yes # use work.`hostname` +FAILOVER_FETCH= yes # insist on the correct checksum +PKG_DEVELOPER?= yes +_ACCEPTABLE= yes + + If you wish to use xpkgwedge for the entire build, then add: + BULK_PREREQ+= pkgtools/xpkgwedge + + Other packages which must be installed during the bulk build to modify + the build behaviour may be added to the BULK_PREREQ variable. Note + that currently the only package for which BULK_PREREQ makes sense is + xpkgwedge. + +4.3.1.2. build.conf + + In pkgsrc/mk/bulk, copy "build.conf-example" to "build.conf" and edit + it, following the comments in that file. This is the config file that + determines where log files are generated after the build, where to + mail the build report, where your pkgsrc is located and which user to + su(8) to do a cvs update. + +4.3.1.3. pre-build.local + + It is possible to configure the bulk build to perform certain site + specific tasks at the end of the pre-build stage. If the file + pre-build.local exists in /usr/pkgsrc/mk/bulk it will be executed (as + a sh(1) script) at the end of the usual pre-build stage. An example + use of pre-build.local is to have the line: +# echo "I do not have enough disk space to build this pig." \ + > pkgsrc/games/crafty-book-enormous/$BROKENF + + to prevent the system from trying to build a particular package which + requires nearly 3 Gb of disk space. + +4.3.2. Other environmental considerations + + As /usr/pkg will be completely deleted at the start of bulk builds, + make sure your login shell is placed somewhere else. Either drop it + into /usr/local/bin (and adjust your login shell in the password + file), or (re-)install it via pkg_add from /etc/rc.local, so you can + login after a reboot (remember that your current process won't die if + the package is removed, you just can't start any new instances of the + shell any more). Also, if you use NetBSD earlier than 1.5, or you + still want to use the pkgsrc version of ssh for some reason, be sure + to install ssh before starting it from rc.local: +( cd /usr/pkgsrc/security/ssh ; make bulk-install ) +if [ -f /usr/pkg/etc/rc.d/sshd ]; then + /usr/pkg/etc/rc.d/sshd +fi + + Not doing so will result in you being not able to log in via ssh after + the bulk build is finished or if the machine gets rebooted or crashes. + You have been warned! :) + +4.3.3. Operation + + Make sure you don't need any of the packages still installed. + +Warning + + During the bulk build, all packages will be removed! + + Be sure to remove all other things that might interfere with builds, + like some libs installed in /usr/local, etc. then become root and + type: +# cd /usr/pkgsrc +# sh mk/bulk/build + + If for some reason your last build didn't complete (power failure, + system panic, ...), you can continue it by running: + # sh mk/bulk/build restart + + At the end of the bulk run, you will get a summary via mail, and find + build logs in the directory specified by FTP in the build.conf file. + +4.3.4. What it does + + The bulk builds consist of three steps: + 1. pre-build + The script updates your pkgsrc via (anon)cvs, then cleans out any + broken distfiles, and removes all packages installed. + 2. the bulk build + This is basically "make bulk-package" with an optimised order in + which packages will be built. Packages that don't require other + packages will be built first, and packages with many depends will + be built later. + 3. post-build + Generates a report that's placed in the directory specified in the + build.conf file named broken.html, a short version of that report + will also be mailed to the build's admin. + + During the build, a list of broken packages will be compiled in + /usr/pkgsrc/.broken (or .../.broken.${MACHINE} if OBJMACHINE is set), + individual build logs of broken builds can be found in the package's + directory. These files are used by the bulk-targets to mark broken + builds to not waste time trying to rebuild them, and they can be used + to debug these broken package builds later. + +4.3.5. Disk space requirements + + Currently, roughly the following requirements are valid for 1.5/i386: + * 1500MB - distfiles (NFS ok) + * 1000MB - full set of all binaries (NFS ok) + * 1500MB - temp space for compiling (local disk recommended) + + For 1.5/alpha: + * 1300MB - full set of all binaries (NFS ok) + + Note that all pkgs will be de-installed as soon as they are turned + into a binary package, and that work-sources are removed, so there is + no huge demand to disk space. Afterwards, if the package is needed + again, it will be installed via "pkg_add" instead of building again, + so there are no cycles wasted by recompiling. + +4.3.6. Setting up a sandbox for chroot'ed builds + + If you don't want all the pkgs nuked from a machine (rendering it + useless for anything but pkg compiling), there is the possibility of + doing the pkg bulk build inside a chroot environment. + + The first step to do so is setting up a chroot sandbox, e.g. + /usr/sandbox. After extracting all the sets from a NetBSD installation + or doing a make distribution DESTDIR=/usr/sandbox in /usr/src/etc, be + sure the following items are present and properly configured: + * kernel + +# cp /netbsd /usr/sandbox + + * /dev/* + +# cd /usr/sandbox/dev ; sh MAKEDEV all + + * /etc/resolv.conf (for security/smtpd and mail): + +# cp /etc/resolv.conf /usr/sandbox/etc + + * working(!) mail config (hostname, sendmail.cf): + +# cp /etc/mail/sendmail.cf /usr/sandbox/etc/mail + + * /etc/localtime (for security/smtpd): + +# ln -sf /usr/share/zoneinfo/GMT /usr/sandbox/etc/localtime + + * /usr/src (system sources, for sysutils/aperture, net/ppp-mppe): + +# ln -s ../disk1/cvs . +# ln -s cvs/src-1.6 src +# ln -s cvs/pkgsrc . + + * create /var/db/pkg (not part of default install): + +# mkdir /usr/sandbox/var/db/pkg + + * create /usr/pkg (not part of default install): + +# mkdir /usr/sandbox/usr/pkg + + * checkout pkgsrc into /usr/sandbox/usr/pkgsrc: + +# cd /usr/sandbox/usr +# cvs -d anoncvs@anoncvs.NetBSD.org:/cvsroot checkout -d -P pkgsrc + + * edit /etc/mk.conf, see Section 4.3.1.1, "/etc/mk.conf". + * adjust mk/bulk/build.conf to suit your needs. + +Note + + Don't forget to install X. + + If you are a developer and want to upload the resulting binary + packages to ftp.NetBSD.org, be sure you are using the default X + version for your architecture and release (that is XFree86 3.3.6 for + 1.5.x, and XFree86 4.2.1 for NetBSD 1.6.1 on cats, i386 and macppc + ports, 3.3.6 on all other ports). + + The next thing you need is a fresh checkout of pkgsrc (e.g. from + anoncvs). Do not mount/link this to the copy of your pkgsrc tree you + do development in, as this will likely cause problems! Adjust + .../pkgsrc/packages and .../pkgsrc/distfiles to point to some places + outside the sandbox if you want to make the files public. + + When the chroot sandbox is setup, you can start the build with the + following steps: +# cd /usr/sandbox/usr/pkgsrc +# sh mk/bulk/do-sandbox-build + + This will just jump inside the sandbox and start building. At the end + of the build, mail will be sent with the results of the build. Created + binary pkgs will be in /usr/sandbox/usr/pkgsrc/packages (wherever that + points/mounts to/from). + +4.4. Creating a multiple CD-ROM packages collection + + After your bulk pkgsrc build has completed, you may wish to create a + CD-ROM set of the resulting binary packages to assist in installing + packages on other machines. The pkgtools/cdpack package provides a + simple tool for creating the ISO 9660 images. cdpack arranges the + packages on the CD-ROMs in a way that keeps all the dependencies for + given package on the same CD as that package. + +4.4.1. Example of cdpack + + Complete documentation for cdpack is found in cdpack(1). The following + short example assumes that the binary packages are left in + /usr/pkgsrc/packages/All and that sufficient disk space exists in /u2 + to hold the ISO 9660 images. +# mkdir /u2/images +# pkg_add /usr/pkgsrc/packages/All/cdpack +# cdpack /usr/pkgsrc/packages/All /u2/images + + If you wish to include a common set of files (COPYRIGHT, README, etc.) + on each CD in the collection, then you need to create a directory + which contains these files. e.g. +# mkdir /tmp/common +# echo "This is a README" > /tmp/common/README +# echo "Another file" > /tmp/common/COPYING +# mkdir /tmp/common/bin +# echo "#!/bin/sh" > /tmp/common/bin/myscript +# echo "echo Hello world" >> /tmp/common/bin/myscript +# chmod 755 /tmp/common/bin/myscript + + Now create the images: + # cdpack -x /tmp/common /usr/pkgsrc/packages/All /u2/images + + Each image will contain README, COPYING, and bin/myscript in their + root directories. + +pkgsrc developer's guide + + Table of Contents + + 5. Package components - files, directories and contents + + 5.1. Makefile + 5.2. distinfo + 5.3. patches/* + 5.4. Other mandatory files + 5.5. Optional files + 5.6. work* + 5.7. files/* + 5.8. Portability of packages + + 6. PLIST issues + + 6.1. Miscellaneous + 6.2. PLIST_SRC + 6.3. PLIST_SUBST + 6.4. Perl5 modules + 6.5. User Interaction + + 7. Notes on fixes for packages + + 7.1. CPP defines + 7.2. Shared libraries - libtool + 7.3. Using libtool on GNU packages that already support libtool + 7.4. GNU Autoconf/Automake + 7.5. Package configuration files + 7.6. Feedback to the author + + 8. The build process + + 8.1. Program location + 8.2. Main targets + 8.3. Other helpful targets + + 9. buildlink2 methodology + + 9.1. Converting packages to use buildlink2 + 9.2. Writing buildlink2.mk files + + 10. Debugging + 11. FAQs & features of the package system + + 11.1. Packages using GNU autoconf + 11.2. Other distrib methods than .tar.gz + 11.3. Packages not creating their own subdirectory + 11.4. Custom configuration process + 11.5. Packages not building in their DISTNAME directory + 11.6. How to fetch all distfiles at once + 11.7. How to fetch files from behind a firewall + 11.8. If your patch contains an RCS ID + 11.9. How to pull in variables from /etc/mk.conf + 11.10. Is there a mailing list for pkg-related discussion? + 11.11. How do I tell make fetch to do passive FTP? + 11.12. Dependencies on other packages + 11.13. Conflicts with other packages + 11.14. Software which has a WWW Home Page + 11.15. How to handle modified distfiles with the 'old' name + 11.16. What does "Don't know how to make + /usr/share/tmac/tmac.andoc" mean? + + 11.17. How to handle incrementing versions when fixing an + existing package + + 11.18. Could not find bsd.own.mk - what's wrong? + 11.19. Restricted packages + 11.20. Packages using (n)curses + 11.21. Automated security check + 11.22. What's the proper way to create an account from a package? + + 11.23. How to handle compiler bugs + 11.24. Packages providing info files + 11.25. Packages whose distfiles aren't available for plain + downloading + + 11.26. Configuration files handling and placement + 11.27. Packages providing login shells + 11.28. Packages providing locale catalogues + 11.29. Using 'sudo' with pkgsrc + 11.30. Packages containing perl scripts + 11.31. Packages that cannot or should not be built + + 12. Submitting and Committing + + 12.1. Submitting your packages + 12.2. Committing: Importing a package into CVS + 12.3. Updating a Package to a Newer Version + 12.4. Moving a Package in pkgsrc + + 13. A simple example of a package: bison + + 13.1. files + 13.2. Steps for building, installing, packaging + +Chapter 5. Package components - files, directories and contents + + Table of Contents + + 5.1. Makefile + 5.2. distinfo + 5.3. patches/* + 5.4. Other mandatory files + 5.5. Optional files + 5.6. work* + 5.7. files/* + 5.8. Portability of packages + + Whenever you're preparing a package, there are a number of files + involved which are described in the following sections. + +5.1. Makefile + + Building, installation and creation of a binary package are all + controlled by the package's Makefile. + + There is a Makefile for each package. This file includes the standard + bsd.pkg.mk file (referenced as ../../mk/bsd.pkg.mk), which sets all + the definitions and actions necessary for the package to compile and + install itself. The mandatory variables are the DISTNAME which + specifies the base name of the distribution file to be downloaded from + the site on the Internet, MASTER_SITES which specifies that site, + CATEGORIES which denotes the categories into which the package falls, + PKGNAME which is the name of the package, the MAINTAINER name, and the + COMMENT variable, which should contain a one-line description of the + package (the package name should not appear, it will be added + automatically). The maintainer variable is there so that anyone who + quibbles with the (always completely correct) decisions taken by the + guy who maintains the port can complain vigorously. + + The MASTER_SITES may be set to one of the predefined sites: +${MASTER_SITE_XCONTRIB} +${MASTER_SITE_GNU} +${MASTER_SITE_PERL_CPAN} +${MASTER_SITE_TEX_CTAN} +${MASTER_SITE_SUNSITE} +${MASTER_SITE_GNOME} +${MASTER_SITE_SOURCEFORGE} + + If one of these predefined sites is chosen, you may require the + ability to specify a subdirectory of that site. Since these macros may + expand to more than one actual site, you must use the following + construct to specify a subdirectory: + ${MASTER_SITE_GNU:=subdirectory/name/} + + Note the trailing slash after the subdirectory name. + +Note + + MASTER_SITE_SUBDIR has been deprecated and should no longer be used. + + If the package has multiple DISTFILES or multiple PATCHFILES from + different sites, set SITES_foo to a list of URI's where file "foo" may + be found. "foo" includes the suffix, e.g. +DISTFILES= ${DISTNAME}${EXTRACT_SUFX} +DISTFILES+= foo-file.tar.gz +SITES_foo-file.tar.gz=http://www.somewhere.com/somehow/ \ + http://www.somewhereelse.com/mirror/somehow/ + + Note that the normal default setting of DISTFILES must be made + explicit if you want to add to it (rather than replace it), as you + usually would. + + Currently the following values are available for CATEGORIES. If more + than one is used, they need to be separated by spaces: +archivers audio benchmarks biology cad +chat comms converters cross databases +devel editors emulators finance fonts +games graphics ham japanese lang +mail math mbone misc net +news parallel print security shells +sysutils textproc time wm www +x11 + + Please pay attention to the following gotchas: + * Add MANCOMPRESSED if manpages are installed in compressed form by + the package; see comment in bsd.pkg.mk. + * Replace /usr/local with "${PREFIX}" in all files (see patches, + below). + * If the package installs any info files, see Section 11.24, + "Packages providing info files". + * Adjust MAINTAINER to be either yourself, if you plan to maintain + the package for future updates, or set it to the default + maintainer <tech-pkg@NetBSD.org>. + * If there exists a home page for the software in question, please + add the variable HOMEPAGE right after MAINTAINER. The value of + this variable should be the URL for the home page. + * Be sure to set the COMMENT variable to a short description of the + package. + +5.2. distinfo + + Most important, the mandatory message digest, or checksum, of all the + distfiles needed for the package to compile, confirming they match the + original file distributed by the author. This ensures that the + distfile retrieved from the Internet has not been corrupted during + transfer or altered by a malign force to introduce a security hole. It + is best generated using the make makesum command. The digest algorithm + used was, at one stage, md5, but that was felt lacking compared to + sha1, and so sha1 is now the default algorithm. The distfile size is + also generated and stored in new distinfo files. The pkgtools/digest + utility calculates all of the digests in the distinfo file, and it + provides various different algorithms. At the current time, the + algorithms provided are: md5, rmd160, sha1, sha256, sha384 and sha512. + + Some packages have different sets of distfiles on a per architecture + basis (a good example is www/navigator). These are kept in the same + distinfo file and care should be taken when upgrading such a package + to ensure distfile information is not lost. + + The message digest/checksum for all the official patches found in the + patches/ directory (see Section 5.3, "patches/*") for the package is + also stored in the distinfo file. This is a message digest/checksum of + all lines in the patch file except the NetBSD RCS Id. This file is + generated by invoking make makepatchsum. + +5.3. patches/* + + This directory contains files that are used by the patch(1) command to + modify the sources as distributed in the distribution file into a form + that will compile and run perfectly on NetBSD. The files are applied + successively in alphabetic order (as returned by a shell + "patches/patch-*" glob expansion), so patch-aa is applied before + patch-ab, etc. + + The patch-?? files should be in diff -bu format, and apply without a + fuzz to avoid problems (To force patches to apply with fuzz you can + set PATCH_FUZZ_FACTOR=-F2). Furthermore, do not put changes for more + than one file into a single patch-file, as this will make future + modifications more difficult. + + Similar, a file should be patched at most once, not several times by + several different patches. If a file needs several patches, they + should be combined into one file. + + One important thing to mention is to pay attention that no RCS IDs get + stored in the patch files, as these will cause problems when later + checked into the NetBSD CVS tree. Use the pkgtools/pkgdiff package to + avoid these problems. + + For even more automation, we recommend using mkpatches from the same + package to make a whole set of patches. You just have to backup files + before you edit them to filename.orig, e.g. with cp -p filename + filename.orig or, easier, by using pkgvi from the same package. If you + upgrade a package this way, you can easily compare the new set of + patches with the previously existing one with patchdiff. + + When you have finished a package, remember to generate the checksums + for the patch files by using the make makepatchsum command, see + Section 5.2, "distinfo". + + If it is desired to store any patches that should not be committed + into pkgsrc, they can be kept outside the pkgsrc tree in the + $LOCALPATCHES directory. The directory tree there is expected to have + the same "category/package" structure as pkgsrc, and patches are + expected to be stored inside these dirs (also known as + $LOCALPATCHES/$PKGPATH). For example if you want to keep a private + patch for pkgsrc/graphics/png, keep it in + $LOCALPATCHES/graphics/png/mypatch. All files in the named directory + are expected to be patch files, and they are applied after pkgsrc + patches are applied. + +5.4. Other mandatory files + + * DESCR + A multi-line description of the piece of software. This should + include any credits where they are due. Please bear in mind that + others do not share your sense of humour (or spelling + idiosyncrasies), and that others will read everything that you + write here. + * PLIST + This file governs the files that are installed on your system: all + the binaries, manual pages, etc. There are other directives which + may be entered in this file, to control the creation and deletion + of directories, and the location of inserted files. + +5.5. Optional files + + * INSTALL + Shell script invoked twice during pkg_add. First time after + package extraction and before files are moved in place, the second + time after the files to install are moved in place. This can be + used to do any custom procedures not possible with @exec commands + in PLIST. See pkg_add(1) and pkg_create(1) for more information. + * DEINSTALL + This script is executed before and after any files are removed. It + is this script's responsibility to clean up any additional messy + details around the package's installation, since all pkg_delete + knows is how to delete the files created in the original + distribution. See pkg_delete(1) and pkg_create(1) for more + information. + * MESSAGE + Display this file after installation of the package. Useful for + things like legal notices on almost-free software, etc. Please + note that you can modify variables in it easily by using + MESSAGE_SUBST in the package's Makefile: + +MESSAGE_SUBST+= SOMEVAR="somevalue" + + replaces "${SOMEVAR}" with "somevalue" in MESSAGE. + +5.6. work* + + When you type make the distribution files are unpacked into this + directory. It can be removed by running make clean. + + This directory is also used to keep various timestamp files. + +5.7. files/* + + If you have any files that you wish to be placed in the package prior + to configuration or building, you could place these files here and use + a "${CP}" command in the pre-configure target to achieve this. + Alternatively, you could simply diff the file against /dev/null and + use the patch mechanism to manage the creation of this file. + +5.8. Portability of packages + + One appealing feature of pkgsrc is that it runs on many different + platforms. As a result, it is important to ensure, where possible, + that packages in pkgsrc are portable. There are some particular + details you should pay attention to while working on pkgsrc. + +5.8.1. ${INSTALL}, ${INSTALL_DATA_DIR}, ... + + The BSD-compatible install supplied with some operating systems will + not perform more than one operation at a time. As such, you should + call "${INSTALL}", etc. like this: +${INSTALL_DATA_DIR} ${PREFIX}/dir1 +${INSTALL_DATA_DIR} ${PREFIX}/dir2 + +Chapter 6. PLIST issues + + Table of Contents + + 6.1. Miscellaneous + 6.2. PLIST_SRC + 6.3. PLIST_SUBST + 6.4. Perl5 modules + 6.5. User Interaction + + This section addresses some special issues that one needs to pay + attention to when dealing with the PLIST file (or files, see below!). + +6.1. Miscellaneous + + * NetBSD RCS Id + Be sure to add a RCS ID line as the first thing in any PLIST file + you write: + +@comment $NetBSD: pkgsrc.txt,v 1.1 2003/06/23 07:41:44 grant Exp $ + + * ${MACHINE_ARCH}, ${MACHINE_GNU_ARCH} + Some packages like emacs and perl embed information about which + architecture they were built on into the pathnames where they + install their file. To handle this case, PLIST will be + preprocessed before actually used, and the symbol + "${MACHINE_ARCH}" will be replaced by what uname -p gives. The + same is done if the string ${MACHINE_GNU_ARCH} is embedded in + PLIST somewhere - use this on packages that have GNU autoconf + created configure scripts. + Legacy note: There used to be a symbol "<$ARCH>" that was replaced + by the output of uname -m, but that's no longer supported and has + been removed. + * ${OPSYS}, ${LOWER_OPSYS}, ${OS_VERSION} + Some packages want to embed the OS name and version into some + paths. To do this, use these variables in the PLIST: + + ${OPSYS} - output of "uname -s" + + ${LOWER_OPSYS} - lowercase common name (eg. "solaris") + + ${OS_VERSION} - "uname -r" + * ${PKGLOCALEDIR} + Packages that install locale files should list them in the PLIST + as "${PKGLOCALEDIR}/locale/de/LC_MESSAGES/..." instead of + "share/locale/de/LC_MESSAGES/...". This properly handles the fact + that different OS's expect locale files to be either in share or + lib by default. + * Manpage-compression + Manpages should be installed in compressed form if MANZ is set (in + bsd.own.mk), and uncompressed otherwise. To handle this in the + PLIST file, the suffix ".gz" is appended/removed automatically for + manpages according to MANZ and MANCOMPRESSED being set or not, see + above for details. This modification of the PLIST file is done on + a copy of it, not PLIST itself. + * Platform specific and differing PLISTs + Some packages decide to install a different set of files based on + the operating system being used. These differences can be + automatically handled by using the following files: + + PLIST.common + + PLIST.${OPSYS} + + PLIST.common_end + If PLIST.${OPSYS} exists, these files are used instead of PLIST. + This allows packages which behave in this way to be handled + gracefully. Manually overriding PLIST_SRC for other more exotic + uses is also possible. + * Semi-automatic PLIST generation + You can use the make print-PLIST command to output a PLIST that + matches any new files since the package was extracted. See below + for more information on this target. + +6.2. PLIST_SRC + + To use one or more files as source for the PLIST used in generating + the binary package, set the variable PLIST_SRC to the names of that + file(s). The files are later concatenated using cat(1), and order of + things is important. + +6.3. PLIST_SUBST + + Similar to MESSAGE_SUBST (see above), you can add variables and their + expansions to this variable in the following way: + PLIST_SUBST+= SOMEVAR="somevalue" + + which replaces all occurrences of "${SOMEVAR}" in the PLIST with + "somevalue". For the values which are replaced by default, please look + in bsd.pkg.mk (and search for PLIST_SUBST). + +6.4. Perl5 modules + + Makefile of packages providing perl5 modules should include the + makefile fragment lang/perl5/module.mk. It provides a do-configure + target for the standard perl configuration for such modules as well as + various hooks to tune this configuration. See comments in this file + for details. + + Perl5 modules will install into different places depending on the + version of perl used during the build process. To address this, the + NetBSD packages system will append lines to the PLIST corresponding to + the files listed in the installed .packlist file generated by most + perl5 modules. This is invoked by defining PERL5_PACKLIST to a + space-separated list of paths to packlist files: + PERL5_PACKLIST= ${PERL5_SITEARCH}/auto/Pg/.packlist + + The variables PERL5_SITELIB, PERL5_SITEARCH, and PERL5_ARCHLIB + represent the three locations in which perl5 modules may be installed, + and may be used by perl5 packages that don't have a packlist. These + three variables are also substituted for in the PLIST. + +6.5. User Interaction + + Occasionally, packages require interaction from the user, and this can + be in a number of ways: + * help in fetching the distfiles + * help to configure the package before it is built + * help during the build process + * help during the installation of a package + + The INTERACTIVE_STAGE definition is provided to notify the pkgsrc + mechanism of an interactive stage which will be needed, and this + should be set in the package's Makefile. e.g. + INTERACTIVE_STAGE= build + + Multiple interactive stages can be specified: + INTERACTIVE_STAGE= configure install + +Chapter 7. Notes on fixes for packages + + Table of Contents + + 7.1. CPP defines + 7.2. Shared libraries - libtool + 7.3. Using libtool on GNU packages that already support libtool + 7.4. GNU Autoconf/Automake + 7.5. Package configuration files + 7.6. Feedback to the author + +7.1. CPP defines + + To port an application to NetBSD, it's usually necessary for the + compiler to be able to judge the system on which it's compiling, and + we use definitions so that the C pre-processor can do this. + + To test whether you are working on a 4.4 BSD-derived system, you + should use the BSD definition, which is defined in <sys/param.h> on + said systems. + #include <sys/param.h> + + and then you can surround the BSD-specific parts of your port using + the conditional: +#if (defined(BSD) && BSD >= 199306) + ... +#endif + + Please use the "__NetBSD__" definition sparingly - it should only + apply to features of NetBSD that are not present in other 4.4-lite + derived BSDs. + +7.2. Shared libraries - libtool + + pkgsrc supports many different machines, with different object formats + like a.out and ELF, and varying abilities to do shared library and + dynamic loading at all. To accompany this, varying commands and + options have to be passed to the compiler, linker, etc. to get the + Right Thing, which can be pretty annoying especially if you don't have + all the machines at your hand to test things. The devel/libtool pkg + can help here, as it just "knows" how to build both static and dynamic + libraries from a set of source files, thus being platform independent. + + Here's how to use libtool in a pkg in seven simple steps: + 1. Add USE_LIBTOOL=yes to the package Makefile. + 2. For library objects, use "${LIBTOOL} --mode=compile ${CC}" in + place of "${CC}". You could even add it to the definition of CC, + if only libraries are being built in a given Makefile. This one + command will build both PIC and non-PIC library objects, so you + need not have separate shared and non-shared library rules. + 3. For the linking of the library, remove any "ar", "ranlib", and "ld + -Bshareable" commands, and use instead: + +${LIBTOOL} --mode=link ${CC} -o ${.TARGET:.a=.la} ${OBJS:.o=.lo} -rpath ${PREFI +X}/lib -version-info major:minor + + Note that the library is changed to have a .la extension, and the + objects are changed to have a .lo extension. Change OBJS as + necessary. This automatically creates all of the .a, + .so.major.minor, and ELF symlinks (if necessary) in the build + directory. Be sure to include "-version-info", especially when + major and minor are zero, as libtool will otherwise strip off the + shared library version. + The "-release" option will produce different results for a.out and + ELF (excluding symlinks) in only one case. An ELF library of the + form "libfoo-release.so.x.y" will have a symlink of + "libfoo.so.x.y" on an a.out platform. This is handled + automatically. + The "-rpath argument" is the install directory of the library + being built. + In the PLIST, include all of the .a, .la, and .so, .so.major and + .so.major.minor files. + 4. When linking shared object (.so) files, i.e. files that are loaded + via dlopen(3), NOT shared libraries, use "-module -avoid-version" + to prevent them getting version tacked on. + PLIST gets the foo.so entry. + 5. When linking programs that depend on these libraries before they + are installed, preface the cc or ld line with "${LIBTOOL} + --mode=link", and it will find the correct libraries (static or + shared), but please be aware that libtool will not allow you to + specify a relative path in -L (such as "-L../somelib"), because it + expects you to change that argument to be the .la file. e.g. + +${LIBTOOL} --mode=link ${CC} -o someprog -L../somelib -lsomelib + + should be changed to: + +${LIBTOOL} --mode=link ${CC} -o someprog ../somelib/somelib.la + + and it will do the right thing with the libraries. + 6. When installing libraries, preface the install or cp command with + "${LIBTOOL} --mode=install", and change the library name to .la. + e.g. + +${LIBTOOL} --mode=install ${BSD_INSTALL_DATA} ${SOMELIB:.a=.la} ${PREFIX}/lib + + This will install the static .a, shared library, any needed + symlinks, and run ldconfig. + +7.3. Using libtool on GNU packages that already support libtool + + Add USE_LIBTOOL=yes and LTCONFIG_OVERRIDE=${WRKSRC}/ltconfig to the + package Makefile as the quick way to bypass the pkg's own libtool. The + pkg's own libtool is created by ltconfig script at do-configure + target. If USE_LIBTOOL and LTCONFIG_OVERRIDE are defined, the + specified ltconfig is overridden, using devel/libtool instead of the + pkg's own libtool. For newer versions of libtool (without ltconfig) it + may be necessary to use LIBTOOL_OVERRIDE=${WRKSRC}/libtool instead. + + If your package makes use of the platform independent library for + loading dynamic shared objects, that comes with libtool (libltdl), you + should include the libtool buildlink2.mk (and set USE_BUILDLINK2=YES). + + Some packages use libtool incorrectly so that the package may not work + or build in some circumstances. Some of the more common errors are: + * The inclusion of a shared object (-module) as a dependent library + in an executable or library. This in itself isn't a problem if one + of two things has been done: + 1. The shared object is named correctly, i.e. libfoo.la, not + foo.la + 2. The -dlopen option is used when linking an executable. + * The use of libltdl without the correct calls to initialisation + routines. The function lt_dlinit() should be called and the macro + LTDL_SET_PRELOADED_SYMBOLS included in executables. + +7.4. GNU Autoconf/Automake + + If a package needs GNU autoconf or automake to be executed to + regenerate the configure script and Makefile.in makefile templates, + then they should be executed in a pre-configure target. Two Makefile + fragments are provided in pkgsrc/mk/autoconf.mk and + pkgsrc/mk/automake.mk to help dealing with these tools. See comments + in these files for details. + + For packages that need only autoconf: +AUTOCONF_REQD= 2.50 # if default version is not good enough +... + +pre-configure: + cd ${WRKSRC}; ${AUTOCONF} + +... +.include "../../mk/autoconf.mk" + + and for packages that need automake and autoconf: +AUTOMAKE_REQD= 1.7.1 # if default version is not good enough +... + +pre-configure: + cd ${WRKSRC}; \ + ${ACLOCAL}; \ + ${AUTOHEADER}; \ + ${AUTOMAKE} -a --foreign -i; \ + ${AUTOCONF} + +... +.include "../mk/automake.mk" + + There are times when the configure process makes additional changes to + the generated files, which then causes the build process to try to + re-execute the automake sequence. This is prevented by touching + various files in the configure stage. If this causes problems with + your package you can set AUTOMAKE_OVERRIDE=NO in the package Makefile. + +7.5. Package configuration files + + Packages should be taught to look for their configuration files in + ${PKG_SYSCONFDIR}, which is passed through to the configure and build + processes. PKG_SYSCONFDIR may be customized in various ways by setting + other make variables: + * PKG_SYSCONFBASE is the main config directory under which all + package configuration files are to be found. This defaults to + ${PREFIX}/etc, but may be overridden in /etc/mk.conf. + * PKG_SYSCONFSUBDIR is the subdirectory of PKG_SYSCONFBASE under + which the configuration files for a particular package may be + found, e.g. the Apache configuration files may all be found under + the httpd/ subdirectory of ${PKG_SYSCONFBASE}. This should be set + in the package Makefile. + * By default, + PKG_SYSCONFDIR=${PKG_SYSCONFBASE}/${PKG_SYSCONFSUBDIR}, but this + may be overridden by setting PKG_SYSCONFDIR.${PKG_SYSCONFVAR} for + a particular package, where PKG_SYSCONFVAR defaults to ${PKGBASE}. + This is not meant to be set by a package Makefile, but is reserved + for users who wish to override the PKG_SYSCONFDIR setting for a + particular package with a special location. + + The only variables that users should customize are PKG_SYSCONFBASE and + PKG_SYSCONFDIR.${PKG_SYSCONFVAR}. Users will typically want to set + PKG_SYSCONFBASE to /etc, or to accept the default location of + ${PREFIX}/etc. + +7.6. Feedback to the author + + If you have found any bugs in the package you make available, if you + had to do special steps to make it run under NetBSD or if you enhanced + the software in various other ways, be sure to report these changes + back to the original author of the program! With that kind of support, + the next release of the program can incorporate these fixes, and + people not using the NetBSD packages system can win from your efforts. + + Support the idea of free software! + +Chapter 8. The build process + + Table of Contents + + 8.1. Program location + 8.2. Main targets + 8.3. Other helpful targets + + The basic steps for building a program are always the same. First the + program's source (distfile) must be brought to the local system and + then extracted. After any patches to compile properly on NetBSD are + applied, the software can be configured, then built (usually by + compiling), and finally the generated binaries, etc. can be put into + place on the system. These are exactly the steps performed by the + NetBSD package system, which is implemented as a series of targets in + a central Makefile, pkgsrc/mk/bsd.pkg.mk. + +8.1. Program location + + Before outlining the process performed by the NetBSD package system in + the next section, here's a brief discussion on where programs are + installed, and which variables influence this. + + The automatic variable PREFIX indicates where all files of the final + program shall be installed. It is usually set to LOCALBASE (/usr/pkg), + or CROSSBASE for pkgs in the "cross" category, though its value + becomes that of X11BASE if USE_IMAKE or USE_X11BASE is set. The value + of PREFIX needs to be put into the various places in the program's + source where paths to these files are encoded. See Section 5.3, + "patches/*" and Section 7.2, "Shared libraries - libtool" for more + details. + + When choosing which of these variables to use, follow the following + rules: + * PREFIX always points to the location where the current pkg will be + installed. When referring to a pkg's own installation path, use + "${PREFIX}". + * LOCALBASE is where all non-X11 pkgs are installed. If you need to + construct a -I or -L argument to the compiler to find includes and + libraries installed by another non-X11 pkg, use "${LOCALBASE}". + * X11BASE is where the actual X11 distribution (from xsrc, etc.) is + installed. When looking for standard X11 includes (not those + installed by a pkg), use "${X11BASE}". + * X11 based pkgs are special in that they may be installed in either + X11BASE or LOCALBASE. To install X11 packages in LOCALBASE, simply + install pkgtools/xpkgwedge. If you need to find includes or + libraries installed by a pkg that has USE_IMAKE or USE_X11BASE in + its pkg Makefile, you need to use both "${X11BASE}" and + "${LOCALBASE}". + * X11PREFIX should be used to refer to the installed location of an + X11 package. X11PREFIX will be set to X11BASE if xpkgwedge is not + installed, and to LOCALBASE if xpkgwedge is installed. + * If xpkgwedge is installed, it is possible to have some packages + installed in X11BASE and some in LOCALBASE. To determine the + prefix of an installed package, the EVAL_PREFIX definition can be + used. It takes pairs in the format "DIRNAME=<package>", and the + make(1) variable DIRNAME will be set to the prefix of the + installed package <package>, or "${X11PREFIX}" if the package is + not installed. + This is best illustrated by example. + The following lines are taken from pkgsrc/wm/scwm/Makefile: + +EVAL_PREFIX+= GTKDIR=gtk+ +CONFIGURE_ARGS+= --with-guile-prefix=${LOCALBASE} \ + --with-gtk-prefix="${GTKDIR}" \ + --enable-multibyte + + Specific defaults can be defined for the packages evaluated using + EVAL_PREFIX, by using a definition of the form: + +GTKDIR_DEFAULT= ${LOCALBASE} + + where GTKDIR corresponds to the first definition in the + EVAL_PREFIX pair. + +8.2. Main targets + + The main targets used during the build process defined in bsd.pkg.mk + are: + * fetch + This will check if the file(s) given in the variables DISTFILES + and PATCHFILES (as defined in the package's Makefile) are present + on the local system in /usr/pkgsrc/distfiles. If they are not + present, an attempt will be made to fetch them using commands of + the form: + +${FETCH_CMD} ${FETCH_BEFORE_ARGS} ${site}${file} ${FETCH_AFTER_ARGS} + + where ${site} varies through several possibilities in turn: first, + MASTER_SITE_OVERRIDE is tried, then the sites specified in either + SITES_file if defined, else MASTER_SITES or PATCH_SITES, as + applies, then finally the value of MASTER_SITE_BACKUP. The order + of all except the first can be optionally sorted by the user, via + setting either MASTER_SORT_AWK or MASTER_SORT_REGEX. + * checksum + After the distfile(s) are fetched, their checksum is generated and + compared with the checksums stored in the distinfo file. If the + checksums don't match, the build is aborted. This is to ensure the + same distfile is used for building, and that the distfile wasn't + changed, e.g. by some malign force, deliberately changed distfiles + on the master distribution site or network lossage. + * extract + When the distfiles are present on the local system, they need to + be extracted, as they are usually in the form of some compressed + archive format, most commonly .tar.gz. If only some of the + distfiles need to be uncompressed, the files to be uncompressed + should be put into EXTRACT_ONLY. If the distfiles are not in + .tar.gz format, they can be extracted by setting EXTRACT_CMD, + EXTRACT_BEFORE_ARGS and/or EXTRACT_AFTER_ARGS. + * patch + After extraction, all the patches named by the PATCHFILES, those + present in the patches subdirectory of the package as well as in + $LOCALPATCHES/$PKGPATH (e.g. /usr/local/patches/graphics/png) are + applied. Patchfiles ending in .Z or .gz are uncompressed before + they are applied, files ending in .orig or .rej are ignored. Any + special options to patch(1) can be handed in PATCH_DIST_ARGS. See + Section 5.3, "patches/*" for more details. + By default patch is given special args to make it fail if the + patches with some lines of fuzz. Please fix (regen) the patches so + that they apply cleanly. The rationale behind this is that patches + that apply cleanly may end up being applied in the wrong place, + and cause severe harm there. + * configure + Most pieces of software need information on the header files, + system calls, and library routines which are available in NetBSD. + This is the process known as configuration, and is usually + automated. In most cases, a script is supplied with the source, + and its invocation results in generation of header files, + Makefiles, etc. + If the program's distfile contains its own configure script, this + can be invoked by setting HAS_CONFIGURE. If the configure script + is a GNU autoconf script, GNU_CONFIGURE should be specified + instead. In either case, any arguments to the configure script can + be specified in the CONFIGURE_ARGS variable, and the configure + script's name can be set in CONFIGURE_SCRIPT if it differs from + the default "configure". + If the program uses an Imakefile for configuration, the + appropriate steps can be invoked by setting USE_IMAKE to "YES". + (If you only want the package installed in $X11PREFIX but xmkmf + not being run, set USE_X11BASE instead!) + * build + Once configuration has taken place, the software can be built on + NetBSD by invoking $MAKE_PROGRAM on $MAKEFILE with $ALL_TARGET as + the target to build. The default MAKE_PROGRAM is "gmake" if + USE_GMAKE is set, "make" otherwise. MAKEFILE is set to "Makefile" + by default, and ALL_TARGET defaults to "all". Any of these + variables can be set to change the default build process. + * install + Once the build stage has completed, the final step is to install + the software in public directories, for users. As in the + build-target, $MAKE_PROGRAM is invoked on $MAKEFILE here, but with + the $INSTALL_TARGET instead, the latter defaulting to "install" + (plus "install.man", if USE_IMAKE is set). + + If no target is specified, the default is "build". If a subsequent + stage is requested, all prior stages are made: e.g. make build will + also perform the equivalent of: +make fetch +make checksum +make extract +make patch +make configure +make build + +8.3. Other helpful targets + + * pre/post-* + For any of the main targets described in the previous section, two + auxiliary targets exist with "pre-" and "post-" used as a prefix + for the main target's name. These targets are invoked before and + after the main target is called, allowing extra configuration or + installation steps, for example, which program's configure script + or install target omitted. + * do-* + Should one of the main targets do the wrong thing, and should + there be no variable to fix this, you can redefine it with the + do-* target. (Note that redefining the target itself instead of + the do-* target is a bad idea, as the pre-* and post-* targets + won't be called anymore, etc.) You will not usually need to do + this. + * reinstall + If you did a make install and you noticed some file was not + installed properly, you can repeat the installation with this + target, which will ignore the "already installed" flag. + * deinstall + This target does a pkg_delete(1) in the current directory, + effectively de-installing the package. The following variables can + be used either on the command line or in /etc/mk.conf to tune the + behaviour: + + PKG_VERBOSE + Add a "-v" to the pkg_delete(1) command. + + DEINSTALLDEPENDS + Remove all packages that require (depend on) the given + package. This can be used to remove any packages that may + have been pulled in by a given package, e.g. if make + deinstall DEINSTALLDEPENDS=1 is done in pkgsrc/x11/kde, this + is likely to remove whole KDE. Works by adding "-R" to the + pkg_delete command line. + * update + This target causes the current package to be updated to the latest + version. The package and all depending packages first get + de-installed, then current versions of the corresponding packages + get compiled and installed. This is similar to manually noting + which packages are currently installed, then performing a series + of make deinstall and make install (or whatever UPDATE_TARGET is + set to) for these packages. + You can use the "update" target to resume package updating in case + a previous make update was interrupted for some reason. However, + in this case, make sure you don't call make clean or otherwise + remove the list of dependent packages in WRKDIR. Otherwise you + lose the ability to automatically update the current package along + with the dependent packages you have installed. + Resuming an interrupted make update will only work as long as the + package tree remains unchanged. If the source code for one of the + packages to be updated has been changed, resuming make update will + most certainly fail! + The following variables can be used either on the command line or + in /etc/mk.conf to alter the behaviour of make update: + + UPDATE_TARGET + Install target to recursively use for the updated package and + the dependent packages. Defaults to DEPENDS_TARGET if set, + "install" otherwise for make update. e.g. make update + UPDATE_TARGET=package + + NOCLEAN + Don't clean up after updating. Useful if you want to leave + the work sources of the updated packages around for + inspection or other purposes. Be sure you eventually clean up + the source tree (see the "clean-update" target below) or you + may run into troubles with old source code still lying around + on your next make or make update. + + REINSTALL + Deinstall each package before installing (making + DEPENDS_TARGET). This may be necessary if the "clean-update" + target (see below) was called after interrupting a running + make update. + + DEPENDS_TARGET + Allows you to disable recursion and hardcode the target for + packages. The default is "update" for the update target, + facilitating a recursive update of prerequisite packages. + Only set DEPENDS_TARGET if you want to disable recursive + updates. Use UPDATE_TARGET instead to just set a specific + target for each package to be installed during make update + (see above). + * clean-update + Clean the source tree for all packages that would get updated if + make update was called from the current directory. This target + should not be used if the current package (or any of its depending + packages) have already been de-installed (e.g., after calling make + update) or you may lose some packages you intended to update. As a + rule of thumb: only use this target before the first time you run + make update and only if you have a dirty package tree (e.g., if + you used NOCLEAN). + If you unsure about whether your tree is clean you can either + perform a make clean at the top of the tree, or use the following + sequence of commands from the directory of the package you want to + update (before running make update for the first time, otherwise + you lose all the packages you wanted to update!): + +# make clean-update +# make clean CLEANDEPENDS=YES +# make update + + The following variables can be used either on the command line or + in /etc/mk.conf to alter the behaviour of make clean-update: + + CLEAR_DIRLIST + After make clean, do not reconstruct the list of directories + to update for this package. Only use this if make update + successfully installed all packages you wanted to update. + Normally, this is done automatically on make update, but may + have been suppressed by the NOCLEAN variable (see above). + * info + This target invokes pkg_info for the current package. You can use + this to check which version of a package is installed. + * readme + This target generates a README.html file, which can be viewed + using a browser such as www/navigator or www/lynx. The generated + files contain references to any packages which are in the PACKAGES + directory on the local host. The generated files can be made to + refer to URLs based on FTP_PKG_URL_HOST and FTP_PKG_URL_DIR. For + example, if I wanted to generate README.html files which pointed + to binary packages on the local machine, in the directory + /usr/packages, set FTP_PKG_URL_HOST=file://localhost and + FTP_PKG_URL_DIR=/usr/packages. The ${PACKAGES} directory and its + subdirectories will be searched for all the binary packages. + * readme-all + Use this target to create a file README-all.html which contains a + list of all packages currently available in the NetBSD Packages + Collection, together with the category they belong to and a short + description. This file is compiled from the pkgsrc/*/README.html + files, so be sure to run this after a make readme. + * cdrom-readme + This is very much the same as the "readme" target (see above), but + is to be used when generating a pkgsrc tree to be written to a + CD-ROM. This target also produces README.html files, and can be + made to refer to URLs based on CDROM_PKG_URL_HOST and + CDROM_PKG_URL_DIR. + * show-distfiles + This target shows which distfiles and patchfiles are needed to + build the package. (DISTFILES and PATCHFILES, but not patches/*) + * show-downlevel + This target shows nothing if the package is not installed. If a + version of this package is installed, but is not the version + provided in this version of pkgsrc, then a warning message is + displayed. This target can be used to show which of your installed + packages are downlevel, and so the old versions can be deleted, + and the current ones added. + * show-pkgsrc-dir + This target shows the directory in the pkgsrc hierarchy from which + the package can be built and installed. This may not be the same + directory as the one from which the package was installed. This + target is intended to be used by people who may wish to upgrade + many packages on a single host, and can be invoked from the + top-level pkgsrc Makefile by using the "show-host-specific-pkgs" + target. + * show-installed-depends + This target shows which installed packages match the current + package's DEPENDS. Useful if out of date DEPENDS are causing build + problems. + * check-shlibs + After a package is installed, check all its binaries and (on ELF + platforms) shared libraries to see if they find the shared libs + they need. Run by default if PKG_DEVELOPER is set in /etc/mk.conf. + * print-PLIST + After a "make install" from a new or upgraded pkg, this prints out + an attempt to generate a new PLIST from a find -newer + work/.extract_done. An attempt is made to care for shared libs + etc., but it is strongly recommended to review the result before + putting it into PLIST. On upgrades, it's useful to diff the output + of this command against an already existing PLIST file. + If the package installs files via tar(1) or other methods that + don't update file access times, be sure to add these files + manually to your PLIST, as "find -newer" won't catch them! + * bulk-package + Used to do bulk builds. If an appropriate binary package already + exists, no action is taken. If not, this target will compile, + install and package it (and it's depends, if PKG_DEPENDS is set + properly. See Section 4.3.1, "Configuration". After creating the + binary package, the sources, the just-installed package and it's + required packages are removed, preserving free disk space. + * bulk-install + Used during bulk-installs to install required packages. If an + appropriate binary package is available, it will be installed via + pkg_add. If not, make bulk-package will be executed, but the + installed binary not be removed. A binary package is "appropriate" + to be installed via pkg_add if: + + None of the package's files (Makefile, ...) were modified + since it was built. + + None of the package's required (binary) packages were + modified since it was built. + +Chapter 9. buildlink2 methodology + + Table of Contents + + 9.1. Converting packages to use buildlink2 + 9.2. Writing buildlink2.mk files + + buildlink2 is a pkgsrc framework that controls what headers and + libraries are seen by a package's configure and build processes. This + is implemented in a two step process: + 1. Symlink headers and libraries for dependencies into BUILDLINK_DIR, + which by default is a subdirectory of WRKDIR. + 2. Create wrapper scripts that are used in place of the normal + compiler tools that translate "-I${LOCALBASE}/include" and + "-L${LOCALBASE}/lib" into references to BUILDLINK_DIR. + + This normalizes the environment in which a package is built so that + the package may be built consistently despite what may other software + may installed. Please refer to pkgsrc/mk/buildlink2/buildlink2.txt for + some FAQs and answers regarding buildlink2, and to + pkgsrc/mk/buildlink2/README for a description of how buildlink2 is + implemented in pkgsrc. + +9.1. Converting packages to use buildlink2 + + The process of converting packages to use the buildlink2 framework is + fairly straightforward. The package Makefile must define + USE_BUILDLINK2. If a dependency on a particular package, e.g. foo, is + required for its libraries and headers, then we replace: + DEPENDS+= foo>=1.1.0:../../category/foo + + with + .include "../../category/foo/buildlink2.mk" + + There are several buildlink2.mk files in pkgsrc/mk that handle special + package issues: + * motif.buildlink2.mk checks for a system-provided Motif + installation or adds a dependency on x11/lesstif or x11/openmotif; + * ossaudio.buildlink2.mk defines several variables that may be used + by packages that use the Open Sound System (OSS) API; + * pthread.buildlink2.mk uses the value of PTHREAD_OPTS and checks + for native pthreads or adds a dependency on devel/pth as needed; + * xaw.buildlink2.mk uses the value of XAW_TYPE to choose a + particular Athena widgets library. + + The comments in those buildlink2.mk files provide a more complete + description of how to use them properly. + +9.2. Writing buildlink2.mk files + + A simple example of a buildlink2.mk file for a mythical package foo + follows: +BUILDLINK_PACKAGES+= foo +BUILDLINK_PKGBASE.foo= foo +BUILDLINK_DEPENDS.foo?= foo>=1.0 +BUILDLINK_PKGSRCDIR.foo?= ../../category/foo + +EVAL_PREFIX+= BUILDLINK_PREFIX.foo=foo +BUILDLINK_PREFIX.foo_DEFAULT= ${LOCALBASE} +BUILDLINK_FILES.foo= include/foo.h +BUILDLINK_FILES.foo+= include/bar.h +BUILDLINK_FILES.foo+= lib/libfoo.* + +BUILDLINK_TARGETS+= foo-buildlink + +foo-buildlink: _BUILDLINK_USE + + The first section controls how the dependency on foo is added. The + dependency is constructed from four parts: + 1. BUILDLINK_PACKAGES is the global list of packages for which + dependencies will be added by buildlink2; + 2. BUILDLINK_DEPENDS.foo is the actual dependency recorded in the + installed package; + 3. BUILDLINK_PKGSRCDIR.foo is the location of the foo pkgsrc + directory; + 4. BUILDLINK_DEPMETHOD.foo (not shown above) controls whether we use + BUILD_DEPENDS or DEPENDS to add the foo dependency, where the full + dependency is added if BUILDLINK_DEPMETHOD.foo contains "full". + + The second section controls which files are linked into BUILDLINK_DIR: + 1. BUILDLINK_PREFIX.foo is the installation prefix of the package + which we derive by using EVAL_PREFIX; + 2. BUILDLINK_FILES.foo is a list of files (shell globs allowed) + relative to the BUILDLINK_PREFIX.foo directory and will be + symlinked into BUILDLINK_DIR; + 3. BUILDLINK_FILES_CMD.foo (not shown above) is a shell pipeline that + outputs a list of files relative to the BUILDLINK_PREFIX.foo + directory and will be symlinked into BUILDLINK_DIR. + + The remaining parts create the "foo-buildlink" target that actually + performs the symlinking and adds the "foo-buildlink" target to + BUILDLINK_TARGETS, which is the global list of targets to execute at + do-buildlink time. + +Chapter 10. Debugging + + To check out all the gotchas when building a package, here are the + steps that I do in order to get a package working. Please note this is + basically the same as what was explained in the previous sections, + only with some debugging aids. + * Be sure to set PKG_DEVELOPER=1 in /etc/mk.conf + * Install pkgtools/url2pkg and run + +# url2pkg http://www.example.com/path/to/distfile.tar.gz + + * Edit the Makefile as requested. + * Fill in DESCR + * Run make configure + * Add any dependencies glimpsed from the configure step to the + package's Makefile. + * Make the package compile, doing multiple rounds of + +# make +# pkgvi ${WRKSRC}/some/file/that/does/not/compile +# mkpatches +# patchdiff +# mv ${WRKDIR}/.newpatches/* patches +# make mps +# make clean + + Doing as non-root user will ensure that no files are modified that + shouldn't be, especially during the build phase. + * Look at Makefile, fix if necessary; see Section 5.1, "Makefile". + * Generate a PLIST: + +# make install +# make print-PLIST > PLIST +# make deinstall +# make install +# make deinstall + + You usually need to be root to do this. Look if there are any + files left: + +# make print-PLIST + + If this reveals any files that are missing in PLIST, add them. + * Now that the PLIST is OK, install the package again and make a + binary package: + +# make reinstall && make package + + * Delete the installed package: + +# pkg_delete blub + + * Repeat the above find command, which shouldn't find anything now: + +# make print-PLIST + + * Reinstall the binary package: + +# pkgadd .../blub.tgz + + * Play with it. Make sure everything works. + * Run pkglint from pkgtools/pkglint, and fix the problems it + reports. + +# pkglint + + * Submit (or commit, if you have cvs access); see Chapter 12, + Submitting and Committing. + +Chapter 11. FAQs & features of the package system + + Table of Contents + + 11.1. Packages using GNU autoconf + 11.2. Other distrib methods than .tar.gz + 11.3. Packages not creating their own subdirectory + 11.4. Custom configuration process + 11.5. Packages not building in their DISTNAME directory + 11.6. How to fetch all distfiles at once + 11.7. How to fetch files from behind a firewall + 11.8. If your patch contains an RCS ID + 11.9. How to pull in variables from /etc/mk.conf + 11.10. Is there a mailing list for pkg-related discussion? + 11.11. How do I tell make fetch to do passive FTP? + 11.12. Dependencies on other packages + 11.13. Conflicts with other packages + 11.14. Software which has a WWW Home Page + 11.15. How to handle modified distfiles with the 'old' name + 11.16. What does "Don't know how to make /usr/share/tmac/tmac.andoc" + mean? + + 11.17. How to handle incrementing versions when fixing an existing + package + + 11.18. Could not find bsd.own.mk - what's wrong? + 11.19. Restricted packages + 11.20. Packages using (n)curses + 11.21. Automated security check + 11.22. What's the proper way to create an account from a package? + 11.23. How to handle compiler bugs + 11.24. Packages providing info files + 11.25. Packages whose distfiles aren't available for plain downloading + + 11.26. Configuration files handling and placement + 11.27. Packages providing login shells + 11.28. Packages providing locale catalogues + 11.29. Using 'sudo' with pkgsrc + 11.30. Packages containing perl scripts + 11.31. Packages that cannot or should not be built + +11.1. Packages using GNU autoconf + + If your package uses GNU autoconf created configure scripts, add the + following to your package's Makefile: + GNU_CONFIGURE= yes + + Note that this appends "--prefix=${PREFIX}" to CONFIGURE_ARGS, so you + don't have to do that yourself, but may not be desired. + +11.2. Other distrib methods than .tar.gz + + If your package uses a different distribution method from .tar.gz, + take a look at the package for editors/sam, which uses a gzipped shell + archive (shar), but the quick solution is to set EXTRACT_SUFX to the + name after the DISTNAME field, and add the following to your package's + Makefile: +EXTRACT_SUFX= .msg.gz +EXTRACT_CMD= zcat +EXTRACT_BEFORE_ARGS= +EXTRACT_AFTER_ARGS= |sh + +11.3. Packages not creating their own subdirectory + + Your package doesn't create a subdirectory for itself (like GNU + software does, for instance), but extracts itself in the current + directory: see editors/sam again, but the quick answer is: + WRKSRC= ${WRKDIR} + + Please note that the old: + NO_WRKSUBDIR= yes + + has been deprecated and should not be used. + +11.4. Custom configuration process + + Your package uses a weird Configure script, eg. sysutils/top. The + quick answer is: +HAS_CONFIGURE= yes +CONFIGURE_SCRIPT= Configure +CONFIGURE_ARGS+= netbsd13 + +11.5. Packages not building in their DISTNAME directory + + Your package builds in a different directory from its base DISTNAME + (see lang/tcl and x11/tk). + WRKSRC= ${WRKDIR}/${DISTNAME}/unix + +11.6. How to fetch all distfiles at once + + You would like to download all the distfiles in a single batch from + work or university, where you can't run a make fetch. There is an + archive of distfiles on ftp.NetBSD.org, but downloading the entire + directory may not be appropriate. + + The answer here is to do a make fetch-list in /usr/pkgsrc, carry the + resulting list to your machine at work/school and use it there If you + don't have a NetBSD-compatible ftp(1) (like lukemftp) at work, don't + forget to set FETCH_CMD to something that fetches an URL: + + At home: +% cd /usr/pkgsrc +% make fetch-list FETCH_CMD=wget DISTDIR=/tmp/distfiles >/tmp/fetch.sh +% scp /tmp/fetch.sh work:/tmp + + At work: + % sh /tmp/fetch.sh + + then tar up /tmp/distfiles and take it home. + + If you have a machine running NetBSD, and you want to get all + distfiles (even ones that aren't for your machine architecture), you + can do so by using the above-mentioned make fetch-list approach, or + fetch the distfiles directly by running: + % make mirror-distfiles + + If you even decide to ignore NO_{SRC,BIN}_ON_{FTP,CDROM}, then you can + get everything by running: + % make fetch NO_SKIP=yes + +11.7. How to fetch files from behind a firewall + + If you are sitting behind a firewall which does not allow direct + connections to Internet hosts (i.e. non-NAT), you may specify the + relevant proxy hosts. This is done using an environment variable in + the form of a URL e.g. in Amdahl, the machine "orpheus.amdahl.com" is + one of the firewalls, and it uses port 80 as the proxy port number. So + the proxy environment variables are: +ftp_proxy=ftp://orpheus.amdahl.com:80/ +http_proxy=http://orpheus.amdahl.com:80/ + +11.8. If your patch contains an RCS ID + + See Section 5.3, "patches/*" for information on how to remove RCS IDs + from patch files. + +11.9. How to pull in variables from /etc/mk.conf + + The problem with package-defined variables that can be overridden via + MAKECONF or /etc/mk.conf is that make(1) expands a variable as it is + used, but evaluates preprocessor like statements (.if, .ifdef and + .ifndef) as they are read. So, to use any variable (which may be set + in /etc/mk.conf) in one of the .if* statements, the file /etc/mk.conf + must be included before that .if* statement. + + Rather than have a number of ad-hoc ways of including /etc/mk.conf, + should it exist, or MAKECONF, should it exist, include the + pkgsrc/mk/bsd.prefs.mk file in the package Makefile before any + preprocessor-like .if, .ifdef, or .ifndef statements: +.include "../../mk/bsd.prefs.mk" + +.if defined(USE_MENUS) + ... +.endif + +11.10. Is there a mailing list for pkg-related discussion? + + Yes, <tech-pkg@NetBSD.org> is the list for discussing package related + issues. To subscribe do: + % echo subscribe tech-pkg | mail majordomo@NetBSD.org + +11.11. How do I tell make fetch to do passive FTP? + + This depends on which utility is used to retrieve distfiles. From + bsd.pkg.mk, FETCH_CMD is assigned the first available command from the + following list: +/usr/bin/fetch +${LOCALBASE}/bsd/bin/ftp +/usr/bin/ftp + + On a default NetBSD install, this will be /usr/bin/ftp, which + automatically tries passive connections first, and falls back to + active connections if the server refuses to do passive. For the other + tools, add the following to your /etc/mk.conf file: PASSIVE_FETCH=1. + + Having that option present will prevent /usr/bin/ftp from falling back + to active transfers. + +11.12. Dependencies on other packages + + Your package may depend on some other package being present - and + there are various ways of expressing this dependency. NetBSD supports + the BUILD_DEPENDS and DEPENDS definitions, as well as dependencies via + buildlink2.mk (see Chapter 9, buildlink2 methodology). + + The basic difference between the two definitions is as follows: The + DEPENDS definition registers that pre-requisite in the binary package, + whilst the BUILD_DEPENDS definition does not. + + This means that if you only need a package present whilst you are + building, it should be noted as a BUILD_DEPENDS. + + The format for a BUILD_DEPENDS and a DEPENDS definition is: + <pre-req-package-name>:../../<category>/<pre-req-package> + + Please note that the "pre-req-package-name" may include any of the + wildcard version numbers recognised by pkg_info(1). + 1. If your package needs to use another package to build itself, this + is specified using the BUILD_DEPENDS definition. + +BUILD_DEPENDS+= autoconf-2.13:../../devel/autoconf + + 2. If your package needs a library with which to link, this is + specified using the DEPENDS definition. An example of this is the + print/lyx package, which uses the xpm library, version 3.4j to + build. + +DEPENDS+= xpm-3.4j:../../graphics/xpm + + You can also use wildcards in package dependences: + +DEPENDS+= xpm-[0-9]*:../../graphics/xpm + + Note that such wildcard dependencies are retained when creating + binary packages. The dependency is checked when installing the + binary package and any package which matches the pattern will be + used. Wildcard dependencies should be used with care. + The -[0-9]* should be used instead of -* to avoid potentially + ambiguous matches such as tk-postgresql matching a tk-* DEPENDS. + 3. If your package needs some executable to be able to run correctly, + this is specified using the DEPENDS definition. The print/lyx + package needs to be able to execute the latex binary from the + teTeX package when it runs, and that is specified: + +DEPENDS+= teTeX-[0-9]*:../../print/teTeX + + The comment about wildcard dependencies from previous paragraph + applies here, too. + + If your package needs files from another package to build, see the + first part of the "do-configure" target print/ghostscript5 package (it + relies on the jpeg sources being present in source form during the + build): +if [ ! -e ${_PKGSRCDIR}/graphics/jpeg/${WRKDIR:T}/jpeg-6b ]; then \ + cd ${_PKGSRCDIR}/../../graphics/jpeg && ${MAKE} extract; \ +fi + + If you build any other packages that way, please make sure the working + files are deleted too when this package's working files are cleaned + up. The easiest way to do so is by adding a pre-clean target: +pre-clean: + cd ${_PKGSRCDIR}/../../graphics/jpeg && ${MAKE} clean + + Please also note the BUILD_USES_MSGFMT and BUILD_USES_GETTEXT_M4 + definitions, which are provided as convenience definitions. The former + works out whether msgfmt(1) is part of the base system, and, if it + isn't, installs the devel/gettext package. The latter adds a build + dependency on either an installed version of an older gettext package, + or if it isn't, installs the devel/gettext-m4 package. + +11.13. Conflicts with other packages + + Your package may conflict with other packages a user might already + have installed on his system, e.g. if your package installs the same + set of files like another package in our pkgsrc tree. + + In this case you can set CONFLICTS to a space separated list of + packages (including version string) your package conflicts with. + + For example x11/Xaw3d and x11/Xaw-Xpm install provide the same shared + library, thus you set in pkgsrc/x11/Xaw3d/Makefile: + CONFLICTS= Xaw-Xpm-[0-9]* + + and in pkgsrc/x11/Xaw-Xpm/Makefile: + CONFLICTS= Xaw3d-[0-9]* + + Packages will automatically conflict with other packages with the name + prefix and a different version string. "Xaw3d-1.5" e.g. will + automatically conflict with the older version "Xaw3d-1.3". + +11.14. Software which has a WWW Home Page + + The NetBSD packages system now supports a variable called HOMEPAGE. If + the software being packaged has a home page, the Makefile should + include the URL for that page in the HOMEPAGE variable. The definition + of the variable should be placed immediately after the MAINTAINER + variable. + +11.15. How to handle modified distfiles with the 'old' name + + Sometimes authors of a software package make some modifications after + the software was released, and they put up a new distfile without + changing the package's version number. If a package is already in + pkgsrc at that time, the md5 checksum will no longer match. The + correct way to work around this is to update the package's md5 + checksum to match the package on the master site (beware, any mirrors + may not be up to date yet!), and to remove the old distfile from + ftp.NetBSD.org's /pub/NetBSD/packages/distfiles directory. + Furthermore, a mail to the package's author seems appropriate making + sure the distfile was really updated on purpose, and that no trojan + horse or so crept in. + +11.16. What does "Don't know how to make /usr/share/tmac/tmac.andoc" mean? + + When compiling the pkgtools/pkg_install package, you get the error + from make that it doesn't know how to make /usr/share/tmac/tmac.andoc? + This indicates that you don't have installed the "text" set on your + machine (nroff, ...). It is recommended to do that. + + In the case of the pkgtools/pkg_install package, you can get away with + setting NOMAN=YES either in the environment or in /etc/mk.conf. + +11.17. How to handle incrementing versions when fixing an existing package + + When making fixes to an existing package it can be useful to change + the version number in PKGNAME. To avoid conflicting with future + versions by the original author, a "nb1", "nb2", ... suffix can be + used on package versions by setting PKGREVISION=1 (2,. ..). The "nb" + is treated like a "." by the pkg tools. e.g. +DISTNAME= foo-17.42 +PKGREVISION= 9 + + will result in a PKGNAME of "foo-17.42nb9". + + When a new release of the package is released, the PKGREVISION should + be removed. e.g. on a new minor release of the above package, things + should be like: + DISTNAME= foo-17.43 + +11.18. Could not find bsd.own.mk - what's wrong? + + You didn't install the compiler set, comp.tgz, when you installed your + NetBSD machine. Please get it and install it, by extracting it in /: +# cd / +# tar --unlink -zxvpf .../comp.tgz + + comp.tgz is part of every NetBSD release. Get the one that corresponds + to your release (determine via uname -r). + +11.19. Restricted packages + + Some licenses restrict how software may be re-distributed. In order to + satisfy these restrictions, the package system defines five make + variables that can be set to note these restrictions: + * RESTRICTED + This variable should be set whenever a restriction exists + (regardless of its kind). Set this variable to a string containing + the reason for the restriction. + * NO_BIN_ON_CDROM + Binaries may not be placed on CD-ROM. Set this variable to + ${RESTRICTED} whenever a binary package may not be included on a + CD-ROM. + * NO_BIN_ON_FTP + Binaries may not be placed on an FTP server. Set this variable to + ${RESTRICTED} whenever a binary package may not not be made + available on the Internet. + * NO_SRC_ON_CDROM + Distfiles may not be placed on CD-ROM. Set this variable to + ${RESTRICTED} if re-distribution of the source code or other + distfile(s) is not allowed on CD-ROMs. + * NO_SRC_ON_FTP + Distfiles may not be placed on FTP. Set this variable to + ${RESTRICTED} if re-distribution of the source code or other + distfile(s) via the Internet is not allowed. + + Please note that the use of NO_PACKAGE, IGNORE, NO_CDROM, or other + generic make variables to denote restrictions is deprecated, because + they unconditionally prevent users from generating binary packages! + +11.20. Packages using (n)curses + + Some packages need curses functionality that wasn't present in + NetBSD's own curses prior to 1.4Y. + + If ../../devel/ncurses/buildlink2.mk is included in a package's + Makefile, then a curses library and headers with ncurses functionality + are linked into ${BUILDLINK_DIR} at pre-configure time. If ncurses is + actually required, then define USE_NCURSES in the package's Makefile. + +11.21. Automated security check + + Please be aware that there can often be bugs in third-party software, + and some of these bugs can leave a machine vulnerable to exploitation + by attackers. In an effort to lessen the exposure, the NetBSD packages + team maintains a database of known-exploits to packages which have at + one time been included in pkgsrc. The database can be downloaded + automatically, and a security audit of all packages installed on a + system can take place. To do this, install the security/audit-packages + package. It has two components: + 1. "download-vulnerability-list", an easy way to download a list of + the security vulnerabilities information. This list is kept up to + date by the NetBSD security officer and the NetBSD packages team, + and is distributed from the NetBSD ftp server: + ftp://ftp.NetBSD.org/pub/NetBSD/packages/distfiles/vulnerabilities + 2. "audit-packages", an easy way to audit the current machine, + checking each vulnerability which is known. If a vulnerable + package is installed, it will be shown by output to stdout, + including a description of the type of vulnerability, and a URL + containing more information. + + Use of the audit-packages package is strongly recommended. + + The following message is displayed as part of the audit-packages + installation procedure: +====================================================================== +You may wish to have the vulnerabilities file downloaded daily so that +it remains current. This may be done by adding an appropriate entry +to the root users crontab(5) entry. For example the entry + +# download vulnerabilities file +0 3 * * * ${PREFIX}/sbin/download-vulnerability-list >/dev/null 2>&1 + +will update the vulnerability list every day at 3AM. + +In addition, you may wish to run the package audit from the daily +security script. This may be accomplished by adding the following +lines to /etc/security.local + +if [ -x ${PREFIX}/sbin/audit-packages ]; then + ${PREFIX}/sbin/audit-packages +fi +====================================================================== + + Note to package developers: When a vulnerability is found, this should + be noted in localsrc/security/advisories/pkg-vulnerabilities, and + after the commit of that file, it should be copied to + /pub/NetBSD/packages/distfiles/vulnerabilities on ftp.NetBSD.org. + +11.22. What's the proper way to create an account from a package? + + There are two make variables used to control the creation of + package-specific groups and users at pre-install time. The first is + PKG_GROUPS, which is a list of group[:groupid] elements, where the + groupid is optional. The second is PKG_USERS, which is a list of + elements of the form: + user:group[:[userid][:[description][:[home][:shell]]]] + + where only the user and group are required, the rest being optional. A + simple example is: +PKG_GROUPS= foogroup +PKG_USERS= foouser:foogroup + + A more complex example is that creates two groups and two users is: +PKG_GROUPS= group1 group2:1005 +PKG_USERS= first:group1::First\\ User \ + second:group2::Second\\ User:/home/second:${SH} + + By default, a new user will have home directory /nonexistent, and + login shell /sbin/nologin unless they are specified as part of the + user element. + + The package Makefile must also include ../../mk/bsd.pkg.install.mk + prior to the inclusion of bsd.pkg.mk. This will cause the users and + groups to be created at pre-install time, and the admin will be + prompted to remove them at post-deinstall time. Automatic creation of + the users and groups can be toggled on and off by setting the + environment variable PKG_CREATE_USERGROUP prior to package + installation. + +11.23. How to handle compiler bugs + + Some source files trigger bugs in the compiler, based on combinations + of compiler version and architecture and almost always relation to + optimisation being enabled. Common symptoms are gcc internal errors or + never finishing compiling a file. + + Typically a workaround involves testing the MACHINE_ARCH and compiler + version, disabling optimisation for that file/MACHINE_ARCH/compiler + combination, and documenting it in doc/HACKS. See doc/HACKS for + examples. + +11.24. Packages providing info files + + Some packages install info files or use the "makeinfo" or + "install-info" commands. Each info file: + * is considered to be installed in the directory + ${PREFIX}/${INFO_DIR}, + * is registered in the Info directory file + ${PREFIX}/${INFO_DIR}/dir, + * and must be listed as a filename in the INFO_FILES variable in the + package Makefile. + + INFO_DIR defaults to "info" and can be overridden in the package + Makefile. INSTALL and DEINSTALL scripts will be generated for handling + registration of the info files in the Info directory file. The command + "install-info" used for the info files registration is either provided + by the system, or by a special purpose package automatically added as + dependency if needed. + + A package which need the "makeinfo" command at build time must define + the variable USE_MAKEINFO in its Makefile. If a minimum version of the + "makeinfo" command is needed it should be noted with the TEXINFO_REQD + variable in the package Makefile. By default, a minimum version of + 3.12 is required. If the system does not provide a "makeinfo" command + or if it does not match the required minimum, a build dependency on + the devel/gtexinfo package is added automatically. + + The installation process of the software provided by the package must + not use "install-info", as the registration of info files is the task + of the package INSTALL sript, and it must use the right "makeinfo". + + If the package use buildlink2 framework no special action should be + needed to achieve this goal. + + If the package does not use the buildlink2 framework patch files are + likely to be needed so the build and installation process of the + software picks up the possibly dummy values of INSTALL_INFO and + MAKEINFO variables. + +Note + + Temporarily, the variable USE_NEW_TEXINFO must be defined in the + package Makefile. Previously, info files, "install-info" and + "makeinfo" were handled somewhat differently and the two ways will + coexist for a short period of time until all older packages are + updated. + +11.25. Packages whose distfiles aren't available for plain downloading + + If you need to download from a dynamic URL you can set + DYNAMIC_MASTER_SITES and a make fetch will call files/getsite.sh with + the name of each file to download as an argument, expecting it to + output the URL of the directory from which to download it. + graphics/ns-cult3d is an example of this usage. + + If the download can't be automated, because the user must submit + personal information to apply for a password, or must pay for the + source, or whatever, you can set _FETCH_MESSAGE to a macro which + displays a message explaining the situation. _FETCH_MESSAGE must be + executable shell commands, not just a message. (Generally, it executes + ${ECHO}). As of this writing, the following packages use this: + audio/realplayer, cad/simian, devel/ipv6socket, + emulators/vmare-module, fonts/acroread-jpnfont, + sysutils/storage-manager, www/ap-aolserver, www/openacs. Try to be + consistent with them. + +11.26. Configuration files handling and placement + + The global variable PKG_SYSCONFBASE (and some others) can be set by + the system administrator in /etc/mk.conf to define the place where + configuration files get installed. Therefore, packages must be adapted + to support this feature. Keep in mind that you should only install + files that are strictly necessary in the configuration directory, + files that can go to $PREFIX/share should go there. + + We will take a look at available variables first (bsd.pkg.mk contains + more information). PKG_SYSCONFDIR is where the configuration files for + a package may be found (that is, the full path, e.g. /etc or + /usr/pkg/etc). This value may be customized in various ways: + 1. PKG_SYSCONFBASE is the main config directory under which all + package configuration files are to be found. Users will typically + want to set it to /etc, or accept the default location of + $PREFIX/etc. + 2. PKG_SYSCONFSUBDIR is the subdirectory of PKG_SYSCONFBASE under + which the configuration files for a particular package may be + found. Defaults to ${SYSCONFBASE}. + 3. PKG_SYSCONFVAR is the special suffix used to distinguish any + overriding values for a particular package (see next item). It + defaults to ${PKGBASE}, but for a collection of related packages + that should all have the same PKG_SYSCONFDIR value, it can be set + in each of the package Makefiles to a common value. + 4. PKG_SYSCONFDIR.${PKG_SYSCONFVAR} overrides the value of + ${PKG_SYSCONFDIR} for packages with the same value for + PKG_SYSCONFVAR. + As an example, all the various KDE packages may want to set + PKG_SYSCONFVAR to "kde" so admins can set PKG_SYSCONFDIR.kde in + /etc/mk.conf to define where to install KDE config files. + + Programs' configuration directory should be defined during the + configure stage. Packages that use GNU autoconf can usually do this by + using the "--sysconfdir" parameter, but this brings some problems as + we will see now. When you change this pathname in packages, you should + not allow them to install files in that directory directly. Instead + they need to install those files under share/examples/${PKGNAME} so + PLIST can register them. + + Once you have the required configuration files in place (under the + share/examples directory) the variable CONF_FILES should be set to + copy them into PKG_SYSCONFDIR. The contents of this variable is formed + by pairs of filenames; the first element of the pair specifies the + file inside the examples directory (registered by PLIST) and the + second element specifies the target file. This is done this way to + allow binary packages to place files in the right directory using + INSTALL/DEINSTALL scripts which are created automatically. The package + Makefile must also include ../../mk/bsd.pkg.install.mk prior to the + inclusion of bsd.pkg.mk to use these automatically generated scripts. + The automatic copying of config files can be toggled by setting the + environment variable PKG_CONFIG prior to package installation. + + Here is an example, taken from mail/mutt/Makefile: +EGDIR= ${PREFIX}/share/doc/mutt/samples +CONF_FILES= ${EGDIR}/Muttrc ${PKG_SYSCONFDIR}/Muttrc + + As you can see, this package installs configuration files inside + EGDIR, which are registered by PLIST. After that, the variable + CONF_FILES lists the installed file first and then the target file. + Users will also get an automatic message when files are installed + using this method. + +11.27. Packages providing login shells + + If the purpose of the package is to provide a login shell, the + variable PKG_SHELL should contain the full pathname of the shell + executable installed by this package. The package Makefile also must + include ../../mk/bsd.pkg.install.mk prior to the inclusion of + bsd.pkg.mk to use the automatically generated INSTALL/DEINSTALL + scripts. + + An example taken from shells/zsh: +PKG_SHELL= ${PREFIX}/bin/zsh +.include "../../mk/bsd.pkg.install.mk" + + The shell is registered into /etc/shells file automatically in the + post-install target by the INSTALL script generated by + bsd.pkg.install.mk and removed in the deinstall target by the + DEINSTALL script. + +11.28. Packages providing locale catalogues + + If the package provides its own locale catalogues, the variable + USE_PKGLOCALEDIR should be defined. It will ensure that the package's + Makefile template files are fixed and point to the correct locale + directories (which may vary, depending on OS), if necessary. See + Section 6.1, "Miscellaneous" for details about PKGLOCALEDIR. This + functionality is buildlink2-only. + +11.29. Using 'sudo' with pkgsrc + + When installing packages as non-root user and using the just-in-time + su(1) feature of pkgsrc, it can become annoying to type in the root + password for each required package installed. To avoid this, the sudo + package can be used, which does password caching over a limited time. + To use it, install sudo (either as binary package or from + security/sudo) and then put the following into your /etc/mk.conf: + SU_CMD=/usr/pkg/bin/sudo /bin/sh -c + +11.30. Packages containing perl scripts + + If your package contains interpreted perl scripts, set REPLACE_PERL to + ensure that the proper interpreter path is set. REPLACE_PERL should + contain a list of scripts, relative to WRKSRC, that you want adjusted. + +11.31. Packages that cannot or should not be built + + There are several reasons why a package might be instructed to not + build under certain circumstances. If the package builds and runs on + most platforms, the exceptions should be noted with NOT_FOR_PLATFORM. + If the package builds and runs on a small handful of platforms, set + ONLY_FOR_PLATFORM instead. If the package should be skipped (for + example, because it provides functionality already provided by the + system), set PKG_SKIP_REASON to a descriptive message. If the package + should fail because some preconditions are not met, set + PKG_FAIL_REASON to a descriptive message. + + IGNORE is deprecated because it didn't provide enough information to + determine whether the build should fail. + +Chapter 12. Submitting and Committing + + Table of Contents + + 12.1. Submitting your packages + 12.2. Committing: Importing a package into CVS + 12.3. Updating a Package to a Newer Version + 12.4. Moving a Package in pkgsrc + +12.1. Submitting your packages + + You have to separate between binary and "normal" (source) packages + here: + * precompiled binary packages + Our policy is that we accept binaries only from NetBSD developers + to guarantee that the packages don't contain any trojan horses + etc. This is not to piss anyone off but rather to protect our + users! You're still free to put up your home-made binary packages + and tell the world where to get them. + * packages + First, check that your package is complete, compiles and runs + well; see Chapter 10, Debugging and the rest of this document. + Next, generate a gzipped tar-file of all the files needed for the + package, preferably with all files in a single directory. Place + this tar-file to a place where the package maintainers can fetch + it using FTP or HTTP (WWW). Finally, send-pr with category "pkg", + a synopsis which includes the package name and version number, a + short description of your package (contents of the COMMENT + variable are OK) and the URL of your tar-file. + You will be notified if your PR has been addressed so you can + remove the tar-file. + If you want to submit several packages, please send a separate PR + for each one, it's easier for us to track things that way. + +12.2. Committing: Importing a package into CVS + + This section is only of interest for NetBSD developers with write + access to the NetBSD pkgsrc repository. Please remember that cvs + imports files relative to the cwd, and that the pathname that you give + the cvs import command is so that it knows where to place the files in + the repository. Newly created packages should be imported with a + vendor tag of "TNF" and a release tag of "pkgsrc-base", e.g: +% cd +.../pkgsrc/<category>/<pkgname> +% cvs import pkgsrc/<category>/<pkgname> TNF pkgsrc-base + + Remember to move the directory from which you imported out of the way, + or cvs will complain the next time you "cvs update" your source tree. + Also don't forget to add the new package to the category's Makefile. + + The commit message of the initial import should include part of the + DESCR file, so people reading the mailing lists know what the package + is/does. + + Please note all package updates/additions in pkgsrc/doc/CHANGES. It's + very important to keep this file up to date and conforming to the + existing format, because it will be used by scripts to automatically + update pages on www.NetBSD.org and other sites. + + For new packages, "cvs import" is preferred to "cvs add" because the + former gets everything with a single command, and provides a + consistent tag. + +12.3. Updating a Package to a Newer Version + + Please always put a concise, appropriate and relevant summary of the + changes between old and new versions into the commit log when updating + a package. There are various reasons for this: + * A URL is volatile, and can change over time. It may go away + completely or its information may be overwritten by newer + information. + * Having the change information between old and new versions in our + CVS repository is very useful for people who use either cvs or + anoncvs. + * Having the change information between old and new versions in our + CVS repository is very useful for people who read the + pkgsrc-changes mailing list, so that they can make tactical + decisions about when to upgrade the package. + + Please also recognise that, just because a new version of a package + has been released, it should not automatically be upgraded in the CVS + repository. We prefer to be conservative in the packages that are + included in pkgsrc - development or beta packages are not really the + best thing for most places in which pkgsrc is used. Please use your + judgement about what should go into pkgsrc, and bear in mind that + stability is to be preferred above new and possibly untested features. + +12.4. Moving a Package in pkgsrc + + 1. Make a copy of the directory somewhere else. + 2. Remove all CVS dirs. + Alternatively to the first two steps you can also do: + +% cvs -d user@cvs.NetBSD.org:/cvsroot export -D today pkgsrc/category/package + + and use that for further work. + 3. Fix CATEGORIES and any DEPENDS paths that just did "../package" + instead of "../../category/package". + 4. cvs import the modified package in the new place. + 5. Check if any package depends on it: + +% cd /usr/pkgsrc +% grep /package */*/Makefile* */*/buildlink* + + 6. Fix paths in packages from step 5 to point to new location. + 7. cvs rm (-f) the package at the old location. + 8. Remove from oldcategory/Makefile. + 9. Add to newcategory/Makefile. + 10. Commit the changed and removed files: + +% cvs commit oldcategory/package oldcategory/Makefile newcategory/Makefile + + (and any packages from step 5, of course). + +Chapter 13. A simple example of a package: bison + + Table of Contents + + 13.1. files + 13.2. Steps for building, installing, packaging + + I checked to find a piece of software that wasn't in the packages + collection, and picked GNU bison. Quite why someone would want to have + bison when Berkeley yacc is already present in the tree is beyond me, + but it's useful for the purposes of this exercise. + +13.1. files + +13.1.1. Makefile + +# $NetBSD: pkgsrc.txt,v 1.1 2003/06/23 07:41:44 grant Exp $ +# + +DISTNAME= bison-1.25 +CATEGORIES= devel +MASTER_SITES= ${MASTER_SITE_GNU} + +MAINTAINER= thorpej@NetBSD.org +HOMEPAGE= http://www.gnu.org/software/bison/bison.html +COMMENT= GNU yacc clone + +GNU_CONFIGURE= yes +INFO_FILES= bison.info + +.include "../../mk/bsd.pkg.mk" + +13.1.2. DESCR + +GNU version of yacc. Can make re-entrant parsers, and numerous other +improvements. Why you would want this when Berkeley yacc(1) is part +of the NetBSD source tree is beyond me. + +13.1.3. PLIST + +@comment $NetBSD: pkgsrc.txt,v 1.1 2003/06/23 07:41:44 grant Exp $ +bin/bison +man/man1/bison.1.gz +info/bison.info +info/bison.info-1 +info/bison.info-2 +info/bison.info-3 +info/bison.info-4 +info/bison.info-5 +share/bison.simple +share/bison.hairy + +13.1.4. Checking a package with pkglint + + The NetBSD package system comes with pkgtools/pkglint which helps to + check the contents of these files. After installation it is quite easy + to use, just change to the directory of the package you wish to + examine and execute pkglint: +% pkglint +OK: checking ./DESCR. +OK: checking Makefile. +OK: checking distinfo. +OK: checking patches/patch-aa. +looks fine. + + Depending on the supplied command line arguments (see pkglint(1)) more + verbose checks will be performed. Use e.g. pkglint -v for a very + verbose check. + +13.2. Steps for building, installing, packaging + + Create the directory where the package lives, plus any auxiliary + directories: +# cd /usr/pkgsrc/lang +# mkdir bison +# cd bison +# mkdir patches + + Create Makefile, DESCR and PLIST (see Chapter 5, Package components - + files, directories and contents) then continue with fetching the + distfile: +# make fetch +>> bison-1.25.tar.gz doesn't seem to exist on this system. +>> Attempting to fetch from ftp://prep.ai.mit.edu/pub/gnu//. +Requesting ftp://prep.ai.mit.edu/pub/gnu//bison-1.25.tar.gz (via ftp://orpheus. +amdahl.com:80/) +ftp: Error retrieving file: 500 Internal error + +>> Attempting to fetch from ftp://wuarchive.wustl.edu/systems/gnu//. +Requesting ftp://wuarchive.wustl.edu/systems/gnu//bison-1.25.tar.gz (via ftp:// +orpheus.amdahl.com:80/) +ftp: Error retrieving file: 500 Internal error + +>> Attempting to fetch from ftp://ftp.freebsd.org/pub/FreeBSD/distfiles//. +Requesting ftp://ftp.freebsd.org/pub/FreeBSD/distfiles//bison-1.25.tar.gz (via +ftp://orpheus.amdahl.com:80/) +Successfully retrieved file. + + Generate the checksum of the distfile into distinfo: + # make makesum + + Now compile: +# make +>> Checksum OK for bison-1.25.tar.gz. +===> Extracting for bison-1.25 +===> Patching for bison-1.25 +===> Ignoring empty patch directory +===> Configuring for bison-1.25 +creating cache ./config.cache +checking for gcc... cc +checking whether we are using GNU C... yes +checking for a BSD compatible install... /usr/bin/install -c -o bin -g bin +checking how to run the C preprocessor... cc -E +checking for minix/config.h... no +checking for POSIXized ISC... no +checking whether cross-compiling... no +checking for ANSI C header files... yes +checking for string.h... yes +checking for stdlib.h... yes +checking for memory.h... yes +checking for working const... yes +checking for working alloca.h... no +checking for alloca... yes +checking for strerror... yes +updating cache ./config.cache +creating ./config.status +creating Makefile +===> Building for bison-1.25 +cc -c -DSTDC_HEADERS=1 -DHAVE_STRING_H=1 -DHAVE_STDLIB_H=1 -DHAVE_MEMORY_H=1 -D +HAVE_ALLOCA=1 -DHAVE_STRERROR=1 -I./../include -g LR0.c +cc -c -DSTDC_HEADERS=1 -DHAVE_STRING_H=1 -DHAVE_STDLIB_H=1 -DHAVE_MEMORY_H=1 -D +HAVE_ALLOCA=1 -DHAVE_STRERROR=1 -I./../include -g allocate.c +cc -c -DSTDC_HEADERS=1 -DHAVE_STRING_H=1 -DHAVE_STDLIB_H=1 -DHAVE_MEMORY_H=1 -D +HAVE_ALLOCA=1 -DHAVE_STRERROR=1 -I./../include -g closure.c +cc -c -DSTDC_HEADERS=1 -DHAVE_STRING_H=1 -DHAVE_STDLIB_H=1 -DHAVE_MEMORY_H=1 -D +HAVE_ALLOCA=1 -DHAVE_STRERROR=1 -I./../include -g conflicts.c +cc -c -DSTDC_HEADERS=1 -DHAVE_STRING_H=1 -DHAVE_STDLIB_H=1 -DHAVE_MEMORY_H=1 -D +HAVE_ALLOCA=1 -DHAVE_STRERROR=1 -I./../include -g derives.c +cc -c -DXPFILE=\"/usr/pkg/share/bison.simple\" -DXPFILE1=\"/usr/pkg/share/biso +n.hairy\" -DSTDC_HEADERS=1 -DHAVE_STRING_H=1 -DHAVE_STDLIB_H=1 -DHAVE_MEMORY_H= +1 -DHAVE_ALLOCA=1 -DHAVE_STRERROR=1 -g ./files.c +cc -c -DSTDC_HEADERS=1 -DHAVE_STRING_H=1 -DHAVE_STDLIB_H=1 -DHAVE_MEMORY_H=1 -D +HAVE_ALLOCA=1 -DHAVE_STRERROR=1 -I./../include -g getargs.c +cc -c -DSTDC_HEADERS=1 -DHAVE_STRING_H=1 -DHAVE_STDLIB_H=1 -DHAVE_MEMORY_H=1 -D +HAVE_ALLOCA=1 -DHAVE_STRERROR=1 -I./../include -g gram.c +cc -c -DSTDC_HEADERS=1 -DHAVE_STRING_H=1 -DHAVE_STDLIB_H=1 -DHAVE_MEMORY_H=1 -D +HAVE_ALLOCA=1 -DHAVE_STRERROR=1 -I./../include -g lalr.c +cc -c -DSTDC_HEADERS=1 -DHAVE_STRING_H=1 -DHAVE_STDLIB_H=1 -DHAVE_MEMORY_H=1 -D +HAVE_ALLOCA=1 -DHAVE_STRERROR=1 -I./../include -g lex.c +cc -c -DSTDC_HEADERS=1 -DHAVE_STRING_H=1 -DHAVE_STDLIB_H=1 -DHAVE_MEMORY_H=1 -D +HAVE_ALLOCA=1 -DHAVE_STRERROR=1 -I./../include -g main.c +cc -c -DSTDC_HEADERS=1 -DHAVE_STRING_H=1 -DHAVE_STDLIB_H=1 -DHAVE_MEMORY_H=1 -D +HAVE_ALLOCA=1 -DHAVE_STRERROR=1 -I./../include -g nullable.c +cc -c -DSTDC_HEADERS=1 -DHAVE_STRING_H=1 -DHAVE_STDLIB_H=1 -DHAVE_MEMORY_H=1 -D +HAVE_ALLOCA=1 -DHAVE_STRERROR=1 -I./../include -g output.c +cc -c -DSTDC_HEADERS=1 -DHAVE_STRING_H=1 -DHAVE_STDLIB_H=1 -DHAVE_MEMORY_H=1 -D +HAVE_ALLOCA=1 -DHAVE_STRERROR=1 -I./../include -g print.c +cc -c -DSTDC_HEADERS=1 -DHAVE_STRING_H=1 -DHAVE_STDLIB_H=1 -DHAVE_MEMORY_H=1 -D +HAVE_ALLOCA=1 -DHAVE_STRERROR=1 -I./../include -g reader.c +cc -c -DSTDC_HEADERS=1 -DHAVE_STRING_H=1 -DHAVE_STDLIB_H=1 -DHAVE_MEMORY_H=1 -D +HAVE_ALLOCA=1 -DHAVE_STRERROR=1 -I./../include -g reduce.c +cc -c -DSTDC_HEADERS=1 -DHAVE_STRING_H=1 -DHAVE_STDLIB_H=1 -DHAVE_MEMORY_H=1 -D +HAVE_ALLOCA=1 -DHAVE_STRERROR=1 -I./../include -g symtab.c +cc -c -DSTDC_HEADERS=1 -DHAVE_STRING_H=1 -DHAVE_STDLIB_H=1 -DHAVE_MEMORY_H=1 -D +HAVE_ALLOCA=1 -DHAVE_STRERROR=1 -I./../include -g warshall.c +cc -c -DSTDC_HEADERS=1 -DHAVE_STRING_H=1 -DHAVE_STDLIB_H=1 -DHAVE_MEMORY_H=1 -D +HAVE_ALLOCA=1 -DHAVE_STRERROR=1 -I./../include -g version.c +cc -c -DSTDC_HEADERS=1 -DHAVE_STRING_H=1 -DHAVE_STDLIB_H=1 -DHAVE_MEMORY_H=1 -D +HAVE_ALLOCA=1 -DHAVE_STRERROR=1 -I./../include -g getopt.c +cc -c -DSTDC_HEADERS=1 -DHAVE_STRING_H=1 -DHAVE_STDLIB_H=1 -DHAVE_MEMORY_H=1 -D +HAVE_ALLOCA=1 -DHAVE_STRERROR=1 -I./../include -g getopt1.c +cc -g -o bison LR0.o allocate.o closure.o conflicts.o derives.o files.o + getargs.o gram.o lalr.o lex.o main.o nullab +le.o output.o print.o reader.o reduce.o symtab.o warshall.o version.o getopt. +o getopt1.o +./files.c:240: warning: mktemp() possibly used unsafely, consider using mkstemp +() +rm -f bison.s1 +sed -e "/^#line/ s|bison|/usr/pkg/share/bison|" < ./bison.simple > bison.s1 + + Everything seems OK, so install the files: +# make install +>> Checksum OK for bison-1.25.tar.gz. +===> Installing for bison-1.25 +sh ./mkinstalldirs /usr/pkg/bin /usr/pkg/share /usr/pkg/info /usr/pkg/man/man1 +rm -f /usr/pkg/bin/bison +cd /usr/pkg/share; rm -f bison.simple bison.hairy +rm -f /usr/pkg/man/man1/bison.1 /usr/pkg/info/bison.info* +install -c -o bin -g bin -m 555 bison /usr/pkg/bin/bison +/usr/bin/install -c -o bin -g bin -m 644 bison.s1 /usr/pkg/share/bison.simple +/usr/bin/install -c -o bin -g bin -m 644 ./bison.hairy /usr/pkg/share/bison.hai +ry +cd .; for f in bison.info*; do /usr/bin/install -c -o bin -g bin -m 644 $f /us +r/pkg/info/$f; done +/usr/bin/install -c -o bin -g bin -m 644 ./bison.1 /usr/pkg/man/man1/bison.1 +===> Registering installation for bison-1.25 + + You can now use bison, and also - if you decide so - remove it with + pkg_delete bison. Should you decide that you want a binary package, do + this now: +# make package +>> Checksum OK for bison-1.25.tar.gz. +===> Building package for bison-1.25 +Creating package bison-1.25.tgz +Registering depends:. +Creating gzip'd tar ball in '/u/pkgsrc/lang/bison/bison-1.25.tgz' + + Now that you don't need the source and object files any more, clean + up: +# make clean +===> Cleaning for bison-1.25 + +Appendix A. Build logs + + Table of Contents + + A.1. Building top + A.2. Packaging top + +A.1. Building top + +# make +>> top-3.5beta5.tar.gz doesn't seem to exist on this system. +>> Attempting to fetch from ftp://ftp.groupsys.com/pub/top/. +Requesting ftp://ftp.groupsys.com/pub/top/top-3.5beta5.tar.gz (via ftp://orpheu +s.amdahl.com:80/) +Successfully retrieved file. +>> Checksum OK for top-3.5beta5.tar.gz. +===> Extracting for top-3.5beta5 +===> Patching for top-3.5beta5 +===> Applying NetBSD patches for top-3.5beta5 +===> Configuring for top-3.5beta5 +/bin/cp /u/pkgsrc/sysutils/top/files/defaults /u/pkgsrc/sysutils/top/work/top-3 +.5beta5/.defaults +chmod a-x /u/pkgsrc/sysutils/top/work/top-3.5beta5/install + +Reading configuration from last time... + +Using these settings: + Bourne Shell /bin/sh + C compiler cc + Compiler options -DHAVE_GETOPT -O + Awk command awk + Install command /usr/bin/install + + Module netbsd13 + LoadMax 5.0 + Default TOPN -1 + Nominal TOPN 18 + Default Delay 2 +Random passwd access yes + Table Size 47 + Owner root + Group Owner kmem + Mode 2755 + bin directory $(PREFIX)/bin + man directory $(PREFIX)/man/man1 + man extension 1 + man style man + +Building Makefile... +Building top.local.h... +Building top.1... +Doing a "make clean". +rm -f *.o top core core.* sigdesc.h +To create the executable, type "make". +To install the executable, type "make install". +===> Building for top-3.5beta5 +cc -DHAVE_GETOPT -DORDER -DHAVE_GETOPT -O -c top.c +awk -f sigconv.awk /usr/include/sys/signal.h >sigdesc.h +cc -DHAVE_GETOPT -DORDER -DHAVE_GETOPT -O -c commands.c +cc -DHAVE_GETOPT -DORDER -DHAVE_GETOPT -O -c display.c +cc -DHAVE_GETOPT -DORDER -DHAVE_GETOPT -O -c screen.c +cc -DHAVE_GETOPT -DORDER -DHAVE_GETOPT -O -c username.c +cc -DHAVE_GETOPT -DORDER -DHAVE_GETOPT -O -c utils.c +utils.c: In function `errmsg': +utils.c:348: warning: return discards `const' from pointer target type +cc -DHAVE_GETOPT -DORDER -DHAVE_GETOPT -O -c version.c +cc -DHAVE_GETOPT -DORDER -DHAVE_GETOPT -O -c getopt.c +cc "-DOSREV=12G" -DHAVE_GETOPT -DORDER -DHAVE_GETOPT -O -c machine.c +rm -f top +cc -o top top.o commands.o display.o screen.o username.o utils.o version.o get +opt.o machine.o -ltermcap -lm -lkvm +# +# make install +>> Checksum OK for top-3.5beta5.tar.gz. +===> Installing for top-3.5beta5 +/usr/bin/install -o root -m 2755 -g kmem top /usr/pkg/bin +/usr/bin/install top.1 /usr/pkg/man/man1/top.1 +strip /usr/pkg/bin/top +===> Registering installation for top-3.5beta5 + +A.2. Packaging top + +# make package +>> Checksum OK for top-3.5beta5.tar.gz. +===> Building package for top-3.5beta5 +Creating package top-3.5beta5.tgz +Registering depends:. +Creating gzip'd tar ball in '/u/pkgsrc/sysutils/top/top-3.5beta5.tgz' + +Appendix B. Layout of the FTP server's package archive + + Layout for precompiled binary packages on ftp.NetBSD.org: +/pub/NetBSD/packages/ + README + distfiles/ + pkgsrc -> /pub/NetBSD/NetBSD-current/pkgsrc + 1.6/ + i386/ + All/ + archivers/ + foo -> ../All/foo + ... + m68k/ + All/ + archivers/ + foo -> ../All/foo + ... + amiga -> m68k + atari -> m68k + ... |